Image category determination method and device

Abstract

The embodiments of the present specification provide an image category determination method and device, wherein the image category determination method includes receiving an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples; determining an initial image category set corresponding to the image samples based on the image labels, and constructing a category chart corresponding to the image samples based on the initial image category set; and determining a target image category corresponding to the image samples according to the category chart.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of machine learning technology, and in particular to a method for determining image categories. One or more embodiments of this specification also relate to a classification model training method, an image category determination device, a classification model training device, a computing device, and a computer-readable storage medium. Background Art

[0002] In recent years, there has been a wave of breakthroughs in the field of artificial intelligence, driven by advances in deep learning and the explosive growth of datasets. Along with this trend, large-scale classification involving an unusually large number of classes has become an important task.

[0003] But large-scale classification brings many new challenges, among which the computational difficulty in training may be the most prominent. Specifically, current classifiers usually adopt a deep network structure, which usually includes a series of transformation layers for feature extraction and a softmax layer that connects the feature representation with the response of each class. The parameter size of this softmax layer is proportional to the number of classes. When training such a classifier, for each small batch of samples, the responses of all relevant classes will be calculated by taking the dot product between the class-specific weights and the extracted features.

[0004] When there are a large number of classes, the above algorithm will face two major difficulties: first, the parameter size may exceed the memory capacity, especially when the network is trained on a GPU with limited capacity; second, the computational cost will increase significantly, even to a prohibitive level.

[0005] Therefore, there is an urgent need to provide a category determination method that can speed up the training of the softmax layer without losing training accuracy. Summary of the invention

[0006] In view of this, the present specification provides an image category determination method. One or more embodiments of the present specification also relate to a classification model training method, an image category determination device, a classification model training device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0007] According to a first aspect of an embodiment of this specification, a method for determining an image category is provided, including:

[0008] Receiving an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples;

[0009] Determine an initial image category set corresponding to the image sample based on the image label, and construct a category chart corresponding to the image sample based on the initial image category set;

[0010] The target image category corresponding to the image sample is determined according to the category chart.

[0011] According to a second aspect of an embodiment of this specification, a classification model training method is provided, including:

[0012] Receiving an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples;

[0013] Determine an initial image category set corresponding to the image sample based on the image label, and construct a category chart corresponding to the image sample based on the initial image category set;

[0014] Determining the target image category corresponding to the image sample according to the category chart;

[0015] The classification model is trained based on the image sample set and the target image category to obtain the classification model.

[0016] According to a third aspect of an embodiment of this specification, there is provided an image category determination device, including:

[0017] A first image sample receiving module is configured to receive an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples;

[0018] A first chart construction module is configured to determine an initial image category set corresponding to the image sample based on the image label, and to construct a category chart corresponding to the image sample based on the initial image category set;

[0019] The first category determination module is configured to determine the target image category corresponding to the image sample according to the category chart.

[0020] According to a fourth aspect of the embodiments of this specification, a classification model training device is provided, including:

[0021] A second image sample receiving module is configured to receive an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples;

[0022] A second chart construction module is configured to determine an initial image category set corresponding to the image sample based on the image label, and to construct a category chart corresponding to the image sample based on the initial image category set;

[0023] A second category determination module is configured to determine the target image category corresponding to the image sample according to the category chart;

[0024] The model training module is configured to train the classification model based on the image sample set and the target image category to obtain the classification model.

[0025] According to a fifth aspect of an embodiment of this specification, a computing device is provided, including:

[0026] Memory and processor;

[0027] The memory is used to store computer-executable instructions, and when the processor executes the computer-executable instructions, it implements the steps of the image category determination method, or implements the steps of the classification model training method.

[0028] According to the sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the image category determination method, or implement the steps of the classification model training method.

[0029] An embodiment of the present specification implements an image category determination method and device, wherein the image category determination method includes receiving an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples; determining an initial image category set corresponding to the image samples based on the image labels, and constructing a category chart corresponding to the image samples based on the initial image category set; determining a target image category corresponding to the image samples according to the category chart; the image category determination method constructs a category chart corresponding to the image samples through a normalized initial image category set, so that the image samples can effectively determine a small batch of target image categories that are close to the image samples based on the category chart, and adaptively adjusts the allocation of computing resources through this k-nearest neighbor classification algorithm, which can greatly improve the training performance of the model in large-scale classification and save computing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is an example diagram of a specific application scenario of a category determination method provided by an embodiment of this specification;

[0031] Figure 1a It is a schematic diagram of the architecture of a machine learning model specifically applied to a category determination method provided in one embodiment of this specification;

[0032] Figure 2 is a process flow chart of a category determination method provided by an embodiment of this specification;

[0033] Figure 3It is a structural diagram of a category determination method provided by an embodiment of the present specification, in which a ring structure is used to realize the transmission and similarity calculation between category vectors on different computing nodes;

[0034] Figure 4 is a processing flow chart of a classification model training method provided by an embodiment of this specification;

[0035] Figure 5 is a structural schematic diagram of a category determination device provided by an embodiment of this specification;

[0036] Figure 6 It is a structural diagram of a classification model training device provided by an embodiment of this specification;

[0037] Figure 7 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0038] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0039] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0040] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0041] First, the terms involved in one or more embodiments of this specification are explained.

[0042] kNN: k-NearestNeighbor classification algorithm, English abbreviation: kNN, English full name: k-NearestNeighbor,

[0043] FC layer: The fully connected layer in the deep learning network. In the general classification task model, the last layer is FClayer, and its number of output layers is consistent with the number of classifications.

[0044] Softmax: A loss used to solve classification tasks, usually placed in the last module of the deep learning network, after the FC layer.

[0045] CUDA: Program to compute inner products of vectors.

[0046] In this specification, an image category determination method is provided. One or more embodiments of this specification also involve a classification model training method, an image category determination device, a classification model training device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0047] See also Figure 1 , Figure 1 An example diagram showing a specific application scenario of an image category determination method provided by an embodiment of the present specification.

[0048] Figure 1 The application scenario includes a terminal and a server. Specifically, a user sends an image sample set to a server through a terminal, wherein the image sample set includes image samples and image labels. After receiving the image samples and image labels, the server obtains an initial image category set through the image labels, and then constructs a category chart corresponding to the image samples based on the initial image category set, and finally obtains a target image category corresponding to the image samples according to the category chart.

[0049] See also Figure 1a , Figure 1a A schematic diagram of the architecture of a machine learning model specifically applied to a category determination method provided in one or more embodiments of this specification.

[0050] Figure 1a The model architecture in includes input layer, feature extraction layer (i.e. convolutional layer), fully connected layer, softmax layer and output layer.

[0051] In a specific implementation, the input layer receives the image samples in the acquired image sample set and the image labels corresponding to the image samples, and then inputs the image samples into the feature extraction layer;

[0052] The feature extraction layer extracts features from the image samples to obtain image sample features of the image samples, and sends the image sample features and the image labels corresponding to the image sample features to the fully connected layer;

[0053] The fully connected layer determines the initial category set corresponding to the image sample features through the label, and performs a multi-class classification on the image sample features and the initial category in the initial category set ( Figure 1a The W in is L2 normalized, and a complete k-nearest neighbor Graph is constructed based on the result of the initial category normalization ( Figure 1a Then, by adding or deleting the categories in the Graph through the neighboring categories corresponding to the image sample features, we finally obtain the target category set corresponding to the image sample features ( Figure 1a The selectedW in , and the image sample features and the corresponding target category set are sent to the softmax layer;

[0054] The softmax layer receives the image sample features and the corresponding target category set transmitted by the fully connected layer, outputs multiple vectors with the same dimension as the target category, and then measures the error of the vector through the cross entropy loss function and outputs it; specifically, the value in the vector is the probability value of each image sample belonging to each target category, and the cross entropy loss function measures the quality of the probability obtained by the softmax layer. The lower the cross entropy, the better the probability value.

[0055] In practical applications, the machine learning model for the specific application of the category determination method is carried on a GPU-based deep learning software system, which includes at least one GPU.

[0056] pass Figure 1a It can be seen that the category determination method provided in the embodiments of this specification is mainly used in the fully connected layer of the machine learning model to obtain the target category corresponding to the training data. The specific implementation method can be found in the following embodiments.

[0057] See also Figure 2 , Figure 2 A flow chart of a category determination method provided according to an embodiment of the present specification is shown, comprising the following steps:

[0058] Step 202: Receive an image sample set.

[0059] The image sample set includes image samples and image labels corresponding to the image samples.

[0060] In practical applications, image samples include but are not limited to pictures, videos, etc.

[0061] In specific implementation, in order to reduce the amount of data processing and increase the processing speed of the processor, the image sample set will be transmitted to multiple GPUs for parallel processing, and the processing steps of the image samples allocated to it on each GPU are the same.

[0062] Specifically, the image sample set includes multiple image samples and an image label corresponding to each image sample, wherein the label is the thing to be predicted, such as the future price of the commodity shown in the image, the animal breed shown in the image, etc., and when the image sample contains animals, the corresponding image label can be understood as the animal breed shown in the image, etc.

[0063] For example, in the case where the image samples are images, receiving the image sample set may be understood as: receiving the image sample set including a plurality of image samples and an image label corresponding to each image sample.

[0064] Step 204: Determine an initial image category set corresponding to the image sample based on the image label, and construct a category chart corresponding to the image sample based on the initial image category set.

[0065] Specifically, after receiving the image sample set, the initial image category set corresponding to the image sample is determined based on the image label corresponding to each image sample; for example, the image labels corresponding to the image sample are tiger, wild goose, peach blossom, etc., then based on the label, it can be determined that the initial image categories corresponding to the image sample include the first category: tiger, the second category: wild goose, the third category: peach blossom, etc., and these categories form the initial image category set corresponding to the image sample.

[0066] In a specific implementation, constructing a category chart corresponding to the image sample based on the initial image category set includes:

[0067] Preprocessing the initial image categories in the initial image category set to obtain an image category vector set of the initial image categories;

[0068] A category chart corresponding to the image sample is constructed according to the image category vector set.

[0069] Specifically, preprocessing the initial image categories in the initial image category set can be understood as performing L2 normalization on the initial image categories in the initial image category set to obtain a category vector for each initial image category, and the category vectors of all initial image categories constitute an image category vector set.

[0070] L2 normalization of the initial image categories is to vectorize them according to the characteristics of each initial image category to obtain a category vector of the same dimension for each initial image category in the image category vector set. The category vector obtained after L2 normalization can enable the machine learning model to converge quickly during training without reducing the accuracy. After L2 norm normalization, the Euclidean distance between each two initial image categories can be calculated.

[0071] In a specific implementation, constructing a category chart corresponding to the image sample according to the image category vector set includes:

[0072] Determine k nearest neighbor category vectors of each category vector in the image category vector set by linear search, where k is a positive integer;

[0073] A category chart corresponding to the image sample is constructed based on k neighboring category vectors of each category vector in the image category vector set.

[0074] Among them, linear search is to calculate each category vector in the image category vector set and other category vectors. For example, there are 1000 category vectors in the image category vector set. Each category vector and other 999 category vectors are calculated in turn to determine their respective k nearest neighbor category vectors.

[0075] Specifically, k can be set according to actual applications, without any limitation herein, for example, it can be set to 10, 12 or 15, etc.

[0076] In practical applications, the k nearest neighbor category vectors of each category vector in the image category vector set are determined by linear search, which can be understood as calculating the k nearest neighbor category vectors of each category vector in the image category vector set by linear search, and then constructing a category chart corresponding to the image sample based on the k nearest neighbor category vectors of each category vector in the image category vector set; using linear search to search for the k nearest neighbor category vectors of each category vector can avoid the loss of the nearest neighbor category vectors of the category vector, so as to ensure the accuracy of the nearest neighbor category vector of each category vector found.

[0077] Since constructing a category chart corresponding to an image sample is a relatively time-consuming process, the embodiment of this specification implements the composition through several iterations, and does not occupy additional GPU computing resources, and reuses the GPU during training to implement the construction of the category chart. The specific implementation method is as follows:

[0078] The step of determining k nearest neighbor category vectors of each category vector in the image category vector set by linear search includes:

[0079] Allocating the category vectors in the image category vector set to different computing nodes based on a preset allocation method;

[0080] The similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes are calculated in turn to obtain k nearest neighbor category vectors of each category vector in the image category vector set.

[0081] Among them, the preset allocation method can be set according to actual needs, such as equal allocation or allocation by quantity, etc. For ease of understanding, in the embodiments of this specification, the preset allocation method is introduced by taking equal allocation as an example.

[0082] In the specific implementation, the category vectors in the image category vector set are first evenly distributed to different computing nodes in an average distribution manner; then the similarities between the category vectors allocated on each computing node and the category vectors allocated on other computing nodes are calculated in turn, and then the k nearest neighbor category vectors corresponding to each category vector in the image category vector set are determined based on the similarity; wherein different computing nodes can be understood as different GPUs, that is, the above-mentioned GPU that can realize processing of the allocated image samples, at this time, the machine learning model training process on the GPU will be paused to reuse the GPU during training to realize the construction of the category graph, so as to avoid occupying additional GPU computing resources.

[0083] In actual application, since the category vectors are distributed and stored on different computing nodes, a ring structure can be used to realize the transfer of category vectors between different computing nodes, which can avoid transferring all category vectors to one computing node and causing the phenomenon of video memory explosion. The specific implementation method is as follows:

[0084] The step of sequentially calculating the similarity between each category vector assigned to each computing node and the category vectors assigned to other computing nodes includes:

[0085] Calculate the Euclidean distance between each first category vector allocated on the i-th computing node, where i is a positive integer;

[0086] Sending the first category vector allocated on the i-th computing node to the i+1-th computing node, and calculating the Euclidean distance between each second category vector allocated on the i+1-th computing node and each first category vector;

[0087] receiving the third category vector allocated on the i-1th computing node, and calculating the Euclidean distance between each first category vector allocated on the i-th computing node and each third category vector;

[0088] Sending the third category vector allocated on the i-1th computing node to the i+1th computing node, and calculating the Euclidean distance between each second category vector allocated on the i+1th computing node and each third category vector;

[0089] Until the Euclidean distance between the category vector assigned to each computing node and the category vectors assigned to other computing nodes is calculated, the similarity between each category vector assigned to each computing node and the category vectors assigned to other computing nodes is determined by the Euclidean distance.

[0090] Specifically, i is regarded as one of all computing nodes, and one computing node is taken as an example to introduce in detail the calculation of k nearest neighbor category vectors of the category vector allocated to each computing node.

[0091] First, the i-th computing node will calculate the first category vector assigned to itself, and calculate the Euclidean distance between each category vector and other category vectors. For example, the first category vectors assigned to the i-th computing node include a1, a2, and a3. Then the Euclidean distance between each first category vector assigned to the i-th computing node is calculated, that is, the Euclidean distance between a1 and a2 and a3, the Euclidean distance between a2 and a1 and a3, and the Euclidean distance between a3 and a1 and a2.

[0092] Then, the first category vector allocated on the i-th computing node is sent to the i+1-th computing node, and the Euclidean distance between each second category vector allocated on the i+1-th computing node and each first category vector is calculated, wherein the i+1-th computing node is the next computing node of the i-th computing node; still taking the above as an example, if the second category vectors allocated on the i+1-th computing node include b1, b2, and b3, at this time, the first category vectors a1, a2, and a3 are sent to the i+1-th computing node, and the Euclidean distances with the second category vectors b1, b2, and b3 are calculated, that is, the Euclidean distances between a1 and a2, a3, b1, b2, and b3 are calculated respectively, and the Euclidean distances between a2 and a1, a3, b1, b2, and b3 are calculated respectively... until the Euclidean distances between each category vector in the i+1-th computing node and other category vectors are calculated;

[0093] At the same time, when the first category vector allocated on the i-th computing node is sent to the i+1-th computing node, the i-th computing node receives the third category vector allocated on the i-1-th computing node, and then calculates the Euclidean distance between each first category vector allocated on the i-th computing node and each third category vector; wherein the i-1-th computing node is the previous computing node of the i-th computing node; still taking the above as an example, if the third category vector allocated on the i-1-th computing node includes c1, c2, and c3, at this time, the third category vectors c1, c2, and c3 are sent to the i-th computing node, and the Euclidean distances with the first category vectors a1, a2, and a3 are calculated, that is, the Euclidean distances between a1 and a2, a3, c1, c2, and c3 are calculated respectively, and the Euclidean distances between a2 and a1, a3, c1, c2, and c3 are calculated respectively... until the Euclidean distances between each category vector in the i-th computing node and other category vectors are calculated;

[0094] Finally, the i-th computing node sends the third category vector allocated on the i-1-th computing node to the i+1-th computing node, and calculates the Euclidean distance between each second category vector allocated on the i+1 computing node and each third category vector; still taking the above as an example, the third category vectors c1, c2, c3 are sent to the i+1-th computing node, and the Euclidean distances between the third category vectors b1, b2, b3 are calculated;

[0095] Until the Euclidean distance between the category vector assigned to each computing node and the category vectors assigned to other computing nodes is calculated, the similarity between each category vector assigned to each computing node and the category vectors assigned to other computing nodes is determined by the Euclidean distance; that is, it can be understood that the smaller the Euclidean distance, the higher the similarity; in this case, the Euclidean distance between each category vector and other category vectors can be sorted in descending order, and then the first k category vectors are taken as the neighbor category vectors of each category vector.

[0096] In order to more clearly understand the use of a ring structure to achieve the transfer of category vectors between different computing nodes and the calculation of similarity, see Figure 3 , Figure 3 A schematic diagram of a structure in which a ring structure is used in a category determination method provided in one or more embodiments of this specification to implement transmission and similarity calculation between category vectors on different computing nodes.

[0097] Figure 3 In the example, each id corresponds to a computing node, that is, a GPU, and different category vectors are distributed and stored on each GPU.

[0098] Taking the category vector a stored on the GPU with id=1, the category vector b stored on the GPU with id=2, the category vector c stored on the GPU with id=3, the category vector d stored on the GPU with id=0, and the category vector f stored on the GPU with id=T-1 as examples, the transfer of category vectors on different computing nodes and the similarity calculation are introduced.

[0099] First, in the first iteration, each GPU will calculate the Euclidean distance between each category vector stored in itself. In the second iteration, the category vector a stored on the GPU with id=1 is passed to the GPU with id=2, and the category vector b stored on the GPU with id=2 is passed to the GPU with id=3... The category vector d stored on the GPU with id=0 is passed to the GPU with id=1, and then the Euclidean distance between the passed category vectors and the category vectors stored in each GPU is calculated. When each GPU transfers the category vector, the category vector stored in itself always exists. In the third iteration, the category vector d received on the GPU with id=1 is passed to The GPU with id=2 passes the category vector a received on the GPU with id=2 to the GPU with id=3... passes the category vector f received on the GPU with id=0 to the GPU with id=1, and then performs respective Euclidean distance calculations on the passed category vectors and the category vectors stored on each GPU... Through the above iterative process, the category vectors stored on each GPU will perform a Euclidean distance calculation with the category vectors stored on other GPUs, that is, the category vector on each computing node can perform a Euclidean distance calculation with the category vectors on all other computing nodes to ensure the accuracy of the neighbor vectors of each category vector and avoid the loss of neighbor vectors.

[0100] To put it simply, when a ring structure is used to implement category vector category transfer on different GPUs, each computing node copies its own category vector to produce an identical category vector, and then continues to pass the copied category vector through other computing nodes until it receives its own copied category vector. This means that the entire iteration process is complete and the transfer is finished.

[0101] In order to further speed up the efficiency of the composition, when calculating the k nearest category vectors of each category vector in the category vector set determined by the ring structure, the category vector will be converted to half precision, so as to use tensorcore to accelerate the matrix multiplication to calculate the Euclidean distance between each category vector. The specific implementation method is as follows:

[0102] Before sequentially calculating the similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes, the method further includes:

[0103] The category vector allocated to each computing node is converted to a first precision.

[0104] In practical applications, when training machine learning models, the standard precision is float32. In order to improve the computing speed, the precision of the category vector allocated to each computing node can be converted from standard precision to half precision, that is, float16.

[0105] In order to avoid losing precision after converting the category vector to half precision, the nearest category vector of each category vector is enlarged, and then the nearest vector of each enlarged category vector is calculated with full precision. The specific implementation is as follows:

[0106] The similarity between each category vector assigned to each computing node and the category vectors assigned to other computing nodes is calculated in sequence to obtain k nearest neighbor category vectors of each category vector in the image category vector set, including:

[0107] Based on the converted first precision, first similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes are calculated in sequence;

[0108] Obtain k*n nearest neighbor category vectors of each category vector in the image category vector set, where n is a positive number;

[0109] Convert each category vector and the corresponding k*n nearest neighbor category vectors to a second precision, and calculate a second similarity between each category vector and the corresponding k*n nearest neighbor category vectors based on the converted second precision;

[0110] Based on the second similarity, k nearest neighbor category vectors of each category vector in the image category vector set are determined.

[0111] Specifically, the above-mentioned ring structure is used to calculate the first similarity between each half-precision category vector assigned to each computing node and the half-precision category vectors assigned to other computing nodes, and then k*n neighbor category vectors of each half-precision category vector are obtained based on the first similarity, where n is set according to the actual application. For example, n can be 1.1, that is, k*1.1 neighbor category vectors of each half-precision category vector are obtained according to the first similarity. In this way, the neighbor category vectors of each half-precision category vector recalled can be enlarged.

[0112] Then, each half-precision category vector and the k*n nearest neighbor category vectors of each half-precision category vector are converted to full precision, i.e., float32. Then, the second similarity between each float32 category vector and the corresponding k*n nearest neighbor category vectors is calculated. Then, based on the second similarity, the k*n nearest neighbor category vectors are sorted in descending order, and the top k nearest neighbor category vectors of each category vector are taken as the final nearest neighbor category vectors.

[0113] In the embodiments of the present specification, in order to calculate the similarity of each category vector in the image category vector set, in order to determine the k nearest neighbor category vectors of each category vector, a brute force linear search is used to ensure the accuracy of the obtained k nearest neighbor category vectors of each category vector, and a ring structure is used to transfer the category vectors on different GPUs. Based on the efficient GPU link, the k nearest neighbor category vectors of each category vector are obtained, and the category graph can be constructed within a reasonable time range. In order not to occupy additional GPU computing resources, the GPU used during training will be reused during the category graph construction to realize the composition (the training will be paused at this time); in addition, since the category vectors have been L2 normalized, the Euclidean distance and inner product between the category vectors are equivalent, and the inner product calculation on CUDA is a matrix. Matrix multiplication, so the Euclidean distance can be simplified into a multiplication op, and the Euclidean distance between each category vector and other category vectors is calculated in this way; and in order to further accelerate the efficiency of composition, each category vector will be transformed into half precision during the specific calculation, so as to use tensorcore to accelerate matrix multiplication to calculate the Euclidean distance between each category vector and other category vectors, and in order not to lose precision, when recalling the nearest category vector of each category vector, the recalled k nearest category vectors will be enlarged first, and then each category vector and the enlarged recalled k nearest category vectors will be calculated by standard float32 to determine the final k nearest category vectors of each category vector. In this way, the composition time of the category chart can be accelerated by about 6 times without losing precision.

[0114] In practice, training cannot be performed while the category graph is being constructed, and the graph is constructed after each epoch (a training set of data). For example, on a 128-card V100 graphics card, it takes about 1.5 hours to construct a k-nearest neighbor category graph with 100 million categories.

[0115] In another embodiment of the present specification, after constructing the category chart corresponding to the image sample based on the k nearest category vectors of each category vector in the image category vector set, the method further includes:

[0116] Based on the image sample features of the image samples stored in each computing node, the category vectors in the category chart are filtered, and the filtered category charts are respectively stored in the corresponding computing nodes.

[0117] Specifically, a portion of image samples is allocated to each GPU, and then feature extraction is performed on the image samples through a feature extraction layer on each GPU to extract image sample features of the extracted image samples.

[0118] A portion of the category vectors are distributed and stored on each GPU, so the category vectors in the category chart can be screened based on the image sample features of the image samples stored in each computing node. The specific implementation method is as follows:

[0119] The screening of the category vectors in the category graph based on the image sample features stored in each computing node includes:

[0120] Determine the image label corresponding to the image sample feature stored in each computing node;

[0121] Determine a category vector corresponding to the image sample feature based on the label;

[0122] The category vector corresponding to the image sample feature is matched with the category vector in the category chart, and the category vector without a matching relationship in the category chart is deleted.

[0123] Specifically, first determine the image label corresponding to the image sample feature stored in each GPU, then determine the category vector corresponding to the label based on the label, then match the category vector corresponding to the label with the category vector in the category chart, and delete the category vector that does not have a matching relationship in the category chart.

[0124] For example, the image labels corresponding to the image sample features stored on computing node 1 are label 1 and label 2, the category vector corresponding to label 1 is category vector 1, and the category vector corresponding to label 2 is category vector 2. In the category chart, there are category vector 1 and its k nearest neighbor category vectors, category vector 2 and its k nearest neighbor category vectors, and category vector 3 and its k nearest neighbor category vectors. At this time, after matching the category vector corresponding to the label with the category vector in the category chart, it is determined that category vector 3 in the category chart has no matching relationship with both label 1 and label 2. At this time, category vector 3 and its k nearest neighbor category vectors are deleted; and the category chart with category vector 3 and its k nearest neighbor category vectors deleted is stored on computing node 1.

[0125] In addition, it is necessary to determine whether the k nearest neighbor category vectors of category vector 1 and the k nearest neighbor category vectors of category vector 2 are all stored on computing node 1. If not, the nearest neighbor category vectors that do not exist on computing node 1 are deleted.

[0126] In practical applications, the above method is used to compress the category chart stored on each computing node, so as to store the category chart without occupying a large video memory space.

[0127] Step 206: Determine the target image category corresponding to the image sample according to the category chart.

[0128] Specifically, since the category chart stores the k nearest neighbor category vectors corresponding to each category vector, and each image sample can determine its corresponding category vector through the corresponding image label, and then find the corresponding k nearest neighbor category vectors based on the category vector; therefore, all the nearest neighbor category vectors corresponding to all image samples can be found based on the category chart.

[0129] After compressing the category graph, the K nearest category vectors of each category vector in the category graph are no longer the same. In order to ensure that the training data can still quickly obtain the corresponding target image category based on the category graph, the following method is used:

[0130] Determining the target image category corresponding to the image sample based on the category chart includes:

[0131] The target image category corresponding to the image sample is determined in the category chart based on a preset function.

[0132] In the specific implementation, although the process of building the category graph is completely implemented on the GPU, there is still a problem in implementing training completely on the GPU: the storage of the Graph (i.e., the category graph). In the case of hundreds of millions of categories, let K = 1000, and the Graph size is 100 million * 1000, about 372G, and each node (i.e., GPU) needs to query the complete Graph to select active classes (i.e., the target training category of the image sample) during training. In other words, each node needs to have a complete Graph, and the complete Graph requires a lot of memory and cannot be stored in each node.

[0133] In order to fully utilize the GPU to implement machine learning model training, there are two methods to load the Graph into the GPU memory (for example, 32G V100 memory):

[0134] 1) Graph compression: Although a complete graph is required on each node, since the w (i.e., the category vector) on each node is local, it means that the w that is not on a certain node cannot be selected by the mini-batch (image sample) on that node. Therefore, the graph on the node can delete the w that is not on the node. Assuming 128 cards, on average, it can be compressed to 372G / 128=2.9G;

[0135] 2) Fast access: After the Graph is compressed, the K (i.e., k nearest category vectors) of each W in the Graph is no longer the same. For example, a 100 million*K two-dimensional tensor is turned into a one-dimensional tensor with continuous storage. At this time, there will be a problem: the mini-batch cannot quickly obtain W_active (i.e., the target training category) based on the Graph. In order to solve this problem, a kernel function can be added to the bottom layer of pytorch to complete the fast access to the compressed Graph: that is, first count the K value of each W after compression, and use another tensor to accumulate the K value before the corresponding position W (the accumulated result is the offset of the W in the compressed Graph). During the training process, different threads in the kernel are used to concurrently find the offsets of all x (training data features) in the mini-batch in the Graph, so as to quickly get the activeclass.

[0136] The neighbor category vectors corresponding to different image samples may be repeated, and in order to ensure the training speed, the number of target training categories in the target image category set of the image samples will be limited. The specific implementation method is as follows:

[0137] Determining the target image category corresponding to the image sample based on the category chart includes:

[0138] Determining a true category of the image sample based on an image label corresponding to the image sample;

[0139] Based on a preset function, a target category corresponding to a category vector of a real category neighbor of the image sample is determined in the category chart, and a corresponding target image category is determined based on the target category.

[0140] Specifically, the image label corresponding to the image sample, that is, the true category of the image sample, is used to quickly access the compressed Graph based on the kernel function, determine the target category corresponding to the category vector of the true category neighbor of all image samples, and then determine the corresponding target image category based on the target category.

[0141] In a specific implementation, determining the corresponding target image category based on the target category includes:

[0142] Deduplication of the target category;

[0143] Determine whether the number of target categories after deduplication is greater than or equal to the preset number threshold.

[0144] If yes, then a first preset number of deduplicated target categories are deleted to form a target image category,

[0145] If not, a second preset number of target categories corresponding to category vectors that are not adjacent to the true category of the image sample are selected from the category chart and added to the target category after deduplication to form a target image category.

[0146] The preset quantity threshold may be set according to actual application and is not limited here. For example, the preset quantity threshold may be 1,000 or 10,000.

[0147] In practical applications, the kernel function is used to quickly access the compressed Graph to determine the target categories corresponding to the category vectors of the true category neighbors of all image samples. The target category vectors corresponding to the category vectors of the true category neighbors of different image samples may be repeated. For example, the target categories corresponding to the category vectors of the true category neighbors of image sample 1 are category 1 and category 2, while the target categories corresponding to the category vectors of the true category neighbors of image sample 2 are category 1 and category 3. Therefore, the target category 1 of image sample 1 and image sample 2 is repeated. At this time, the target category can be deduplicated by deleting category 1.

[0148] In practical applications, in order to ensure the training results, the number of target training categories corresponding to a batch of image samples is limited, for example, the number of target training categories needs to be 1,000.

[0149] At this time, after deduplicating the target categories, it is necessary to determine whether the number of target categories after deduplication is greater than or equal to 1000. If so, some target categories are randomly deleted so that the number of target training categories is 1000 to form target image categories. Specifically, the first preset number is the number of target categories greater than the preset number threshold. If not, target categories corresponding to the true categories of the image samples and category vectors that do not have a neighboring relationship are selected from the category chart and added to the deduplicated target categories, so that the number of target training categories is 1000 to form target image categories. Specifically, the second preset number is obtained by subtracting the current number of target categories from the preset number threshold.

[0150] In specific implementation, the category determination method provided in the embodiment of this specification includes: first, during the training process, L2 normalize x (image sample features extracted from the mini-batch) and W, and then construct a complete k-nearest neighbor Graph based on all W (after L2 normalization), such as Graph(N*K). Based on the Graph, the Active Classes of the mini-batch can be quickly obtained through the labels of the image samples in the mini-batch: W_active = [list_y(1); :::; list_y(m)], where list_y(j) is the K-nearest neighbor of w_y(j). Since w has been normalized, the w corresponding to the data label must be the first in the corresponding row of the Graph, so it will definitely be selected. Then, after deduplicating W_active, compare the relationship between the remaining W and M (the number of active classes) after deduplication, and withdraw more and supplement less to obtain the target image category of the mini-batch. By adaptively adjusting the allocation of computing resources in this way, the training performance of machine learning models in large-scale classification can be greatly improved and computing costs can be saved.

[0151] In another embodiment of the present specification, before receiving the image sample set, the method further includes:

[0152] Displaying an image sample input interface to the user based on a call request from the user;

[0153] Receiving an image sample set input by the user based on the image sample input interface. Or

[0154] Before receiving the image sample set, the method further includes:

[0155] A calling request sent by a user is received, wherein the calling request carries an image sample set.

[0156] It can be seen from this that the image category determination method provided in the embodiment of this specification can also be provided to other callers as an image category determination service. When the caller does not have professional usage capabilities, the method can be provided to the caller in the form of interface input. When the caller has professional usage capabilities, the method can be directly called by the caller based on the caller's call request. The method provides different interfaces for the caller according to its capabilities, which greatly enhances the user experience.

[0157] The category determination method provided in one or more embodiments of the present specification is implemented based on the application in the machine learning model. When there are a large number of categories in the machine learning model, the active class selection algorithm based on the kNN Graph, the distributed GPU graph composition algorithm and the Graph storage access optimization algorithm can be used to enable the machine learning model to be trained on a GPU with limited capacity without increasing the computing cost. In addition, the category determination method is applied to the softmax of kNN, which is implemented through the full GPU with high execution efficiency, and through L2 normalization and lossless Graph construction, it achieves almost the same accuracy as the original softmax on the scale of 1M, 10M, and 100M, see Table 1.

[0158] Table 1

[0159] #methods 1M 10M 100M Selective Softmax 86.39％ 79.02％ 71.98％ MACH 80.11％ 71.34％ 59.82％ KNN Softmax 87.46％ 80.99％ 74.54％ Full Softmax 87.43％ 81.01％ 74.52％

[0160] See also Figure 4 , Figure 4 A flow chart of a classification model training method provided according to an embodiment of the present specification is shown, comprising the following steps:

[0161] Step 402: Receive an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples.

[0162] Step 404: Determine an initial image category set corresponding to the image sample based on the image label, and construct a category chart corresponding to the image sample based on the initial image category set.

[0163] Step 406: Determine the target image category corresponding to the image sample according to the category chart.

[0164] Step 408: Training a classification model based on the image sample set and the target image category to obtain the classification model.

[0165] In the embodiment of the present specification, the target image category corresponding to the picture sample is first determined based on steps 402 to 408, and then the classification model is quickly trained based on the feature vector of the picture sample in the picture sample and the obtained target image category, and the accuracy of the classification model can be guaranteed.

[0166] The process of determining the target image category corresponding to the image sample according to steps 402 to 408 can be referred to the above embodiment and will not be described in detail here.

[0167] In an embodiment of the present specification, after the target image category of the image sample set is obtained based on the above-mentioned image category determination method, the image sample set and the corresponding target image category can be cached in the classification model; specifically, a unique identifier is set for each image sample in the image sample set in the classification model, and then each image sample carrying the unique identifier and the target image category corresponding to each image sample are cached in the cache created for the classification model. When the image sample is subsequently used for model training, the model training can be implemented directly based on the image sample and the corresponding target image category, without the need to screen the target image category of the image sample through the above-mentioned image category method, thereby saving model training time and improving user experience.

[0168] In an embodiment of the present specification, after receiving a picture sample set, the classification model training method determines a corresponding initial image category set through image labels corresponding to the picture samples, then constructs a category chart corresponding to the picture samples through the normalized initial image category set, and then determines target image categories corresponding to the picture samples, whose number is less than the initial image category set, based on the category chart. Finally, the classification model is trained based on the picture sample set and the target image categories, so that the trained classification model can be obtained quickly and accurately.

[0169] Corresponding to the above method embodiment, this specification also provides an image category determination device embodiment, Figure 5 A schematic diagram of the structure of an image category determination device provided by an embodiment of the present specification is shown.

[0170] like Figure 5 As shown, the device comprises:

[0171] A first image sample receiving module 1102 is configured to receive an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples;

[0172] A first chart construction module 1104 is configured to determine an initial image category set corresponding to the image sample based on the image label, and to construct a category chart corresponding to the image sample based on the initial image category set;

[0173] The first category determination module 1106 is configured to determine the target image category corresponding to the image sample according to the category chart.

[0174] Optionally, the first chart construction module 1104 is further configured to:

[0175] Preprocessing the initial image categories in the initial image category set to obtain an image category vector set of the initial image categories;

[0176] A category chart corresponding to the image sample is constructed according to the image category vector set.

[0177] Optionally, the first chart construction module 1104 is further configured to:

[0178] Determine k nearest neighbor category vectors of each category vector in the image category vector set by linear search, where k is a positive integer;

[0179] A category chart corresponding to the image sample is constructed based on k neighboring category vectors of each category vector in the image category vector set.

[0180] Optionally, the first chart construction module 1104 is further configured to:

[0181] Allocating the category vectors in the image category vector set to different computing nodes based on a preset allocation method;

[0182] The similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes are calculated in turn to obtain k nearest neighbor category vectors of each category vector in the image category vector set.

[0183] Optionally, the first chart construction module 1104 is further configured to:

[0184] Calculate the Euclidean distance between each first category vector allocated on the i-th computing node, where i is a positive integer;

[0185] Sending the first category vector allocated on the i-th computing node to the i+1-th computing node, and calculating the Euclidean distance between each second category vector allocated on the i+1-th computing node and each first category vector;

[0186] receiving the third category vector allocated on the i-1th computing node, and calculating the Euclidean distance between each first category vector allocated on the i-th computing node and each third category vector;

[0187] Sending the third category vector allocated on the i-1th computing node to the i+1th computing node, and calculating the Euclidean distance between each second category vector allocated on the i+1th computing node and each third category vector;

[0188] Until the Euclidean distance between the category vector assigned to each computing node and the category vectors assigned to other computing nodes is calculated, the similarity between each category vector assigned to each computing node and the category vectors assigned to other computing nodes is determined by the Euclidean distance.

[0189] Optionally, the device further includes:

[0190] The precision conversion module is configured to perform a first precision conversion on the category vector allocated to each computing node.

[0191] Optionally, the first chart construction module 1104 is further configured to:

[0192] Based on the converted first precision, first similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes are calculated in sequence;

[0193] Obtain k*n nearest neighbor category vectors of each category vector in the image category vector set, where n is a positive number;

[0194] Convert each category vector and the corresponding k*n nearest neighbor category vectors to a second precision, and calculate a second similarity between each category vector and the corresponding k*n nearest neighbor category vectors based on the converted second precision;

[0195] Based on the second similarity, k nearest neighbor category vectors of each category vector in the image category vector set are determined.

[0196] Optionally, the device further includes:

[0197] The screening module is configured to screen the category vectors in the category chart based on the image sample features of the image samples stored in each computing node, and store the screened category charts in the corresponding computing nodes respectively.

[0198] Optionally, the screening module is further configured to:

[0199] Determine the image label corresponding to the image sample feature stored in each computing node;

[0200] Determine a category vector corresponding to the image sample feature based on the label;

[0201] The category vector corresponding to the image sample feature is matched with the category vector in the category chart, and the category vector without a matching relationship in the category chart is deleted.

[0202] Optionally, the first category determination module 1106 is further configured to:

[0203] The target image category corresponding to the image sample is determined in the category chart based on a preset function.

[0204] Optionally, the first category determination module 1106 is further configured to:

[0205] Determining a true category of the image sample based on an image label corresponding to the image sample;

[0206] Based on a preset function, a target category corresponding to a category vector of a real category neighbor of the image sample is determined in the category chart, and a corresponding target image category is determined based on the target category.

[0207] Optionally, the first category determination module 1106 is further configured to:

[0208] Deduplication of the target category;

[0209] Determine whether the number of target categories after deduplication is greater than or equal to the preset number threshold.

[0210] If yes, then a first preset number of deduplicated target categories are deleted to form a target image category,

[0211] If not, a second preset number of target categories are selected from the target categories after duplicate removal and added to the target categories after duplicate removal to form a target image category.

[0212] Optionally, the device further includes:

[0213] An interface display module, configured to display an image sample input interface to the user based on a user's call request;

[0214] The sample receiving module is configured to receive an image sample set input by the user based on the image sample input interface.

[0215] Optionally, the device further includes:

[0216] The calling request receiving module is configured to receive a calling request sent by a user, wherein the calling request carries an image sample set.

[0217] The above is a schematic scheme of an image category determination device of this embodiment. It should be noted that the technical scheme of the image category determination device and the technical scheme of the above-mentioned image category determination method belong to the same concept, and the details not described in detail in the technical scheme of the image category determination device can be referred to the description of the technical scheme of the above-mentioned image category determination method.

[0218] Corresponding to the above method embodiment, this specification also provides a classification model training device embodiment, Figure 6 A structural schematic diagram of a classification model training device provided by an embodiment of this specification is shown.

[0219] like Figure 6 As shown, the device comprises:

[0220] The second image sample receiving module 1202 is configured to receive an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples;

[0221] A second chart construction module 1203 is configured to determine an initial image category set corresponding to the image sample based on the image label, and to construct a category chart corresponding to the image sample based on the initial image category set;

[0222] A second category determination module 1204 is configured to determine the target image category corresponding to the image sample according to the category chart;

[0223] The model training module 1206 is configured to train the classification model based on the image sample set and the target image category to obtain the classification model.

[0224] The above is a schematic scheme of a classification model training device of this embodiment. It should be noted that the technical scheme of the classification model training device and the technical scheme of the classification model training method described above belong to the same concept, and the details not described in detail in the technical scheme of the classification model training device can be found in the description of the technical scheme of the classification model training method described above.

[0225] Figure 7 The block diagram of a computing device 1700 according to one embodiment of the present specification is shown. The components of the computing device 1700 include but are not limited to a memory 1710 and a processor 1720. The processor 1720 is connected to the memory 1710 via a bus 1730, and the database 1750 is used to store data.

[0226] The computing device 1700 also includes an access device 1740 that enables the computing device 1700 to communicate via one or more networks 1760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0227] In one embodiment of the present specification, the above components of the computing device 1700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0228] Computing device 1700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 1700 may also be a mobile or stationary server.

[0229] Among them, the processor 1720 is used to implement the steps of the category determination method when executing computer-executable instructions, or to implement the steps of the classification model training method.

[0230] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned image category determination method or classification model training method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned image category determination method or classification model training method.

[0231] An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the image category determination method or the steps of the classification model training method.

[0232] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned image category determination method or classification model training method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned image category determination method or classification model training method.

[0233] Another embodiment of the present specification further provides a category determination method, including:

[0234] Receive a training sample data set, wherein the training sample data set includes training sample data and labels corresponding to the training sample data;

[0235] Determine an initial training category set corresponding to the training sample data based on the label;

[0236] Preprocessing the initial training categories in the initial training category set to obtain a category vector set of the initial training categories;

[0237] Constructing a category chart corresponding to the training sample data according to the category vector set;

[0238] A target training category set corresponding to the training sample data is determined based on the category chart.

[0239] Optionally, constructing a category chart corresponding to the training sample data according to the category vector set includes:

[0240] Determine k nearest neighbor category vectors of each category vector in the category vector set by linear search, where k is a positive integer;

[0241] A category graph corresponding to the training sample data is constructed based on k nearest neighbor category vectors of each category vector in the category vector set.

[0242] Optionally, determining k nearest neighbor category vectors of each category vector in the category vector set by linear search includes:

[0243] Allocating the category vectors in the category vector set to different computing nodes based on a preset allocation method;

[0244] The similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes are calculated in turn to obtain k nearest neighbor category vectors of each category vector in the category vector set.

[0245] Optionally, the sequentially calculating the similarities between each category vector allocated on each computing node and the category vectors allocated on other computing nodes includes:

[0246] Calculate the Euclidean distance between each first category vector allocated on the i-th computing node, where i is a positive integer;

[0247] Sending the first category vector allocated on the i-th computing node to the i+1-th computing node, and calculating the Euclidean distance between each second category vector allocated on the i+1-th computing node and each first category vector;

[0248] receiving the third category vector allocated on the i-1th computing node, and calculating the Euclidean distance between each first category vector allocated on the i-th computing node and each third category vector;

[0249] Sending the third category vector allocated on the i-1th computing node to the i+1th computing node, and calculating the Euclidean distance between each second category vector allocated on the i+1th computing node and each third category vector;

[0250] Until the Euclidean distance between the category vector assigned to each computing node and the category vectors assigned to other computing nodes is calculated, the similarity between each category vector assigned to each computing node and the category vectors assigned to other computing nodes is determined by the Euclidean distance.

[0251] Optionally, before sequentially calculating the similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes, the method further includes:

[0252] The category vector allocated to each computing node is converted to a first precision.

[0253] Optionally, the step of sequentially calculating the similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes to obtain k nearest neighbor category vectors of each category vector in the category vector set includes:

[0254] Based on the converted first precision, first similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes are calculated in sequence;

[0255] Obtain k*n nearest neighbor category vectors of each category vector in the category vector set, where n is a positive number;

[0256] Convert each category vector and the corresponding k*n nearest neighbor category vectors to a second precision, and calculate a second similarity between each category vector and the corresponding k*n nearest neighbor category vectors based on the converted second precision;

[0257] Based on the second similarity, k nearest neighbor category vectors of each category vector in the category vector set are determined.

[0258] Optionally, after constructing the category chart corresponding to the training sample data based on the k nearest category vectors of each category vector in the category vector set, the method further includes:

[0259] Based on the training sample features of the training sample data stored in each computing node, the category vectors in the category chart are filtered, and the filtered category charts are respectively stored in the corresponding computing nodes.

[0260] Optionally, the screening of the category vectors in the category graph based on the training sample features stored in each computing node includes:

[0261] Determine the label corresponding to the training sample feature stored in each computing node;

[0262] Determine a category vector corresponding to the training sample feature based on the label;

[0263] The category vector corresponding to the training sample feature is matched with the category vector in the category chart, and the category vector without a matching relationship in the category chart is deleted.

[0264] Optionally, determining the target training category set corresponding to the training sample data based on the category chart includes:

[0265] A target training category set corresponding to the training sample data is determined in the category chart based on a preset function.

[0266] Optionally, determining the target training category set corresponding to the training sample data based on the category chart includes:

[0267] Determine the true category of the training sample data based on the label corresponding to the training sample data;

[0268] Based on a preset function, a target category corresponding to a category vector of a real category neighbor of the training sample data is determined in the category diagram, and a corresponding target training category set is determined based on the target category.

[0269] Optionally, determining a corresponding target training category set based on the target category includes:

[0270] Deduplication of the target category;

[0271] Determine whether the number of target categories after deduplication is greater than or equal to the preset number threshold.

[0272] If yes, then a first preset number of target categories after deduplication are deleted to form a target training category set,

[0273] If not, a second preset number of target categories are selected from the target categories after duplicate removal, and added to the target categories after duplicate removal to form a target training category set.

[0274] Optionally, the training sample data includes images.

[0275] The category determination method provided in the embodiments of this specification can be applied to any large-scale classification field. The category determination method can reduce the number of training samples and speed up model training while keeping the accuracy of training samples unchanged. The training samples include but are not limited to images, videos or commodities, for example: when the training sample is an image, the target image category of the image can be obtained through the category determination method; when the training sample is a video, the target video label category of the video can be obtained through the category determination method; when the training sample is a commodity, the target commodity label category of the commodity can be obtained through the category determination method; subsequently, a small amount of accurate target training categories obtained by this method can be used to quickly and accurately train machine learning models such as classification models or recognition models, so that the trained machine learning models can be better used in specific applications.

[0276] An embodiment of the present specification implements a category determination method including receiving a training sample data set; determining an initial training category set corresponding to the training sample data based on the label corresponding to the training sample data in the training sample data set; preprocessing the initial training categories in the initial training category set to obtain a category vector set of the initial training categories; constructing a category chart corresponding to the training sample data based on the category vector set; determining a target training category set corresponding to the training sample data based on the category chart; the category determination method constructs a category chart corresponding to the training sample data through a normalized category vector set, so that the training sample data can effectively determine a small batch target training category set that is close to the training sample data based on the category chart, and adaptively adjusts the allocation of computing resources through this k-nearest neighbor classification algorithm, which can greatly improve the training performance of the model in large-scale classification and save computing costs.

[0277] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0278] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0279] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0280] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0281] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A method for determining an image category, comprising: Receiving an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples; Determine an initial image category set corresponding to the image sample based on the image label, and construct a category chart corresponding to the image sample based on the initial image category set; Determining the target image category corresponding to the image sample in the category chart based on a preset function; Among them, constructing the category chart corresponding to the image sample based on the initial image category set includes: vectorizing the features of the initial image category in the initial image category set to obtain an image category vector set of the initial image category, wherein the image category vector set includes neighboring category vectors with the same dimension as the initial image category; and constructing the category chart corresponding to the image sample according to the image category vector set.

2. According to the method of claim 1, constructing a category chart corresponding to the image sample according to the image category vector set comprises: Determine k nearest neighbor category vectors of each category vector in the image category vector set by linear search, where k is a positive integer; A category chart corresponding to the image sample is constructed based on k neighboring category vectors of each category vector in the image category vector set.

3. According to the method of claim 2, the step of determining k nearest neighbor category vectors of each category vector in the image category vector set by linear search comprises: Allocating the category vectors in the image category vector set to different computing nodes based on a preset allocation method; The similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes are calculated in turn to obtain k nearest neighbor category vectors of each category vector in the image category vector set.

4. According to the method of claim 3, the step of sequentially calculating the similarity between each category vector assigned to each computing node and the category vectors assigned to other computing nodes comprises: Calculate the Euclidean distance between each first category vector allocated on the i-th computing node, where i is a positive integer; Sending the first category vector allocated on the i-th computing node to the i+1-th computing node, and calculating the Euclidean distance between each second category vector allocated on the i+1-th computing node and each first category vector; receiving the third category vector allocated on the i-1th computing node, and calculating the Euclidean distance between each first category vector allocated on the i-th computing node and each third category vector; Sending the third category vector allocated on the i-1th computing node to the i+1th computing node, and calculating the Euclidean distance between each second category vector allocated on the i+1th computing node and each third category vector; Until the Euclidean distance between the category vector assigned to each computing node and the category vectors assigned to other computing nodes is calculated, the similarity between each category vector assigned to each computing node and the category vectors assigned to other computing nodes is determined by the Euclidean distance.

5. The method according to claim 3, before sequentially calculating the similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes, further comprising: The category vector allocated to each computing node is converted to a first precision.

6. According to the method of claim 5, the step of sequentially calculating the similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes to obtain k nearest neighbor category vectors of each category vector in the image category vector set comprises: Based on the converted first precision, first similarities between each category vector assigned to each computing node and the category vectors assigned to other computing nodes are calculated in sequence; Obtain k*n nearest neighbor category vectors of each category vector in the image category vector set, where n is a positive number; Convert each category vector and the corresponding k*n nearest neighbor category vectors to a second precision, and calculate a second similarity between each category vector and the corresponding k*n nearest neighbor category vectors based on the converted second precision; Based on the second similarity, k nearest neighbor category vectors of each category vector in the image category vector set are determined.

7. The method according to claim 3, after constructing the category chart corresponding to the image sample based on the k nearest category vectors of each category vector in the image category vector set, further comprising: Based on the image sample features of the image samples stored in each computing node, the category vectors in the category chart are filtered, and the filtered category charts are respectively stored in the corresponding computing nodes.

8. The method according to claim 7, wherein the filtering of the category vectors in the category graph based on the image sample features stored in each computing node comprises: Determine the image label corresponding to the image sample feature stored in each computing node; Determine a category vector corresponding to the image sample feature based on the label; The category vector corresponding to the image sample feature is matched with the category vector in the category chart, and the category vector without a matching relationship in the category chart is deleted.

9. The method according to claim 1, wherein determining the target image category corresponding to the image sample based on the category chart comprises: Determining a true category of the image sample based on an image label corresponding to the image sample; Based on a preset function, a target category corresponding to a category vector of a real category neighbor of the image sample is determined in the category chart, and a corresponding target image category is determined based on the target category.

10. The method according to claim 9, wherein determining the corresponding target image category based on the target category comprises: Deduplication of the target category; Determine whether the number of target categories after deduplication is greater than or equal to the preset number threshold. If yes, then a first preset number of deduplicated target categories are deleted to form a target image category, If not, a second preset number of target categories are selected from the target categories after duplicate removal and added to the target categories after duplicate removal to form a target image category.

11. The method for determining an image category according to claim 1, before receiving the image sample set, further comprising: Displaying an image sample input interface to the user based on a call request from the user; An image sample set input by the user based on the image sample input interface is received.

12. The method for determining an image category according to claim 1, before receiving the image sample set, further comprising: A calling request sent by a user is received, wherein the calling request carries an image sample set.

13. A classification model training method, comprising: Receiving an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples; Determine an initial image category set corresponding to the image sample based on the image label, and construct a category chart corresponding to the image sample based on the initial image category set; Determining the target image category corresponding to the image sample in the category chart based on a preset function; Training a classification model based on the image sample set and the target image category to obtain the classification model; Among them, constructing the category chart corresponding to the image sample based on the initial image category set includes: vectorizing the features of the initial image category in the initial image category set to obtain an image category vector set of the initial image category, wherein the image category vector set includes neighboring category vectors with the same dimension as the initial image category; and constructing the category chart corresponding to the image sample according to the image category vector set.

14. An image category determination device, comprising: A first image sample receiving module is configured to receive an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples; A first chart construction module is configured to determine an initial image category set corresponding to the image sample based on the image label, and to construct a category chart corresponding to the image sample based on the initial image category set; A first category determination module is configured to determine the target image category corresponding to the image sample in the category chart based on a preset function; Among them, the first chart construction module is further configured to vectorize the features of the initial image category in the initial image category set to obtain an image category vector set of the initial image category, wherein the image category vector set includes neighboring category vectors with the same dimension as the initial image category; and construct a category chart corresponding to the image sample according to the image category vector set.

15. A classification model training device, comprising: A second image sample receiving module is configured to receive an image sample set, wherein the image sample set includes image samples and image labels corresponding to the image samples; A second chart construction module is configured to determine an initial image category set corresponding to the image sample based on the image label, and to construct a category chart corresponding to the image sample based on the initial image category set; A second category determination module is configured to determine the target image category corresponding to the image sample in the category chart based on a preset function; A model training module is configured to train a classification model based on the image sample set and the target image category to obtain the classification model; Among them, the second chart construction module is further configured to vectorize the features of the initial image category in the initial image category set to obtain an image category vector set of the initial image category, wherein the image category vector set includes neighboring category vectors with the same dimension as the initial image category; and construct a category chart corresponding to the image sample according to the image category vector set.

16. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein when the processor executes the computer-executable instructions, it implements the steps of the image category determination method described in any one of claims 1 to 12, or implements the steps of the classification model training method described in claim 13.

17. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the image category determination method described in any one of claims 1 to 12, or implement the steps of the classification model training method described in claim 13.

18. A computer program product, characterized in that It includes computer instructions, which, when executed by a processor, implement the steps of the image category determination method described in any one of claims 1 to 12, or implement the steps of the classification model training method described in claim 13.

Images