A method for training a feature extraction network and its apparatus, a classification method and its apparatus, an electronic device, and a computer program.

Abstract

Embodiments of this application provide a method and apparatus for training a feature extraction network, a classification method and apparatus, and an electronic device applicable to various scenarios such as cloud technology, artificial intelligence, intelligent transportation, and driver assistance. The method includes the steps of: adjusting the parameters of a feature extraction network to be trained and the initial cluster center matrix according to the category labels of a first sample image, an initial cluster center matrix corresponding to each of the multiple cluster centers in each of the multiple categories, and a first loss identified by the image features of the first sample image extracted from the feature extraction network to be trained, to obtain a pre-training feature extraction network and a first cluster center matrix corresponding to each of the cluster centers; and adjusting the parameters of a pre-training feature extraction network according to the category labels of a second sample image, each of the first cluster center matrices, and a second loss identified by the image features of the second sample image extracted from the pre-training feature extraction network, to obtain a trained feature extraction network.

Description

[Technical Field]

[0001] [Reciprocity criteria for related applications] This application claims priority to Chinese Patent Application No. 202311431364.0, filed with the China National Intellectual Property Administration on 31 October 2023, with the title of the invention being "Method and apparatus for training a feature extraction network, a classification method and apparatus therefor, and an electronic device," the entirety of which is incorporated herein by reference.

[0002] [Technical field] This application relates to the field of artificial intelligence, and more specifically to a method and apparatus for training a feature extraction network, a classification method and apparatus, and an electronic device. [Background technology]

[0003] Feature extraction networks are deep learning techniques that can extract useful features from input data and perform effective classification and prediction.

[0004] When training an image feature extraction network using a sample set, the sample set typically includes both high-quality, normal sample images and lower-quality, complex sample images such as non-frontal, low-luminance, or low-resolution sample images. [Overview of the Initiative] [Means for solving the problem]

[0005] According to one embodiment of the present invention, a method and apparatus for training a feature extraction network capable of bringing the features of complex samples closer to the main cluster centers of the corresponding categories in order to avoid shifts in cluster centers is provided, as well as a classification method and apparatus and an electronic device.

[0006] According to one embodiment of the present application, in each iteration epoch of the first training phase, the following steps are performed: extracting image features of a first sample image using a feature extraction network to be trained; determining a first loss according to the image features of the first sample image, the category label of the first sample image, and an initial cluster center matrix corresponding to the multiple cluster centers of each category within the multiple categories; and adjusting the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to the multiple cluster centers of each category according to the first loss until the termination condition of the first training phase is met, thereby obtaining a feature extraction network for preliminary training and a first cluster center matrix corresponding to the multiple cluster centers of each category, and then performing a second training... A method for training a feature extraction network is provided, which, per iterative epoch of the training phase, performs the following steps: extracting image features of a second sample image using a pre-training feature extraction network; determining a second loss according to the image features of the second sample image, the category label of the second sample image, and a first cluster center matrix corresponding to the principal cluster centers of each category, wherein the principal cluster center for each category is one of a plurality of cluster centers of that category; and adjusting the parameters of the pre-training feature extraction network and the first cluster center matrix corresponding to the principal cluster centers of each category according to the second loss until the termination condition of the second training phase is met, thereby obtaining a trained feature extraction network for image classification.

[0007] According to one embodiment of the present application, an image classification method is further provided, which includes the steps of: acquiring an image for classification; extracting features from the image for classification using a trained image feature extraction network obtained by the feature extraction network training method described above to obtain target image features; and identifying a classification result for the image for classification according to the target image features.

[0008] According to one embodiment of the present application, a first feature extraction module configured to perform the step of extracting image features of a first sample image using a feature extraction network to be trained; a first loss determination module configured to perform the step of determining a first loss according to the image features of the first sample image, the category label of the first sample image, and an initial cluster center matrix corresponding to a plurality of cluster centers in each of the plurality of categories; a first adjustment module configured to perform the step of adjusting the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to a plurality of cluster centers in each category according to the first loss, thereby obtaining a feature extraction network for preliminary training and a first cluster center matrix corresponding to a plurality of cluster centers in each category; and the preliminary training A feature extraction network training device is further provided, comprising: a second feature extraction module configured to perform the step of extracting image features of a second sample image using a feature extraction network; a second loss determination module configured to perform the step of determining a second loss according to the image features of the second sample image, the category label of the second sample image, and a first cluster center matrix corresponding to the principal cluster centers of each category, wherein the principal cluster center for each category is one of a plurality of cluster centers of that category; and a second adjustment module configured to perform the step of adjusting the parameters of the pre-training feature extraction network and the first cluster center matrix corresponding to the principal cluster centers of each category according to the second loss to obtain a trained feature extraction network for image classification.

[0009] According to one embodiment of the present application, an image classification device is further provided, comprising: an image acquisition module configured to perform the step of acquiring an image for classification; a third feature extraction module configured to perform the step of extracting features from the image for classification and obtaining target image features using a trained image feature extraction network obtained by the feature extraction network training device described above; and a classification result identification module configured to perform the step of identifying the classification result of the image for classification according to the target image features.

[0010] According to one embodiment of the present application, an electronic device is further provided, comprising a processor and memory, wherein one or more programs are stored in the memory and are configured to be executed by the processor to implement the above method.

[0011] According to one embodiment of the present application, a computer-readable storage medium is provided which stores program code, and when the program code is executed by a processor, the above method is realized.

[0012] According to one embodiment of the present application, a computer program product or computer program is further provided, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device retrieves the computer instructions from the computer-readable storage medium and causes the computer device to perform the above method by having the processor execute the computer instructions. [Brief explanation of the drawing]

[0013] [Figure 1] This invention presents an application scenario for a feature extraction network training method according to one embodiment of this application. [Figure 2] A flowchart of a training method for a feature extraction network according to one embodiment of this application is shown. [Figure 3]A schematic diagram for cutting out a palm print pixel region according to an embodiment of the present application is shown. [Figure 4] A schematic diagram of the network structure of a feature extraction network according to an embodiment of the present application is shown. [Figure 5] Another flowchart of a training method for a feature extraction network according to an embodiment of the present application is shown. [Figure 6] A flowchart of an image classification method according to an embodiment of the present application is shown. [Figure 7] A flowchart of the application of a feature extraction network according to an embodiment of the present application is shown. [Figure 8] A connection block diagram of a training device for a feature extraction network according to an embodiment of the present application is shown. [Figure 9] It is a connection block diagram of an image classification device according to an embodiment of the present application. [Figure 10] A structural block diagram of an electronic device for executing a method according to an embodiment of the present application is shown.

Modes for Carrying Out the Invention

[0014] Hereinafter, exemplary embodiments will be more comprehensively described with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as being limited to the reference examples described in this specification. Rather, by providing these embodiments, the present application becomes more comprehensive and complete, and the concept of the exemplary embodiments can be sufficiently conveyed to those skilled in the art.

[0015] Furthermore, the features, structures, or properties described herein can be combined in any suitable manner in one or more embodiments. The following description provides more specific details to enable a full understanding of the embodiments of this application. However, those skilled in the art will recognize that the technical methods of this application can be implemented without one or more of the specific details, or that other methods, components, apparatus, steps, etc., can be employed. Also, in order to avoid ambiguity of aspects of this application under other circumstances, well-known methods, apparatus, implementations, or operations are not illustrated or described in detail.

[0016] The block diagrams shown in the attached drawings represent only functional entities and do not necessarily correspond to physically separate entities. In other words, these functional entities may be implemented in software, one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices, etc.

[0017] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order shown. For example, some operations / steps may be broken down, while others may be combined or partially combined. Therefore, the actual execution order may differ depending on the situation.

[0018] In this specification, "plural" refers to two or more things. "And / or" indicates a relationship between related objects, meaning that there may be three possible relationships. For example, "A and / or B" means that A exists alone, A and B exist together, or B exists alone. " / " usually means that the related objects are in an "or" relationship.

[0019] In some methods, when training an image feature extraction network using a sample set, the sample set may contain a mixture of high-quality, normal sample images and low-quality, complex sample images. When training an image feature extraction network using a sample set, the distribution of cluster centers is prone to shifting during training with complex sample images, which may affect the training effect. To solve the above problem, this application proposes training the feature extraction network in two phases. This will be explained in detail below.

[0020] The method described in this application primarily involves performing image classification using machine learning.

[0021] In this application, multiple cluster centers are defined within each of the multiple categories. For each category, one cluster center corresponds to one subcategory within that category. In other words, in this application, multiple subcategories are defined within each category. Here, each cluster center corresponds to an initial cluster center matrix, which is the feature space of that cluster center. Multiple cluster centers for each category are pre-constructed, and the initial cluster center matrix corresponding to each cluster center is a fully connected matrix. The number of cluster centers in different categories may be the same or different. Before training the feature extraction network, initial values ​​can be set for each initial cluster center matrix corresponding to each of the multiple cluster centers in each category. The initial values ​​of the initial cluster center matrices corresponding to different cluster centers may be the same or different.

[0022] Figure 1 is a schematic diagram showing an application scenario relating to one embodiment of the present invention. As shown in Figure 1, the application scenario includes a terminal device 10 and a server 20 that is connected to the terminal device 10 via a network so as to be able to communicate with it.

[0023] The terminal device 10 may specifically be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, or an in-vehicle terminal. The terminal device 10 may have a client for displaying data. The network may be a wide area network (WAN), a local area network (LAN), or a combination of the two.

[0024] Server 20 may be an independent physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDNs, big data, and artificial intelligence platforms.

[0025] As shown in Figure 1, when training a feature extraction network using a terminal device 10 and a server 20, the terminal device 10 can upload a first sample image and a second sample image to the server 20. After obtaining the first and second sample images, the server 20 performs the following steps per iterative epoch (iteration cycle) of the first training phase: extracting image features from the first sample image using the feature extraction network to be trained; identifying a first loss according to the image features of the first sample image, the category label of the first sample image, and the initial cluster center matrix corresponding to the multiple cluster centers of each category within the multiple categories; and adjusting the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to the multiple cluster centers of each category according to the first loss until the termination condition of the first training phase is met, and then adjusting the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to the multiple cluster centers of each category. The following steps are performed for each iteration epoch of the second training phase: extracting image features of a second sample image using a pre-training feature extraction network; determining a second loss according to the image features of the second sample image, the category label of the second sample image, and a first cluster center matrix corresponding to the principal cluster centers of each category, wherein the principal cluster center for each category is one of a plurality of cluster centers for that category; and adjusting the parameters of the pre-training feature extraction network and the first cluster center matrix corresponding to the principal cluster centers of each category according to the second loss until the termination condition of the second training phase is met, thereby obtaining a trained feature extraction network for image classification.

[0026] By employing the above method, this application enables the following in the training process of a model with a very large number of training samples, many of which are low-quality and complex: setting multiple cluster centers for each category in multiple classifications corresponding to a classification task; and in the first training phase of the feature extraction network, adjusting the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to each of the multiple cluster centers for each category according to the image features of the first sample image, the category label of the first sample image, and a first loss identified by the initial cluster center matrix corresponding to each of the multiple cluster centers for each category. This makes it possible to map the image features of sample images belonging to each category (e.g., normal sample images and complex sample images) to one of the multiple cluster centers corresponding to that category. In the second training phase of the feature extraction network, after the main cluster centers are determined, the parameters of the pre-training feature extraction network and the first cluster center matrix corresponding to the main cluster centers of each class are adjusted according to the image features of the second sample image, the category labels of the second sample image, and the second loss identified in the first cluster center matrix corresponding to the main cluster centers of each class. This allows the image features extracted from complex samples in the second sample image using the pre-training feature extraction network to gradually approach the main cluster centers. As a result, the feature extraction performance of the pre-training feature extraction network for normal samples in the second sample image is maintained, while the feature extraction performance for low-quality complex sample images in the second sample image is improved, thereby improving the feature extraction accuracy of the trained feature extraction network.

[0027] After the feature extraction network has been trained, it can be placed on the server 20. Therefore, after the images for classification are acquired, the trained image feature extraction network can be used to extract features from the classification images and obtain the target image features. Furthermore, subsequent image processing operations, such as image classification operations, can be performed according to the target image features.

[0028] Various embodiments of this application will be described in detail below with reference to the attached drawings.

[0029] Referring to Figure 2, a method for training the feature extraction network according to this application is also shown. This method is applicable to an electronic device which may be the terminal device 10 or server 20 described above. This method includes a first training phase and a second training phase.

[0030] The next steps S110-S130 are executed in each iteration epoch of the first training phase.

[0031] Step S110: Use the feature extraction network to be trained to extract image features from the first sample image.

[0032] The first sample image can be obtained by labeling an image within an image set, or by obtaining multiple pre-stored images with sample category labels from an electronic device or other devices associated with the electronic device as the first sample image. The settings should be configured according to the actual needs.

[0033] There are multiple sample images, each with a category label. These category labels can be set according to the classification task. The category corresponding to the category label of the first sample image belongs to one of several categories within the classification task. If the classification task is to determine whether an image is eligible, the category label of the sample image will be eligible or ineligible. If the classification task is to classify objects in the image (e.g., plants or animals), such as cats, dogs, or pigs, the category label of the sample image can be the specific category to which the object in the image belongs. If the classification task is to determine whether an image is abnormal, the category label of the sample image will be normal or abnormal. If the classification task is to identify specific identifying information corresponding to an image, the category label of the sample image will be the identifying information of a certain object.

[0034] Taking the classification task as an example where the classification is performed based on the biometric information of an object (e.g., a facial image, fingerprint image, palm print image, or iris image), the category label is object identification information that uniquely identifies the object, such as the object's name or ID. In such a case, the first sample image is obtained by extracting a specified region from the initial image. Specifically, keypoint detection is performed on the initial image, and based on the keypoint detection results, a specified region (e.g., a facial pixel region, a fingerprint pixel region, a palm print pixel region, or an iris pixel region) is extracted from the initial image and used as the first sample image.

[0035] As shown in Figure 3, exemplarily, when the biometric information is palm print information, the method for obtaining the first sample image is as follows: Keypoint detection is performed on the initial image to obtain the keypoints within the initial image. The keypoints within the initial image include the first interdigital keypoint A between the index finger and middle finger, the second interdigital keypoint B between the middle finger and ring finger, and the third interdigital keypoint C between the ring finger and little finger. An image coordinate system is created based on the first interdigital keypoint A, the second interdigital keypoint B, and the third interdigital keypoint C in the initial image. The line connecting the first interdigital keypoint A and the third interdigital keypoint C is defined as the horizontal axis (x-axis) of the image coordinate system, and the line passing through the second interdigital keypoint B and perpendicular to the horizontal axis is defined as the vertical axis (y-axis) of the image coordinate system. The intersection of the horizontal and vertical axes is defined as the origin of the image coordinate system. A point in the initial image that lies on the vertical axis of the image coordinate system and is a predetermined distance from the origin of the image coordinate system is defined as the palm print center point D in the initial image. This predetermined distance can be determined by the distance between the first interdigital keypoint A and the third interdigital keypoint C. The palm print center point D and the second interdigital keypoint B are located on opposite sides of the horizontal axis. A sample image is extracted from the initial image based on the palm print center point D and the distance between the first interdigital keypoint A and the third interdigital keypoint B.

[0036] Specifically, after establishing the image coordinate system shown in Figure 3 according to the first interdigital keypoint A, the second interdigital keypoint B, and the third interdigital keypoint C, the palm print center point D is defined as a point located at a distance AC from the coordinate origin along the negative y-axis. Distance DE is 6 / 5 times distance AC. The length of the side of the palm print pixel region is obtained by multiplying the distance from point A to point C by 3 / 2. The palm print pixel region is then cropped as a sample image (i.e., the first sample image) with point D as the center and d as the side length of the square.

[0037] It should be understood that, since the method for extracting an image of a specified region from the initial image differs depending on the biological characteristics of the object, this will not be discussed in detail here.

[0038] Here, the feature extraction network can be constructed using one or more neural networks. Specifically, the neural network may be any neural network capable of image feature extraction, such as ResNet residual network, DenseNet classic network, VGG convolutional neural network, AlexNet deep convolutional neural network, Swin-Transformer network, MaxViT network, or LeNet convolutional neural network, but is not particularly limited in this embodiment.

[0039] Step S120: Identify the first loss according to the image features of the first sample image, the category label of the first sample image, and the initial cluster center matrix corresponding to the multiple cluster centers of each category within the multiple categories.

[0040] Step S120 described above may include the steps of: identifying a category prediction result for the first sample image based on the similarity between the image features of the first sample image and the initial cluster center matrix corresponding to multiple cluster centers of each category; and calculating a loss based on the category prediction result for the first sample image and the category label of the first sample image to obtain a first loss.

[0041] In this form, a method for identifying the category prediction result of a first sample image based on the similarity between the image features of the first sample image and the initial cluster center matrix corresponding to each of the multiple cluster centers of each category may involve, for each category, identifying the mean or maximum value of the similarity between the image features of the first sample image and the initial cluster center matrix corresponding to each of the multiple cluster centers of that category as the reference similarity between the image features of the first sample image and that category, and then identifying the probability that the first sample image belongs to each category based on the reference similarity between the image features of the first sample image and that category. In this method, the reference similarity between the image features of the first sample image and that category reflects the difference between the image features corresponding to the first sample image and the feature space corresponding to that category. The smaller the difference, the higher the probability that the first sample image is predicted to belong to that category. Here, the probability that the first sample image is predicted to belong to each category is, in other words, the category prediction result of the first sample image.

[0042] Step S120 described above may also include the steps of: identifying at least one reference cluster center from multiple cluster centers corresponding to each category based on the similarity between the image features of the first sample image and the initial cluster center matrix corresponding to multiple cluster centers of each category; identifying the category prediction result for the first sample image based on the similarity between each reference cluster center of each category and the image features of the first sample image; and calculating a loss based on the category prediction result for the first sample image and the category label of the first sample image to obtain a first loss.

[0043] Within this framework, the similarity between the image features of the first sample image and the initial cluster center matrices corresponding to each of the multiple cluster centers in each category reflects the differences between the image features corresponding to the first sample image and the feature spaces corresponding to each of the initial cluster center matrices for each of the multiple cluster centers in each category. The similarity between a given reference cluster center and the image features of the first sample image also reflects the differences between the feature space corresponding to the reference cluster center and the image features. Here, it should be understood that the smaller the difference, the more likely the predicted category of the first sample image is to be the category corresponding to that reference cluster center.

[0044] The method for calculating the similarity between an initial cluster center matrix corresponding to a certain cluster center and the image features of the first sample image may involve calculating the cosine similarity or Euclidean distance between the initial cluster center matrix corresponding to that cluster center and the image features of the first sample image.

[0045] When calculating the loss for the category prediction result and category label of the first sample image, a pre-set loss function can be used to calculate the loss for the category prediction result and category label of the first sample image. Here, the pre-set loss function may be a cross-entropy loss function, a mean squared error loss function, or a multi-class cross-entropy loss function, but it should be set according to the actual needs.

[0046] Step S130: Adjust the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to the multiple cluster centers of each category according to the first loss until the termination condition of the first training phase is met, thereby obtaining the feature extraction network for preliminary training and the first cluster center matrix corresponding to the multiple cluster centers of each category.

[0047] In this embodiment, the training of the feature extraction network to be trained is considered to have met the completion conditions for the first training phase when the number of iterations in the first training phase reaches a first predetermined number, when the number of epochs corresponding to the iteration epoch reaches a first predetermined number of epochs, or when the first loss is less than a first predetermined loss threshold. Furthermore, when the training of the feature extraction network to be trained meets the completion conditions for the first training phase, the feature extraction network to be trained after the final iteration adjustment is used as a feature extraction network for preliminary training, and the initial cluster center matrix corresponding to each of the multiple cluster centers in each category after the final iteration adjustment is used as the first cluster center matrix corresponding to each of the multiple cluster centers in each category. Here, the first predetermined number, the first predetermined number of epochs, and the first predetermined loss threshold can be set according to the task requirements, but are not specifically limited here.

[0048] Furthermore, the training process in steps S110 to S130 described above may be configured to use multiple first training sets. Each first training set contains multiple first sample images. When all the first sample images in a single first training set have been used once to train the feature extraction network to be trained, one iterative epoch is considered complete. In some embodiments, the first sample images in each first training set can be input into a feature extraction network to be trained batch by batch for iterative training. The number of batches corresponding to each batch can be set according to the actual needs.

[0049] By iteratively adjusting the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to each of the multiple cluster centers in each category, the feature extraction accuracy of the feature extraction network can be improved, while simultaneously constraining the feature space corresponding to each of the multiple cluster centers in each category.

[0050] Steps S140 to S160 are executed in each iteration epoch of the second training phase.

[0051] Step S140: Use the pre-training feature extraction network to extract image features from the second sample image.

[0052] Within this framework, the method for acquiring the second sample image is the same as or similar to the method for acquiring the first sample image, and the category corresponding to the second sample image also belongs to one of several categories corresponding to the classification task when the pre-training feature extraction network is used for the classification task. Therefore, while the acquisition and processing of the second sample image can be described in the above-mentioned explanation regarding the first sample image, it is not specifically limited to that explanation.

[0053] It should be understood that there may be multiple second sample images, which may differ from the first sample image, or the first sample image mentioned above may be used as a second sample image to train the feature extraction network for preliminary training.

[0054] Details regarding the extraction of a second sample image using a feature extraction network for preliminary training can be found in the specific explanation for step S110 above, but will not be detailed here.

[0055] Step S150: Identify the second loss according to the image features of the second sample image, the category labels of the second sample image, and the first cluster center matrix corresponding to the principal cluster centers of each category.

[0056] The primary cluster center for each category is one of several cluster centers within that category.

[0057] Within that context, the primary cluster center for each category refers to the cluster center corresponding to the feature space (cluster center matrix) where the features of images belonging to that category are mainly distributed.

[0058] In some embodiments, the primary cluster center for each category can be identified according to the number of sample images (e.g., first sample images) assigned to multiple cluster centers within that category. For example, among the multiple cluster centers within that category, the cluster center to which the most first sample images are assigned may be determined as the primary cluster center for that category.

[0059] In some other embodiments, since there are multiple first sample images, when identifying the primary cluster center of a category, the steps are to identify a reference image set corresponding to each category, where the reference image set corresponding to each category includes a category label and multiple first sample images belonging to the same category; for each cluster center of each category, to identify a reference similarity corresponding to that cluster center according to the similarity between the initial cluster center matrix of that cluster center and the image features of each first sample image in the reference image set corresponding to that category; and for each category, to identify the cluster center with the highest reference similarity within that category as the primary cluster center of that category.

[0060] In this embodiment, for each cluster center in each category, the mean, maximum, or median of the similarity between the initial cluster center matrix of that cluster center and the image features of each first sample image in the reference image set corresponding to that category may be determined as the reference similarity corresponding to that cluster center.

[0061] In some embodiments, if there are multiple second sample images, step S150 may also include the steps of: calculating the similarity between the image features of each second sample image and a first cluster center matrix corresponding to the main cluster centers of each category to obtain a second similarity between the image features of each second sample image and the main cluster centers of each category; identifying the category to which the main cluster center with the highest second similarity to the image features of the second sample image belongs among the multiple categories as the predicted category of the second sample image; and identifying a second loss based on the category label of the second sample image and the predicted category of the second sample image.

[0062] Within this framework, the method for calculating the second similarity between the image features of each second sample image and the principal cluster centers of each category may be the same as the method described above for calculating the similarity between the image features of the first sample image and the initial cluster center matrix corresponding to each cluster center. The process for identifying the second loss may also be the same as the process for identifying the first loss described above, but this will not be described in detail here.

[0063] Here, the second method of obtaining the loss described above is merely one example, and many more methods of obtaining the loss are possible, but these will not be described in detail in this embodiment.

[0064] Step S160: Adjust the parameters of the pre-training feature extraction network and the first cluster center matrix corresponding to the main cluster centers of each category according to the second loss until the termination condition of the second training phase is met, thereby obtaining the trained feature extraction network.

[0065] In this embodiment, the trained feature extraction network can be used for image classification.

[0066] In this embodiment, the training of the pre-training feature extraction network may be considered to have met the conditions for the end of the second training phase when the number of iterations of the pre-training feature extraction network reaches a second predetermined number, when the number of epochs corresponding to the iteration epoch reaches a second predetermined number of epochs, or when the second loss is less than a second predetermined loss threshold. Furthermore, when the training of the pre-training feature extraction network meets the conditions for the end of the second training phase, the pre-training feature extraction network after the final iteration adjustment is used as the trained feature extraction network. Here, the second predetermined number, the second predetermined number of epochs, and the second predetermined loss threshold can be set according to the task requirements, but are not specifically limited here.

[0067] During the training process of the pre-training feature extraction network, by adjusting the parameters of the pre-training feature extraction network and the first cluster center matrix corresponding to the main cluster centers of each category according to the second loss, it is possible to control the image features extracted from each second sample image by the pre-training feature extraction network during the training process so that they gradually approach the main cluster centers of the corresponding category. In other words, even if the second sample image is a complex sample such as a non-frontal sample image, a low-luminance sample image, or a low-resolution sample image, it is possible to control the image features extracted from the complex second sample image by the pre-training feature extraction network so that they approach the main cluster centers of the corresponding category. As a result, it is possible to ensure the feature extraction accuracy of the trained feature extraction network for normal images and improve the feature extraction accuracy for low-quality images, thereby enabling accurate classification of low-quality and complex images.

[0068] The feature extraction network training method according to the embodiment of this application sets multiple cluster centers for each category and adjusts the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to the multiple cluster centers for each category according to a first loss identified by the image features of the first sample image, the category label of the first sample image, and the initial cluster center matrix corresponding to the multiple cluster centers of each category, thereby improving the feature extraction accuracy of the feature extraction network to be trained and simultaneously constraining the feature space corresponding to the multiple cluster centers of each category. Subsequently, the parameter of the feature extraction network for preliminary training and the first cluster center matrix corresponding to the main cluster centers of each class are adjusted according to a second loss identified by the image features of the second sample image, the category label of the second sample image, and the first cluster center matrix corresponding to the main cluster centers of each class, so that during the training process, the image features extracted by the feature extraction network for preliminary training gradually approach the main cluster centers of each category. In other words, even if the second sample image is a complex sample, it is possible to control the features of the complex sample to approach the main cluster centers of the corresponding category, thereby solving the problem of cluster center misalignment caused by complex samples. As a result, the feature extraction performance of the trained feature extraction network is ensured for normal samples within the sample image, while simultaneously improving the feature extraction performance for low-quality, complex sample images within the sample image. This allows for improved feature extraction accuracy by the trained feature extraction network.

[0069] Referring to Figure 4, one embodiment of this application provides a method for training a feature extraction network. This method includes the following steps:

[0070] Step S210: Use the feature extraction network to be trained to extract image features from the first sample image.

[0071] Details of step S210 can be found in the specific explanation of step S110 above, but will not be detailed here.

[0072] Step S220: For each category, calculate a first similarity between the image features of the first sample image and the initial cluster center matrix corresponding to each of the multiple cluster centers in that category.

[0073] The process for calculating the similarity between image features and the cluster center matrix can be found in the detailed explanation mentioned earlier, but will not be described in detail here.

[0074] Step S230: Among the multiple initial cluster center matrices for each category, the initial cluster center matrix that yields the highest first similarity to the image features of the first sample image is identified as the first reference cluster center matrix.

[0075] In other words, the first reference cluster center matrix refers to the initial cluster center matrix among multiple cluster center matrices of multiple categories that yields the highest first similarity to the image features of the first sample image.

[0076] Step S240: Identify the first sub-loss according to the first reference cluster center matrix and the category labels of the first sample images.

[0077] The category to which the cluster center corresponding to the first reference cluster center matrix belongs can be used as the predicted category corresponding to the first sample image. The first sub-loss reflects the difference between the predicted category corresponding to the first sample image and the category indicated by the category label of that first sample image. Here, the smaller the difference between the predicted category corresponding to the first sample image and the category indicated by the category label of that first sample image, the higher the accuracy of the image features extracted by the feature extraction network to be trained.

[0078] Step S250: For each category, among the multiple initial cluster center matrices for that category, the initial cluster center matrix that yields the highest first similarity to the image features of the first sample image is identified as the second reference cluster center matrix for that category.

[0079] For each category, the second criterion cluster center matrix for that category refers to the initial cluster center matrix that maximizes the first similarity to the image features of the first sample image within that category.

[0080] Step S260: A second sub-loss is identified according to a first similarity between the image features of the first sample image and a second reference cluster center matrix in each of the multiple categories.

[0081] Specifically, the second sub-loss can be identified from the sum or mean of the first similarity between the second reference cluster center matrix and the image features of the first sample image in each of the multiple categories.

[0082] Step S270: Identify the first loss based on the first sub-loss and the second sub-loss.

[0083] In this context, the method for identifying the first loss based on the first and second sub-losses may be to weight the first and second sub-losses and calculate their sum to determine the first loss, or to identify the sub-loss with the largest loss value among the first and second sub-losses as the first loss. However, the method should be set according to the actual needs.

[0084] In some embodiments, step S270 above includes the following steps:

[0085] Step S270a: Identify the first margin parameter and the first scaling coefficient based on the first epoch number corresponding to the iterative epoch in which the first sample image is involved. The first margin parameter and the first scaling coefficient are negatively correlated with the first epoch number.

[0086] The first epoch number refers to the epoch number corresponding to the iteration epoch in which the first sample image is involved. That is, if the first sample image is involved in the k-th iteration epoch, the first epoch number corresponding to the iteration epoch in which the first sample image is involved is k. It should be understood that different first sample images may be involved in different iteration epochs and correspond to different first epoch numbers.

[0087] By determining the first margin parameter and first scaling coefficient based on the first epoch number corresponding to the iterative epoch involving the first sample image, the parameters can be set adaptively. This ensures that the monitoring loss is relatively mitigated and does not affect the feature distribution of normal samples, while also avoiding situations where the feature extraction network is susceptible to complex samples due to unstable gradients in the early stages of training. Furthermore, since the first margin parameter and first scaling coefficient are negatively correlated with the first epoch number, the larger the first epoch number, the smaller the first margin parameter and first scaling coefficient become. As the number of iterative epochs increases, the loss requirement can be gradually increased by gradually decreasing the first margin parameter and first scaling coefficient.

[0088] Step S270b: Based on the first margin parameter, the first scaling factor, the first sub-loss, and the second sub-loss, calculate the summation angle margin loss and obtain the first loss.

[0089] Specifically, the first loss can be calculated using the first summation angle margin loss calculation formula, based on the first margin parameter, the first scaling factor, the first sub-loss, and the second sub-loss. Here, the first summation angle margin loss calculation formula is as follows:

[0090]

number

number

[0091] In some embodiments, a method for determining a first margin parameter and a first scaling coefficient may specifically include the steps of: determining a first coefficient based on a first epoch number corresponding to a repeating epoch in which a first sample image is involved, wherein the first coefficient is negatively correlated with the first epoch number; determining a first margin adjustment value according to the first coefficient, adding the first margin adjustment value to a first reference margin parameter to obtain a first margin parameter; and determining a first scaling adjustment value according to the first coefficient, adding the first scaling adjustment value to a first reference scaling coefficient to obtain a first scaling coefficient.

[0092] For example, a first difference value is obtained by subtracting the first epoch number corresponding to the iterative epoch in which the first sample image is involved from a first predetermined epoch number. The ratio of the first difference value to the first predetermined epoch number is defined as the first coefficient. A first margin adjustment value is obtained by multiplying the first coefficient by a first reference margin parameter, and a first margin parameter is obtained by adding the first reference margin parameter to the first margin adjustment value. A first scaling adjustment value is obtained by multiplying the first coefficient by a predetermined scaling coefficient, and a first scaling coefficient is obtained by adding the first scaling adjustment value to the first reference scaling coefficient. Here, the first predetermined epoch number can be set as needed, as long as the difference value from the first epoch number is 0 or greater. The specific values ​​of the first reference margin parameter, the predetermined scaling coefficient, and the first reference scaling coefficient are not particularly limited here and should be set according to actual needs.

[0093] Step S280: Adjust the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to the multiple cluster centers of each category according to the first loss until the termination condition of the first training phase is met, thereby obtaining the feature extraction network for preliminary training and the first cluster center matrix corresponding to the multiple cluster centers of each category.

[0094] Step S290: Use the pre-training feature extraction network to extract image features from the second sample image. Here, there are multiple second sample images.

[0095] Step S300: The similarity between the image features of the second sample image and the first cluster center matrix corresponding to the main cluster centers of each category is calculated to obtain a second similarity between the image features of the second sample image and the main cluster centers of each category.

[0096] Step S310: Among multiple categories, the category to which the primary cluster center with the highest second similarity to the image features of the second sample image belongs is identified as the prediction category for the second sample image.

[0097] Step S320: Identify a second loss based on the category label of the second sample image and the predicted category of the second sample image.

[0098] Here, it should be understood that the smaller the second loss, the better the category label of the second sample image matches the predicted category of the second sample image. Conversely, the larger the second loss, the worse the category label of the second sample image matches the predicted category of the second sample image.

[0099] Considering that the parameters of a feature extraction network tend to stabilize in the later stages of training, the main goal of the training phase is to bring complex samples, which are far from the cluster centers corresponding to normal samples, closer to the cluster centers of normal samples. Therefore, more stringent monitoring parameters are required. By designing an adaptive parameter setting method during the parameter tuning process, a more stringent monitoring strategy can be gradually adopted in the later stages of training, thereby improving the model's usability for complex samples while ensuring the identification effect of normal data. Specifically, in some embodiments, step S320 above includes the following steps.

[0100] Step S320a: Identify the second margin parameter and the second scaling coefficient based on the second epoch number corresponding to the iterative epoch in which the second sample image is involved. The second margin parameter and the second scaling coefficient are negatively correlated with the second epoch number.

[0101] The second epoch number refers to the epoch number corresponding to the iteration epoch in which the second sample image is involved. In other words, if the second sample image is involved in the z-th iteration epoch, the second epoch number corresponding to the iteration epoch in which that second sample image is involved is z. Here, it can be understood that different second sample images may be involved in different iteration epochs, and therefore the corresponding second epoch numbers may be different.

[0102] In this context, the method for determining the second margin parameter and the second scaling coefficient may include the steps of: determining the second coefficient based on the second epoch number corresponding to the iterative epoch in which the second sample image is involved, and determining that the second coefficient is negatively correlated with the second epoch number; determining the second margin adjustment value according to the second coefficient, adding the second margin adjustment value to the second reference margin parameter to obtain the second margin parameter; and determining the second scaling adjustment value based on the second coefficient, adding the second scaling adjustment value to the second reference scaling coefficient to obtain the second scaling coefficient.

[0103] For example, a second difference value is obtained by subtracting the second epoch number corresponding to the iterative epoch in which the second sample image is involved from a second predetermined epoch number. The ratio of the second difference value to the second predetermined epoch number is taken as the second coefficient. The second margin adjustment value is obtained by multiplying the second coefficient by the second reference margin parameter, and the second margin parameter is obtained by adding the second reference margin parameter to the second margin adjustment value. The second scaling adjustment value is obtained by multiplying the second coefficient by a predetermined scaling coefficient, and the second scaling coefficient is obtained by adding the second scaling adjustment value to the second reference scaling coefficient. Here, the second predetermined epoch number can be set as needed, as long as the difference value with the second epoch number is 0 or greater. The specific values ​​of the second reference margin parameter, the predetermined scaling coefficient, and the second reference scaling coefficient are not particularly limited here and should be set according to the actual needs.

[0104] In some embodiments, to ensure effective training, the first margin parameter during the training period of the feature extraction network to be trained can be controlled to be less than or equal to the second margin parameter during the training period of the feature extraction network used for preliminary training, and the first scaling factor during the training period of the feature extraction network to be trained can be controlled to be less than or equal to the second scaling factor during the training period of the feature extraction network used for preliminary training.

[0105] Step S320b: Based on the second margin parameter, the second scaling coefficient, the category label of the second sample image, and the predicted category of the second sample image, the summation angle margin loss is calculated to obtain the second loss.

[0106] Specifically, the second loss is obtained by calculating the added angle margin loss based on the second margin parameter, the second scaling coefficient, the second sample image category label, and the second sample image prediction category, using the second added angle margin loss calculation formula. Here, the second added angle margin loss calculation formula is as follows:

[0107]

Number

[0108] Step S330: Until the end condition of the second training phase is satisfied, adjust the parameters of the feature extraction network for pre-training and the first cluster center matrix corresponding to the main cluster center of each category according to the second loss, and obtain a trained feature extraction network.

[0109] Referring to FIG. 5, in one embodiment of the present application, an image classification method applicable to the above electronic device is further provided. This method includes the following steps.

[0110] Step S410: Obtain a classification image.

[0111] Among them, the classification image can be any image to be classified. The acquisition process may be the same as the acquisition methods of the first sample image and the second sample image in the foregoing embodiments, but is not specifically limited herein.

[0112] Here, in order to ensure the accuracy of features obtained by subsequent feature extraction from the classification image, step S410 above may involve acquiring an initial image, performing preprocessing such as denoising, enhancement, and filtering on the initial image, and then performing different processing operations for each classification task after obtaining the preprocessed initial image. For example, if the classification task is object identification, the region in which the object is located in the classification task can be cropped, or the object in the classification task can be scaled.

[0113] In some embodiments, when the classification image is a palm print image, step S410 above includes the following steps.

[0114] Step S412: Obtain an image of the hand.

[0115] Step S414: Keypoint detection is performed on the hand image to obtain interfinger keypoints in the hand image.

[0116] Step S416: Based on the interfacial keypoints in the hand image, the palm print pixel region is extracted from the hand image as a palm print image.

[0117] The specific process for obtaining palm print images can be found in the detailed explanation above, but will not be described in detail here.

[0118] Step S420: Using the trained feature extraction network obtained by the feature extraction network training method, features are extracted from the classification image to obtain the target image features.

[0119] Step S430: Identify the classification result of the classification image according to the target image features.

[0120] It should be understood that the method for determining the classification result of an image may differ depending on the application scenario corresponding to the trained feature extraction network. If the application scenario corresponding to the trained feature extraction network is an identification / authentication scenario, the classification result of the above image will be either authentication success or authentication failure. If the application scenario corresponding to the trained feature extraction network is a multiple classification scenario, the classification result of the above image will be the specific category to which the image belongs. If the trained feature extraction network is used in an image anomaly recognition scenario, the classification result of the above image will be whether the image is normal or abnormal. It should be understood that the above application scenarios for the trained feature extraction network are merely examples, and the classification method and classification results may differ depending on the application scenario.

[0121] When a trained feature extraction network is used in an identification and authentication scenario, step S430 above may result in the classification result of the classification image including an authentication result. Specifically, step S430 above may include the steps of matching a target image feature with a predetermined reference image feature, and, if a predetermined reference image feature that matches the target image feature exists, generating authentication information that indicates successful authentication, or using authentication information associated with the predetermined reference image feature as authentication information for the classification image.

[0122] In this embodiment, step S430 described above may specifically include the following steps.

[0123] Step S432: Calculate the similarity between the target image feature and multiple reference image features in a pre-configured database, and obtain the similarity between the target image feature and each reference image feature.

[0124] Step S434: Identify the target reference image feature that has the greatest similarity to the target image feature, according to the similarity between the target image feature and each reference image feature.

[0125] Step S436: Use the authentication information associated with the target reference image features as the authentication result for the classification image.

[0126] In this configuration, if authentication is successful, subsequent operations such as sending unlock data for a specific application, making payments, unlocking a device, unlocking a gate, or unlocking a smart lock can be performed.

[0127] In some embodiments, after using authentication information associated with target reference image features as the authentication result for the classification image, the method further includes the step of performing a settlement process based on the authentication result for the classification image.

[0128] When a trained feature extraction network is applied to a multi-classification task scenario, each category has one primary cluster center. This primary cluster center can be identified according to the training method of the feature extraction network described above. Step S430 above may include the step of calculating the similarity between the image features of the image to be classified and the primary cluster centers corresponding to each category in the multi-classification task, and identifying the classification result of the image to be classified according to the similarity between the image features and the primary cluster centers of each category.

[0129] It should be understood that the above method for identifying the classification result of the classification image is merely one example, and other identification methods are possible, but these will not be described in detail in this embodiment.

[0130] As shown in Figure 6, one embodiment of this application provides a method for training a feature extraction network. The trained feature extraction network is used in an application scenario in which palm print features of different users are extracted, the trained feature extraction network is used to perform identification and authentication of the palm print image to be identified, and payment is made if identification and authentication are successful. The specific training phase and application phase are as follows.

[0131] Training phase: First, multiple initial images, including a palm image, are obtained. Each initial image is assigned a label that represents the object to which the palm image belongs.

[0132] For each initial image, an object detection algorithm (e.g., the Yolov2 detection algorithm) is used to detect keypoints within the initial image, obtaining the first interdigital keypoint A between the index and middle fingers, the second interdigital keypoint B between the middle and ring fingers, and the third interdigital keypoint C between the ring and little fingers. Next, an image coordinate system is created based on the first interdigital keypoint A, the second interdigital keypoint B, and the third interdigital keypoint C in the initial image. The line connecting the first interdigital keypoint A and the third interdigital keypoint C is defined as the horizontal axis (x-axis) of the image coordinate system, and the line passing through the second interdigital keypoint B and perpendicular to the horizontal axis is defined as the vertical axis (y-axis) of the image coordinate system. The intersection of the horizontal and vertical axes is defined as the origin of the image coordinate system. A point in the initial image that lies on the vertical axis of the image coordinate system and is a predetermined distance from the origin of the image coordinate system is defined as the palm print center point D in the initial image. The predetermined distance can be determined by the distance between the first interdigital keypoint A and the third interdigital keypoint C. The palm print center point D and the second interdigital keypoint B are located on opposite sides of the horizontal axis. A sample image is extracted from the initial image based on the palm print center point D and the distance between the first interdigital keypoint A and the third interdigital keypoint B.

[0133] In this way, the first sample image and the second sample image are obtained. Here, there may be multiple first and second sample images, and they may be identical or different.

[0134] After obtaining the first and second sample images, they can be scaled to the same size for training the subsequent feature extraction network.

[0135] When training a feature extraction network, it is applied to a classification task. The number of categories in the classification task is E, the initial value of the initial cluster center matrix for each category is 0, and the number of cluster centers corresponding to each category is F. The initial cluster center matrix corresponding to each cluster center is a fully connected linear matrix of length N. Here, the category corresponding to the category label in the sample image belongs to one of the multiple categories in the classification task.

[0136] In the first training phase, the number of first sample images input in each iteration is G. Using the feature extraction network to be trained, image features of the first sample images are extracted, and then, for each category, a first similarity is calculated between the image features of the first sample images and the initial cluster center matrices corresponding to the multiple cluster centers of that category. Among the multiple initial cluster center matrices of each category, the initial cluster center matrix that has the highest first similarity to the image features of the first sample images is identified as the first reference cluster center matrix. The first sub-loss is identified according to the first reference cluster center matrix and the category label of the first sample image. Among the multiple initial cluster center matrices of each category, the initial cluster center matrix that has the highest first similarity to the image features of the first sample images is identified as the second reference cluster center matrix for that category. The second sub-loss is identified according to the first similarity between the second reference cluster center matrix for each category and the image features of the first sample images. Next, based on the first epoch number corresponding to the iterative epoch in which the first sample image is involved, a first margin parameter and a first scaling coefficient are identified, and the first margin parameter and the first scaling coefficient are negatively correlated with the first epoch number. Based on the first margin parameter, the first scaling coefficient, the first sub-loss, and the second sub-loss, the summation angle margin loss is calculated to obtain the first loss.

[0137] Specifically, the first loss can be calculated based on the following formula.

[0138]

number

number

[0139] In some embodiments, the first training phase is a phase in which the number of iteration epochs involving the first sample image is less than 10. Because the gradient of model training is unstable and susceptible to the influence of complex samples, an adaptive parameter setting method can be employed, and it is ensured that the supervision loss is relatively loose and does not affect the feature distribution of normal samples. The range of S1 is restricted to [24, 48] and the range of m1 is restricted to [0.3, 0.5]. The actual first margin parameter and first scaling coefficient are calculated from the following formulas.

[0140]

number

[0141] After the first loss is calculated, gradient feedback is used to adjust the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to the multiple cluster centers of each category until the termination condition of the first training phase is met, thereby obtaining a feature extraction network for preliminary training and a first cluster center matrix corresponding to the multiple cluster centers of each category. This allows the image features of the first sample image belonging to each category to be mapped to one of the multiple cluster centers corresponding to that category.

[0142] In the second training phase, there are multiple second sample images. Using a feature extraction network for preliminary training, image features of the second sample images are extracted, and the similarity between the image features of each second sample image and the first cluster center matrix corresponding to the main cluster centers of each category is calculated to obtain a second similarity between the image features of each second sample image and the main cluster centers of each category. Among the multiple categories, the category to which the main cluster center with the highest second similarity to the image features of the second sample image belongs is identified as the predicted category of the second sample image. Based on the category label of the second sample image and the predicted category of the second sample image, the second loss is determined.

[0143] Specifically, the second loss can be calculated based on the following loss function.

[0144]

number

[0145] In some embodiments, the second training phase is a phase in which the number of iteration epochs involving the second sample image is less than 10. In the second training phase, as the model parameters stabilize, more rigorous monitoring parameters are required because the main goal is to bring complex samples, which were originally far from the cluster centers of normal samples, closer to the cluster centers of normal samples. Similarly, the adaptive parameter setting method is designed to restrict the range of S2 to [48,64] and the range of m2 to [0.5,0.7]. The actual second margin parameter and second scaling factor are calculated from the following formulas.

[0146]

number

[0147] Through these two training phases, the feature extraction performance of the trained feature extraction network for normal samples is ensured, while at the same time, the feature extraction performance for low-quality, complex sample images within the sample image is improved, thereby enhancing the feature extraction accuracy of the trained feature extraction network.

[0148] Application phase: Referring to Figure 7, when the above-trained feature extraction network is used for palm print authentication payment, an image of the target object's hand is captured by the terminal payment device's camera. The detection model detects the interfingertip keypoints of the target object. The region of interest of the palm is extracted according to the hand image and the keypoint positions. The above-trained feature extraction network is used to extract a palm print feature encoding vector corresponding to the palm's region of interest. The cosine similarity between the extracted palm print feature encoding vector and the library features is calculated. Here, the cosine similarity is calculated using the following formula.

number

number

number

[0149] When processing payments using palm print authentication as described above, palm prints are less noticeable compared to facial recognition technology, thus protecting user privacy while remaining unaffected by masks, makeup, sunglasses, etc. Therefore, palm print authentication technology is expected to be widely used in commercial scenarios such as mobile payments and identity verification.

[0150] To verify the effectiveness of the pre-trained feature extraction network in this application, we used a dataset of 1000 IDs, each with 50 normal images and 50 complex images assigned to each ID. We then compared the effectiveness of the feature extraction network trained using the training method of this application with that of a feature extraction network trained using a prior art training method. The verification results are shown in Table 1 below.

[0151] [Table 1]

[0152] As shown in Table 1, while the misrecognition rate for complex datasets is significantly higher with conventional techniques than with normal datasets, the feature extraction network trained by the training method of this application shows good performance with both normal and complex datasets. Therefore, it was found that the training method for the feature extraction network of this application can improve the recognition accuracy of complex palm print data.

[0153] Referring to Figure 8, another embodiment of the present application provides a feature extraction network training device 500. The feature extraction network training device 500 includes: a first feature extraction module 510 configured to perform the step of extracting image features from a first sample image using a feature extraction network to be trained; a first loss determination module 520 configured to perform the step of determining a first loss according to the image features of the first sample image, the category labels of the first sample image, and an initial cluster center matrix corresponding to a plurality of cluster centers in each of the plurality of categories; and a first adjustment module configured to perform the step of adjusting the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to a plurality of cluster centers in each category according to the first loss, thereby obtaining a feature extraction network for preliminary training and a first cluster center matrix corresponding to a plurality of cluster centers in each category. The system comprises a 530 network, a second feature extraction module 540 configured to perform the step of extracting image features of a second sample image using a pre-training feature extraction network, a second loss identification module 550 configured to perform the step of identifying a second loss according to the image features of the second sample image, the category label of the second sample image, and a first cluster center matrix corresponding to the main cluster centers of each category, wherein the main cluster center for each category is one of a plurality of cluster centers of that category, and a second adjustment module 560 configured to perform the step of adjusting the parameters of the pre-training feature extraction network and the first cluster center matrix corresponding to the main cluster centers of each category according to the second loss to obtain a trained feature extraction network for image classification.

[0154] Referring to Figure 9, one embodiment of the present application provides an image classification device 600. The image classification device 600 includes an image acquisition module 610 configured to perform the step of acquiring an image for classification; a third feature extraction module 620 configured to perform the step of extracting features from an image for classification and obtaining target image features using a trained feature extraction network obtained by the above-mentioned training device for the trained feature extraction network; and a classification result identification module 630 configured to perform the step of identifying the classification result of an image for classification according to the target image features.

[0155] Each module in the aforementioned device can be implemented, in whole or in part, by software, hardware, or a combination thereof. Each of the aforementioned modules can be incorporated into the processor of a computer device in hardware form, or it can be independent of the processor of the computer device, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the aforementioned modules. Note that the embodiments of the device in this application correspond to the embodiments of the method described above. The specific principles of the embodiments of the device can be found in the contents of the embodiments of the method described above, but will not be described in detail here.

[0156] An electronic device according to one embodiment of this application will be described below with reference to Figure 10.

[0157] Referring to Figure 10, based on the feature extraction network training method according to the embodiment described above, another embodiment of this application further provides an electronic device 100 including a processor 102 capable of performing the above method. Such electronic device 100 may be a server, or a terminal device such as a smartphone, tablet computer, computer, or portable computer.

[0158] The electronic device 100 also includes a memory 104. Here, the memory 104 stores a program that can execute the contents of the embodiment described above. The processor 102 can execute the program stored in the memory 104.

[0159] In this configuration, the processor 102 may include one or more cores for data processing and a message matrix unit. The processor 102 connects various components within the electronic device 100 using various interfaces and circuits. The processor 102 performs various functions within the electronic device 100 and processes data by executing instructions, programs, code sets, or instruction sets stored in memory 104 and by retrieving data stored in memory 104. In some embodiments, the processor 102 may be implemented as at least one hardware form of a digital signal processing unit (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 102 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. In this configuration, the CPU primarily handles the operating system, user interface, and application programs, the GPU is responsible for rendering and drawing display content, and the modem handles wireless communication. It should be understood here that the modem described above may be implemented independently of processor 102 and may be implemented as a separate communication chip.

[0160] Memory 104 includes Random Access Memory (RAM) and Read-Only Memory (ROM). Memory 104 is used to store instructions, programs, code, code sets, or instruction sets. Memory 104 may include a program storage area and a data storage area. The program storage area can store instructions for implementing an operating system, instructions for implementing at least one function, and instructions for implementing various embodiments of the methods described later. The data storage area can also store data collected during the use of the electronic device 100 (e.g., training data or classification images).

[0161] The electronic device 100 may also include a network module and a screen. The network module is used to transmit and receive electromagnetic waves, switch between electromagnetic waves and electrical signals, and communicate with other devices such as communication networks or audio playback devices. The network module may include various existing circuit components to perform these functions, such as an antenna, radio frequency transceiver, digital signal processor, encryption / decryption chip, subscriber identification module (SIM) card, and memory. The network module can communicate with various networks such as the internet, intranet, and wireless network, or communicate with other devices via a wireless network. The wireless networks mentioned above include cellular networks, wireless local area networks, or metropolitan area networks. The screen can display interface content and enable data interactions such as displaying molecular characteristic prediction results of speech to be recognized, or recording speech via the screen.

[0162] In some embodiments, the electronic device 100 may also include a peripheral interface 106 and at least one peripheral device. The processor 102, memory 104, and peripheral interface 106 may be connected via a bus or signal lines. Various peripheral devices may be connected to the peripheral interface via a bus, signal lines, or circuit board. Specifically, peripheral devices may include a radio frequency component 108, and so on.

[0163] The peripheral interface 106 is used to connect at least one peripheral device related to I / O (Input / Output) to the processor 102 and memory 104. In some embodiments, the processor 102, memory 104, and peripheral interface 106 are integrated on the same chip or circuit board. In some other embodiments, one or two of the processor 102, memory 104, and peripheral interface 106 may be implemented on separate chips or circuit boards, but this embodiment is not particularly limited.

[0164] The radio frequency component 108 is used to transmit and receive RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency component 108 communicates with communication networks and other communication devices via electromagnetic signals. The radio frequency component 108 converts electrical signals into electromagnetic signals and transmits them, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency component 108 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identification module card, and the like. The radio frequency component 108 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency component 108 may also include, but is not limited to, circuits related to NFC (Near Field Communication).

[0165] In one embodiment of this application, a structural block diagram of a computer-readable storage medium is further provided. This computer-readable medium stores program code that is invoked by a processor to perform the method described in the above embodiment of the method.

[0166] Computer-readable storage media can be electronic memory such as flash memory, EEPROM (electrically erasable programmable read-only memory), EPROM, hard disk, or ROM. Optionally, computer-readable storage media may include non-transitory computer-readable storage media. Computer-readable storage media have storage space for executing program code to perform any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code may be compressed, for example, in an appropriate format.

[0167] Embodiments of this application further provide a computer program product or computer program that includes computer instructions stored on a computer-readable storage medium. The processor of a computer device can cause the computer device to perform the methods described in the various arbitrary embodiments described above by reading and executing computer instructions from the computer-readable storage medium.

[0168] The embodiments described above are not intended to limit the technical methods of this application, but are used solely for illustrative purposes. The present invention will be described in detail with reference to the embodiments described above, but those skilled in the art should understand that the technical methods described in the embodiments described above can be modified or some of the technical features therein can be replaced with equivalents. However, these modifications or replacements will not cause the essence of the corresponding technical methods to deviate from the spirit and scope of the technical methods of each embodiment of this application.

Claims

1. A method for training a feature extraction network performed by an electronic device, In each iteration epoch of the first training phase, the following steps are taken: The steps include: extracting image features from a first sample image using a feature extraction network to be trained; A step of identifying a first loss according to the image features of the first sample image, the category label of the first sample image, and the initial cluster center matrix corresponding to each of the multiple cluster centers within each of the multiple categories, Until the termination condition of the first training phase is met, the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to each of the multiple cluster centers of each category are adjusted according to the first loss, and the steps of obtaining a feature extraction network for preliminary training and a first cluster center matrix corresponding to each of the multiple cluster centers of each category are performed. In each iteration epoch of the second training phase, the following steps are taken: The steps include: extracting image features from a second sample image using a feature extraction network for preliminary training; A step of identifying a second loss according to the image features of a second sample image, the category label of the second sample image, and a first cluster center matrix corresponding to the principal cluster center of each category, wherein the principal cluster center for each category is one of a plurality of cluster centers of that category. A method for training a feature extraction network, comprising the steps of: adjusting the parameters of the pre-training feature extraction network and a first cluster center matrix corresponding to the main cluster centers of each category according to the second loss until the conditions for the end of the second training phase are met, thereby obtaining a trained feature extraction network for image classification.

2. The step of identifying a first loss according to the image features of the first sample image, the category label of the first sample image, and the initial cluster center matrix corresponding to each of the multiple cluster centers within each of the multiple categories, is as follows: For each category, the first similarity is calculated between the image features of the first sample image and the initial cluster center matrix corresponding to each of the multiple cluster centers in that category. The method according to claim 1, comprising the step of determining a first loss based on a first similarity between the image features of the first sample image and an initial cluster center matrix corresponding to a plurality of cluster centers of each category.

3. The step of determining a first loss based on a first similarity between the image features of the first sample image and the initial cluster center matrix corresponding to each of the multiple cluster centers of each category is as follows: The steps include identifying the initial cluster center matrix among multiple initial cluster center matrices for each category that yields the highest first similarity to the image features of the first sample image, as the first reference cluster center matrix, A step of identifying a first sub-loss according to the first reference cluster center matrix and the category labels of the first sample images, For each category, the step of identifying the initial cluster center matrix among multiple initial cluster center matrices for that category that yields the highest first similarity to the image features of the first sample image is to be designated as the second reference cluster center matrix for that category. A step of identifying a second sub-loss according to a first similarity between a second reference cluster center matrix in each of the multiple categories and the image features of the first sample image, The method according to claim 2, comprising the step of identifying a first loss based on the first sub-loss and the second sub-loss.

4. The step of identifying the first loss based on the first sub-loss and the second sub-loss is: A step of identifying a first margin parameter and a first scaling coefficient based on a first epoch number corresponding to the iterative epoch in which the first sample image is involved, wherein the first margin parameter and the first scaling coefficient are negatively correlated with the first epoch number. The method according to claim 3, comprising the step of calculating an added angle margin loss based on the first margin parameter, the first scaling factor, the first sub-loss, and the second sub-loss, and obtaining a first loss.

5. The step of identifying a first margin parameter and a first scaling coefficient based on a first epoch number corresponding to the iterative epoch in which the first sample image is involved is: A step of identifying a first coefficient based on a first epoch number corresponding to the repeating epoch in which the first sample image is involved, wherein the first coefficient is negatively correlated with the first epoch number. The steps include: identifying a first margin adjustment value according to the first coefficient, adding the first margin adjustment value to a first reference margin parameter, and obtaining a first margin parameter; The method according to claim 4, comprising the steps of: identifying a first scaling adjustment value according to the first coefficient; adding the first scaling adjustment value to a first reference scaling coefficient to obtain a first scaling coefficient.

6. If there are multiple first sample images, the method further, before the step of identifying the second loss based on the image features of the second sample image, the category label of the second sample image, and the first cluster center matrix corresponding to the principal cluster centers of each category, the method further, The steps include: identifying a reference image set corresponding to each category, and ensuring that the reference image set corresponding to each category includes a category label and multiple first sample images belonging to the same category; For each category, the steps include identifying the reference similarity corresponding to that cluster center in that category, according to the similarity between the initial cluster center matrix of each cluster center in that category and the image features of each first sample image in the reference image set corresponding to that category, The method according to any one of claims 1 to 5, comprising the step of identifying, for each category, the cluster center that has the highest criterion similarity within that category as the primary cluster center of that category.

7. If there are multiple second sample images, the step of identifying a second loss based on the image features of the second sample images, the category labels of the second sample images, and the first cluster center matrix corresponding to the principal cluster centers of each category is as follows: The steps include: calculating the similarity between the image features of each of the second sample images and the first cluster center matrix corresponding to the main cluster centers of each category, and obtaining a second similarity between the image features of each of the second sample images and the main cluster centers of each category; The steps include identifying, among multiple categories, the category to which the primary cluster center with the highest second similarity to the image features of the second sample image belongs, as the predicted category for the second sample image, The method according to any one of claims 1 to 6, comprising the step of identifying a second loss based on the category label of the second sample image and the predicted category of the second sample image.

8. The step of identifying a second loss based on the category label of the second sample image and the predicted category of the second sample image is as follows: A step of identifying a second margin parameter and a second scaling coefficient based on a second epoch number corresponding to the iterative epoch in which the second sample image is involved, wherein the second margin parameter and the second scaling coefficient are negatively correlated with the second epoch number. The method according to claim 7, comprising the steps of calculating an added angle margin loss to obtain a second loss based on a second margin parameter, a second scaling coefficient, a category label of the second sample image, and a predicted category of the second sample image.

9. The step of determining a second margin parameter and a second scaling coefficient based on a second epoch number corresponding to the iterative epoch in which the second sample image is involved is as follows: A step of identifying a second coefficient based on a second epoch number corresponding to the repeating epoch in which the second sample image is involved, wherein the second coefficient is negatively correlated with the second epoch number. The steps include: identifying a second margin adjustment value according to the second coefficient, adding the second margin adjustment value to the second reference margin parameter, and obtaining a second margin parameter; The method according to claim 8, comprising the steps of: identifying a second scaling adjustment value according to the second coefficient; adding the second scaling adjustment value to a second reference scaling coefficient to obtain a second scaling coefficient.

10. An image classification method performed by an electronic device, Steps to obtain images for classification, A step of extracting features from the classification image and obtaining target image features using a trained feature extraction network according to any one of claims 1 to 9, An image classification method comprising the step of identifying a classification result of the classification image according to the target image features.

11. If the classification result of the classification image includes an authentication result, the step of identifying the classification result of the classification image according to the target image features is: The process involves calculating the similarity between a target image feature and multiple reference image features in a pre-configured database, and obtaining the similarity between the target image feature and each reference image feature. The steps include identifying a target reference image feature that has the greatest similarity to the target image feature, according to the similarity between the target image feature and each reference image feature, The method according to claim 10, comprising the step of using authentication information associated with the target reference image features as the authentication result of the classification image.

12. After the step of using the authentication information associated with the target reference image features as the authentication result of the classification image, the method further: The method according to claim 11, further comprising the step of performing a payment process based on the authentication result of the classification image.

13. If the classification image is a palm print image, the step of acquiring the classification image is: Steps to obtain an image of a hand, The steps include: performing keypoint detection on the hand image to obtain interfinger keypoints within the hand image; The method according to any one of claims 10 to 12, comprising the step of extracting a palm print pixel region from the hand image as the palm print image based on interfinger keypoints in the hand image.

14. A first feature extraction module is configured to perform the step of extracting image features from a first sample image using a feature extraction network to be trained, A first loss identification module is configured to perform the step of identifying a first loss according to the image features of the first sample image, the category label of the first sample image, and an initial cluster center matrix corresponding to multiple cluster centers in each of the multiple categories, A first adjustment module is configured to perform the steps of adjusting the parameters of the feature extraction network to be trained and the initial cluster center matrix corresponding to the multiple cluster centers of each category, according to the first loss, and obtaining a first cluster center matrix corresponding to the feature extraction network for preliminary training and the multiple cluster centers of each category, respectively. A second feature extraction module is configured to perform the step of extracting image features of a second sample image using the aforementioned pre-training feature extraction network, A second loss identification module is configured to perform the step of identifying a second loss according to the image features of the second sample image, the category label of the second sample image, and a first cluster center matrix corresponding to the primary cluster center of each category, wherein the primary cluster center for each category is one of a plurality of cluster centers of that category. A feature extraction network training device comprising: a second adjustment module configured to perform the step of adjusting the parameters of the preliminary training feature extraction network and a first cluster center matrix corresponding to the main cluster centers of each category according to the second loss, thereby obtaining a trained feature extraction network for image classification.

15. An image acquisition module configured to perform the step of acquiring images for classification, A third feature extraction module configured to perform the step of extracting features from the classification image and obtaining target image features using a trained feature extraction network according to any one of claims 1 to 9, An image classification apparatus comprising: a classification result identification module configured to perform the step of identifying the classification result of the classification image according to the target image features;

16. An electronic device comprising one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors to implement the method according to any one of claims 1 to 9 or 10 to 13.

17. A computer-readable storage medium storing program code, wherein the program code is invoked by a processor to perform the method according to any one of claims 1 to 9 or 10 to 13.

18. A computer program product comprising computer instructions stored in the computer-readable storage medium, wherein when the computer instructions are executed, the method described in any one of claims 1 to 9 or 10 to 13 is implemented.

Images