Model training method, image recognition method, device and electronic equipment

Abstract

Embodiments of the present application provide an image recognition method and device, and electronic equipment, and relate to various technical fields such as cloud technology, artificial intelligence, intelligent transportation, and assisted driving. In training a first feature extraction model in m feature extraction models, a vector for representing class centers of n identity categories is trained, and in training the remaining feature extraction models in the m feature extraction models, based on the n class center vectors for representing the class centers of the n identity categories after training, the remaining feature extraction models are trained, so that the sampling results of the m feature extraction models after joint training can perform image recognition on the same to-be-recognized image, and ensure that the accuracy of image recognition is higher than that of image recognition based on the sampling results of a single feature extraction model.

Description

Technical Field

[0001] The embodiments of this application relate to various technical fields such as cloud technology, artificial intelligence, smart transportation, and assisted driving, and more specifically, to model training methods, image recognition methods, devices, and electronic devices. Background Technology

[0002] To date, image recognition systems have been applied in various practical scenarios such as security, payment, and access control.

[0003] In existing image recognition methods, the image recognition system maps the image to be recognized into a feature vector in the feature space, and performs image recognition based on the mapped feature vector.

[0004] However, this image recognition method typically requires the input image to be high-resolution and reliable. For images with blurriness or various types of noise (such as occlusion or large pose), when the image recognition system maps the image to be recognized into a feature vector in the feature space, the resulting feature vector may not be sufficient to accurately represent the image, leading to low accuracy in image recognition. However, in practical applications, the image to be recognized inevitably contains blurriness or various types of noise (such as occlusion or large pose).

[0005] Therefore, there is an urgent need in this field for a model training method to improve the accuracy of image recognition. Summary of the Invention

[0006] This application provides a model training method, an image recognition method, an apparatus, and an electronic device that can improve the accuracy of image recognition.

[0007] In a first aspect, embodiments of this application provide a model training method, including:

[0008] Acquire training images;

[0009] Using the first feature extraction model among m feature extraction models, feature extraction is performed on the training image to obtain the first feature map of the training image; m is an integer greater than 1.

[0010] Using the fully connected mapping units corresponding to the m feature extraction models, the first feature map is transformed to obtain the first feature vector of the training image;

[0011] Based on the class center vector used to represent n identity categories, the first feature vector is converted into n probability values ​​for the training image; these n probability values ​​are used to represent the probability that the training image belongs to the n identity categories; n is an integer greater than 1.

[0012] Based on the n probability values ​​and the true identity category of the training image, a first loss value is calculated to represent the difference between the predicted identity category and the true identity category of the training image;

[0013] Based on the first loss value, the first feature extraction model, the fully connected mapping unit, and the vector used to represent the class centers of the n identity categories are trained to obtain the trained first feature extraction model, the trained fully connected mapping unit, and the trained n class center vectors used to represent the class centers of the n identity categories.

[0014] Based on the training image and the n class center vectors, train the remaining feature extraction model among the m feature extraction models.

[0015] Secondly, embodiments of this application provide an image recognition method, including:

[0016] Acquire the image to be recognized;

[0017] Using m feature extraction models, features are extracted from the image to be identified, resulting in m feature maps of the image; m is an integer greater than 1.

[0018] The m feature maps are transformed to obtain m feature vectors;

[0019] The confidence of the m feature maps is estimated using a confidence estimation model, and the confidence of the m feature maps is obtained; the confidence estimation model is a confidence estimation model trained based on the m feature extraction models;

[0020] Based on the confidence scores of the m feature maps, feature fusion is performed on the m feature vectors to obtain a fused feature vector;

[0021] Image recognition is performed on the image to be identified based on the fused feature vector.

[0022] Thirdly, embodiments of this application provide a model training apparatus, including:

[0023] Image acquisition unit, used to acquire training images;

[0024] The first feature extraction model among m feature extraction models is used to extract features from the training image to obtain the first feature map of the training image; m is an integer greater than 1.

[0025] The fully connected mapping units corresponding to the m feature extraction models are used to transform the first feature map to obtain the first feature vector of the training image;

[0026] A class center unit is used to convert the first feature vector into n probability values ​​of the training image based on the vector representing the class centers of n identity categories; the n probability values ​​are used to represent the probability that the training image belongs to the n identity categories; n is an integer greater than 1;

[0027] The calculation unit is used to calculate a first loss value, representing the difference between the predicted identity category and the real identity category of the training image, based on the n probability values ​​and the real identity category of the training image;

[0028] The first training unit is used to train the first feature extraction model, the fully connected mapping unit, and the vector representing the class centers of the n identity categories based on the first loss value, so as to obtain the trained first feature extraction model, the trained fully connected mapping unit, and the trained n class center vectors representing the class centers of the n identity categories.

[0029] The second training unit is used to train the remaining feature extraction model among the m feature extraction models based on the training image and the n class center vectors.

[0030] Fourthly, embodiments of this application provide an image recognition device, including:

[0031] An image acquisition unit is used to acquire the image to be recognized.

[0032] m feature extraction models are used to extract features from the image to be identified, resulting in m feature maps of the image to be identified; m is an integer greater than 1.

[0033] A fully connected mapping unit is used to transform the m feature maps to obtain m feature vectors;

[0034] A confidence estimation model is used to estimate the confidence of the m feature maps, thus obtaining the confidence of the m feature maps; this confidence estimation model is a confidence estimation model trained based on the m feature extraction models.

[0035] The feature fusion unit is used to fuse the m feature vectors based on the confidence of the m feature maps to obtain a fused feature vector;

[0036] The recognition unit is used to perform image recognition on the image to be recognized based on the fused feature vector.

[0037] Fifthly, this application provides an electronic device, comprising:

[0038] Processor, adapted to implement computer instructions; and,

[0039] A computer-readable storage medium storing computer instructions adapted for loading by a processor and executing the method of the first to second aspects or their respective implementations described above.

[0040] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when read and executed by a processor of a computer device, cause the computer device to perform the methods described in the first to second aspects or their respective implementations.

[0041] In a seventh aspect, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods described in the first to second aspects or their respective implementations.

[0042] Based on the above technical solution, when training the first feature extraction model among m feature extraction models, the vectors representing the class centers of n identity categories are trained. When training the remaining feature extraction models among the m feature extraction models, the remaining feature extraction models are trained based on the trained n class center vectors representing the class centers of the n identity categories. In other words, when training the m feature extraction models, the method provided in this application ensures that the identity categories targeted by the m feature extraction models are consistent. Based on this, the sampling results of the jointly trained m feature extraction models can perform image recognition on the same image to be recognized, and ensure that its image recognition accuracy is higher than the accuracy of image recognition based on the sampling results of a single feature extraction model. Furthermore, this application trains the m feature extraction models using the same training image, eliminating the need for new training images and reducing the cost of collecting training data. Attached Figure Description

[0043] Figure 1 This is an example of the system framework provided in the embodiments of this application.

[0044] Figure 2 This is a schematic diagram of the model training stage and image recognition stage provided in the embodiments of this application.

[0045] Figure 3 This is a schematic flowchart of the model training method provided in the embodiments of this application.

[0046] Figure 4 This is a schematic flowchart illustrating the training of the first feature extraction model and the fully connected mapping unit provided in the embodiments of this application.

[0047] Figure 5 This is a schematic flowchart of the training residual feature extraction model provided in the embodiments of this application.

[0048] Figure 6 This is a schematic flowchart of the training confidence estimation model provided in the embodiments of this application.

[0049] Figure 7 This is a schematic flowchart of the image recognition method provided in the embodiments of this application.

[0050] Figure 8 This is another illustrative flowchart of the image recognition method provided in the embodiments of this application.

[0051] Figure 9 This is a schematic block diagram of the model training device provided in the embodiments of this application.

[0052] Figure 10 This is a schematic block diagram of the image recognition device provided in the embodiments of this application.

[0053] Figure 11 This is a schematic block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0054] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0055] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0056] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0057] The following is a description and explanation of the relevant terms and technical fields involved in the embodiments of this application:

[0058] This application relates to the field of artificial intelligence (AI). AI is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0059] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0060] Computer Vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and further processes images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0061] Machine Learning (ML) is an interdisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0062] This application may also relate to the transportation sector.

[0063] For example, this application may relate to Intelligent Traffic Systems (ITS), also known as Intelligent Transportation Systems, which effectively integrate advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) into transportation, service control, and vehicle manufacturing, strengthening the connection between vehicles, roads, and users, thereby forming a comprehensive transportation system that ensures safety, improves efficiency, improves the environment, and saves energy. Alternatively;

[0064] For example, this application may relate to Intelligent Vehicle Infrastructure Cooperative Systems (IVICS), which is a development direction of Intelligent Transportation Systems (ITS). IVICS utilizes advanced wireless communication and next-generation Internet technologies to implement comprehensive, real-time dynamic information interaction between vehicles and infrastructure. Based on the collection and fusion of dynamic traffic information across all times and spaces, it conducts active vehicle safety control and cooperative road management, fully realizing effective collaboration between people, vehicles, and roads, ensuring traffic safety, and improving traffic efficiency, thereby forming a safe, efficient, and environmentally friendly road traffic system.

[0065] The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0066] Figure 1 This is an example of the system framework 100 provided in the embodiments of this application.

[0067] like Figure 1As shown, the system framework 100 can be an application system, and this application embodiment does not limit the specific type of the application. The system framework 100 includes: terminal 131, terminal 132, and server cluster 110. Both terminal 131 and terminal 132 can be connected to server cluster 110 via wireless or wired network 120.

[0068] In one implementation, terminals 131 and 132 can be at least one of a smartphone, game console, desktop computer, tablet computer, e-book reader, MP4 player, and laptop computer. In another implementation, terminals 131 and 132 include, but are not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft. Terminals 131 and 132 have applications installed and running. These applications can be any of the following: online video programs, short video programs, image sharing programs, voice social programs, animation programs, wallpaper programs, news push programs, supply and demand information push programs, academic exchange programs, technical exchange programs, policy exchange programs, programs with comment mechanisms, programs with opinion posting mechanisms, and knowledge sharing programs. Terminals 131 and 132 can be terminals used by users 141 and 142 respectively, and user accounts are logged into the applications running on terminals 131 and 132.

[0069] Server cluster 110 includes at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. Server cluster 110 provides background services for applications (e.g., applications on terminals 131 and 132). Optionally, server cluster 110 undertakes the primary computing task, while terminals 131 and 132 undertake secondary computing tasks; or, server cluster 110 undertakes secondary computing tasks, while terminals 131 and 132 undertake the primary computing tasks; or, terminals 131 and 132 collaborate with server cluster 110 using a distributed computing architecture.

[0070] Optionally, taking a web browsing system as an example, the server cluster 110 includes: an access server 112, a web server 111, and a data server 113. There can be one or more access servers 112, which can be deployed in different cities nearby. Access servers 112 receive service requests from terminals 131 and 132 and forward them to the corresponding servers for processing. Web server 111 provides web pages to terminals 131 and 132, which integrate embedded tracking code. Data server 113 receives data (user interaction data, business data, etc.) reported by terminals 131 and 132.

[0071] It should be noted that the system framework 100 is equipped with an image recognition system. For example, terminal 131, terminal 132, or server cluster 110 are equipped with an image recognition system. The image recognition system maps the image to be recognized into a feature vector in a feature space and performs image recognition based on the mapped feature vector.

[0072] However, such image recognition methods typically require high-resolution and reliable input images to be recognized. For images with blurriness or various types of noise (such as occlusion or large pose), when the image recognition system maps the image to be recognized into a feature vector in the feature space, the resulting feature vector may not be sufficient to accurately represent the image, leading to low image recognition accuracy. However, in practical applications, the images to be recognized inevitably contain blurriness or various types of noise (such as occlusion or large pose). Therefore, this application provides a model training method that can improve the accuracy of image recognition. It should be understood that this application can be applied to any image recognition system, without being limited by the application scenario or product.

[0073] Specifically, in the model training method provided in this application embodiment, m feature extraction models (m>1) that can be trained to extract features from the same image can be obtained. Correspondingly, in the image recognition process, the m feature extraction models can be used to sample or extract features from the image to be recognized in the feature space to obtain m feature maps. Then, the confidence of the obtained m feature maps is analyzed, and the confidence of the m feature maps is used to fuse the m feature vectors obtained after dimensional transformation of the feature maps. Finally, the fused feature vectors are used for image recognition, thereby improving the accuracy of image recognition.

[0074] Figure 2 This is a schematic diagram of the model training stage and image recognition stage provided in the embodiments of this application.

[0075] like Figure 2 As shown, in the module training phase, firstly, the first feature extraction model and the fully connected mapping unit corresponding to the m feature extraction models are initialized and trained; then, the remaining feature extraction models among the m feature extraction models are trained based on the training results of the first feature extraction model; next, with the hyperparameters of the m feature extraction models and the fully connected mapping unit fixed, their outputs are used as the input of the confidence estimation model to train the confidence estimation model.

[0076] It is worth noting that when training the confidence estimation model, you can train only the hyperparameters of the confidence estimation model without updating the hyperparameters of the m feature extraction models and the fully connected mapping unit.

[0077] After training the m feature extraction models, fully connected mapping units, and confidence estimation models, the image recognition stage can begin, which is the deployment and operation stage for the m feature extraction models, fully connected mapping units, and confidence estimation models.

[0078] In the image recognition stage, firstly, the image to be recognized can be acquired by the image acquisition unit. Then, the image to be recognized is subjected to feature extraction or feature sampling by the m feature extraction models to obtain m feature maps. Next, the confidence of the m feature maps can be estimated by the confidence estimation model to obtain m confidence scores. Then, the m feature vectors transformed by the m feature maps through the fully connected mapping unit are fused by the feature fusion unit to obtain a fused feature vector. Finally, the fused feature vector is recognized by the recognition unit to realize the recognition of the image to be recognized.

[0079] It should be understood that the hyperparameters involved in this application can be parameters determined through learning.

[0080] Figure 3 A schematic flowchart of a model training method 200 according to an embodiment of this application is shown. This model training method 200 can be executed by any electronic device with data processing capabilities. For example, the electronic device can be implemented as a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations. For ease of description, the model training method provided in this application is described below using a model training device as an example.

[0081] like Figure 3 As shown, the method 200 may include some or all of the following:

[0082] S210, acquire training images.

[0083] S220, using the first feature extraction model among m feature extraction models, perform feature extraction on the training image to obtain the first feature map of the training image; m is an integer greater than 1.

[0084] For example, the m feature extraction models can be feature extraction models used to extract features from different dimensions. For instance, the m feature extraction models can be used to extract feature maps of the same dimension from different dimensions.

[0085] For example, the m feature extraction models can be feature extraction models used to extract features from different scenes (pose, lighting, occlusion, etc.). For instance, the m feature extraction models can be used to extract feature maps of the same dimension from different scenes (pose, lighting, occlusion, etc.).

[0086] S230, using the fully connected mapping units corresponding to the m feature extraction models, the first feature map is transformed to obtain the first feature vector of the training image.

[0087] For example, the fully connected mapping unit corresponding to the m feature extraction models can be one or more. For instance, each of the m feature extraction models can correspond to one fully connected mapping unit. Alternatively, the m feature extraction models can correspond to the same fully connected mapping unit.

[0088] S240, based on the vector representing the class centers of n identity categories, the first feature vector is converted into n probability values ​​of the training image; the n probability values ​​are used to represent the probability that the training image belongs to the n identity categories; n is an integer greater than 1.

[0089] S250, based on the n probability values ​​and the true identity category of the training image, calculate a first loss value to represent the difference between the predicted identity category and the true identity category of the training image.

[0090] S260, Based on the first loss value, the first feature extraction model, the fully connected mapping unit, and the vector used to represent the class centers of the n identity categories are trained to obtain the trained first feature extraction model, the trained fully connected mapping unit, and the trained n class center vectors used to represent the class centers of the n identity categories.

[0091] S270, Based on the training image and the n class center vectors, train the remaining feature extraction model among the m feature extraction models.

[0092] For example, if each of the m feature extraction models corresponds to a fully connected mapping unit, then based on the training image and the n class center vectors, the remaining feature extraction models and their corresponding fully connected mapping units can be trained. If the m feature extraction models correspond to the same fully connected mapping unit, then based on the training image and the n class center vectors, only the remaining feature extraction models among the m feature extraction models can be trained.

[0093] In this embodiment, when training the first feature extraction model among the m feature extraction models, the vectors representing the class centers of the n identity categories are trained. When training the remaining feature extraction models among the m feature extraction models, the remaining feature extraction models are trained based on the trained n class center vectors representing the class centers of the n identity categories. Essentially, when training the m feature extraction models, the method provided in this application ensures that the identity categories targeted by the m feature extraction models are consistent. Based on this, the sampling results of the jointly trained m feature extraction models can perform image recognition on the same image to be recognized, and ensure that its image recognition accuracy is higher than the accuracy of image recognition based on the sampling results of a single feature extraction model. Furthermore, this application trains the m feature extraction models using the same training image, eliminating the need for new training images and reducing the cost of collecting training data.

[0094] In some embodiments, S270 may include:

[0095] While keeping the n class center vectors unchanged, the remaining feature extraction model is trained based on the training image.

[0096] In this embodiment, while keeping the n class center vectors unchanged, the remaining feature extraction model is trained based on the training image. This ensures that the identity category targeted by the remaining feature extraction model during training remains consistent with the identity category targeted by the first feature extraction model. Therefore, the sampling results of the jointly trained m feature extraction models can perform image recognition on the same image to be identified, targeting the same identity category, and guarantee that its image recognition accuracy is higher than that of image recognition based on the sampling results of a single feature extraction model. Furthermore, this application trains m feature extraction models using the same training image, eliminating the need for new training images and reducing the cost of collecting training data.

[0097] It is worth noting that the n class center vectors involved in this application can essentially be a weight matrix. Specifically, during the prediction process, its role is to serve as the weights of the feature vectors obtained by each of the m feature extraction models, multiplying them by the feature vectors obtained by each feature extraction model to output the corresponding n probability values. Since these n class center vectors are obtained during the training of the first feature extraction model, and the trained n class center vectors are kept unchanged during the training of the remaining feature extraction models, it can be ensured that the weights of the feature vectors obtained by each feature extraction model remain unchanged. Therefore, the trained m feature extraction models can use the same weights to convert the extracted feature vectors into n probability values, thus ensuring that the identity category targeted by the remaining feature extraction models during the training process remains consistent with the identity category targeted by the first feature extraction model.

[0098] In some embodiments, the initial values ​​of the hyperparameters of the residual feature extraction model are different from the initial values ​​of the hyperparameters of the first feature extraction model, and / or the hyperparameters in the loss function used by the first feature training model are different from the hyperparameters in the loss function used by the residual feature extraction model.

[0099] In this embodiment, the initial values ​​of the hyperparameters of the remaining feature extraction model are designed to be different from the initial values ​​of the hyperparameters of the first feature extraction model. This ensures that the m feature extraction models are feature extraction models with different hyperparameters, thereby increasing the diversity of sampling results obtained by the m feature extraction models after sampling the same image, which is beneficial to improving image recognition performance. Furthermore, designing the hyperparameters in the loss function used by the first feature training model to be different from the hyperparameters in the loss function used by the remaining feature extraction models ensures the diversity of the m feature extraction models, thereby increasing the diversity of sampling results obtained by the m feature extraction models after sampling the same image, which is beneficial to improving image recognition performance.

[0100] In some embodiments, the first feature map is a feature map distributed in a hyperspherical space.

[0101] In this embodiment, since the hyperspherical space is more consistent with the feature space of a human face, the first feature map is designed to be a feature map distributed in the hyperspherical space, which is beneficial to improving the image recognition effect and universality.

[0102] In some embodiments, S240 may include:

[0103] If the dimension of each vector in the vector representing the class centers of the n identity categories is the same as the dimension of the first feature vector, then the matrix formed by the vector representing the class centers of the n identity categories is obtained, and the value obtained by multiplying the first feature vector and the matrix formed by the vector representing the class centers of the n identity categories is determined as the n probability values.

[0104] Of course, other methods can be used to determine the n probability values, and this application does not impose specific limitations on this.

[0105] Figure 4 This is a schematic flowchart illustrating the training of the first feature extraction model and the fully connected mapping unit provided in the embodiments of this application.

[0106] like Figure 4 As shown, the training process of the first feature extraction model and the fully connected mapping unit involves units and models including but not limited to: image acquisition unit, first feature extraction model, fully connected mapping unit, class center unit, computation unit, termination of training judgment unit, and optimization unit.

[0107] The following section provides an illustrative example of the functions of each model or unit in conjunction with the training process.

[0108] Image acquisition unit: This unit is used to acquire training images and send them to the first feature extraction model. For example, during training, it can be used to read training data, combine the read data into a batch, and send it to the first feature extraction model so that the first feature extraction model can extract features from the training images in the input batch.

[0109] The first feature extraction model is used to extract or sample spatial features from the training image and output a first feature map of the training image, which retains the spatial structure information of the image. This first feature extraction model typically has a convolutional neural network (CNN) structure. For example, this first feature extraction model may include operations such as convolution calculation, ReLU nonlinear activation function calculation, and pooling calculation.

[0110] Fully connected mapping unit: It can be used to map the first feature map extracted by the first feature extraction model into the first feature vector, which can be denoted as u.

[0111] The class center unit stores data consisting of vectors representing the class centers of n identity (ID) categories. These vectors have an shape of n×s, where n represents the class center of each identity (ID) category, and s represents the vector representing each class center. Optionally, the dimension of u is the same as the dimension of s. In this case, performing matrix operations between the vectors representing the class centers of the n identity categories stored in the class center unit and the multidimensional vector u output by the fully connected mapping unit yields the probability values ​​of each image belonging to each category, i.e., n probability values.

[0112] The computation unit takes the n probability values ​​output by the class center unit and the label information of the training images (i.e., information used to determine the true identity category of the training images) as input to calculate the first loss value. The objective function for this first loss value can be various classification functions, such as, but not limited to: the normalization exponent (softmax) function, various types of margin-added softmax functions, or other types of functions.

[0113] Training Termination Judgment Unit: This unit determines whether to terminate the training of the first feature extraction model and the fully connected mapping unit. Specifically, training of the first feature extraction model and the fully connected mapping unit is terminated when the number of calculations of the first loss value meets a set threshold or when the first loss value is less than or equal to a set function value threshold.

[0114] Optimization unit: When the termination training decision unit determines that training of the first feature extraction model and the fully connected mapping unit should not be terminated, it adjusts the first feature map and the first feature vector using gradient descent and recalculates the first loss value. For example, the gradient descent method includes, but is not limited to, stochastic gradient descent and stochastic gradient descent with a driving term.

[0115] It is worth noting that during the training of the first feature extraction model and the fully connected mapping unit, the initial vectors stored in the class center unit for representing the class centers of the n identity categories are also trained to obtain vectors that can be used to accurately represent the class centers of the n identity categories. For example, the initial vectors stored in the class center unit for representing the class centers of the n identity categories can be default vectors, randomly set vectors, or user-set vectors; this embodiment does not specifically limit these.

[0116] Figure 5 This is a schematic flowchart of the training residual feature extraction model provided in the embodiments of this application.

[0117] like Figure 5As shown, the training process of the residual feature extraction model involves the following unit models: image acquisition unit, residual feature extraction model, fully connected mapping unit, class center unit, computation unit, termination of training judgment unit, optimization unit, random seed control unit, and hyperparameter control unit.

[0118] in, Figure 5 The specific functions of the image acquisition unit, residual feature extraction model, fully connected mapping unit, class center unit, computation unit, training termination judgment unit, and optimization unit can be found in the following references. Figure 4 The functions of the image acquisition unit, first feature extraction model, fully connected mapping unit, class center unit, calculation unit, training termination judgment unit, and optimization unit are not described here to avoid repetition.

[0119] It should be noted that this application uses the trained vectors representing the class centers of each identity category obtained after training the first feature extraction model to train the remaining feature extraction model. For example, during the training process of the remaining feature extraction model, the trained vectors representing the class centers of n identity categories obtained after training the first feature extraction model are initialized to the class center vectors representing the n identity categories stored in the class center unit; and while keeping the n class center vectors unchanged, the remaining feature extraction model is trained based on the training image.

[0120] Based on this, it can be guaranteed that the feature space of the feature map obtained by using the remaining feature extraction model is the same as the feature space of the feature map obtained by using the first feature extraction model. That is, it can be guaranteed that the identity categories targeted by the m feature extraction models are consistent, so that the sampling results of the jointly trained m feature extraction models can perform image recognition on the same image to be recognized, and ensure that the accuracy of its image recognition is higher than the accuracy of image recognition based on the sampling results of a single feature extraction model.

[0121] Furthermore, compared to the training process for the first feature extraction model, the training process for the remaining feature extraction models introduces a random seed control unit and a hyperparameter control unit. The random seed control unit ensures that the initial values ​​of the hyperparameters of the remaining feature extraction models differ from those of the first feature extraction model. This guarantees that the m feature extraction models have different hyperparameters, thereby increasing the diversity of sampling results obtained from the same image by the m feature extraction models, which is beneficial for improving image recognition performance. The hyperparameter control unit ensures that the hyperparameters in the loss function used by the first feature extraction model are different from those in the loss function used by the remaining feature extraction models. This also guarantees the diversity of the m feature extraction models, further increasing the diversity of sampling results obtained from the same image by the m feature extraction models, which is beneficial for improving image recognition performance.

[0122] In some embodiments, the method 200 may further include:

[0123] Obtain the first class center vector from the n class center vectors that represents the true identity category of the training image;

[0124] The confidence level of the first feature map is estimated using a confidence estimation model.

[0125] Based on the first class center vector, the confidence of the first feature map, and the first feature vector, a second loss value is calculated to express the accuracy of the predicted identity category of the training image in representing the true identity category.

[0126] The confidence estimation model is trained based on this second loss value.

[0127] In some embodiments, the negative of the product of the ratio of the confidence level of the first feature map to the radius of the hypersphere where the first feature map is located, the first feature vector, and the first class center vector is determined as the first value; the negative of the product of the second value and the logarithm of the confidence level of the first feature map is determined as the fifth value; the second value is the value obtained by subtracting the fourth value from the ratio of the dimension of the first feature vector to the third value; the logarithm of the Bessel function with the confidence level of the first feature map as the independent variable and the second value as the order is determined as the sixth value; the product of the ratio of the dimension of the first feature vector to the third value and the logarithm of the seventh value is determined as the eighth value; the seventh value is the product of the third value, pi, and the square of the radius of the hypersphere; the sum of the first value, the fifth value, the sixth value, and the eighth value is determined as the second loss value.

[0128] For example, the formula for calculating the second loss value is as follows:

[0129]

[0130] Where x represents the training image input to the first feature extraction model, k(x) represents the confidence level of the first feature map, u(x) represents the first feature vector, d represents the dimension of the first feature vector, r represents the radius of the hypersphere space in which the first feature map resides, and w x∈c Let I represent the center vector of the first kind, I be the Bessel function, and π represent pi.

[0131] In some embodiments, gradient descent is used to adjust the confidence of the first feature map and the first feature vector, and the second loss value is recalculated until the number of calculations of the second loss value meets a set threshold or the second loss function value is less than or equal to a set function value threshold, thereby obtaining the trained confidence estimation model.

[0132] For example, the confidence level of the first feature map and the first feature vector can be adjusted using the following formula:

[0133]

[0134] Where x represents the training image input to the first feature extraction model, k(x) represents the confidence level of the first feature map, u(x) represents the first feature vector, d represents the dimension of the first feature vector, r represents the radius of the hypersphere space in which the first feature map resides, and w x∈c Let I represent the center vector of the first kind, I be the Bessel function, and π represent pi.

[0135] Figure 6 This is a schematic flowchart of the training confidence estimation model provided in the embodiments of this application.

[0136] like Figure 6 As shown, the training process for the confidence estimation model involves units and models including, but not limited to: an image acquisition unit, a first feature extraction model, a fully connected mapping unit, a confidence estimation model, a computation unit, a termination training judgment unit, and an optimization unit. That is, the confidence estimation model needs to be trained by combining the m pre-trained feature extraction models and the fully connected mapping unit. It is worth noting that when training the confidence estimation model, only the hyperparameters of the confidence estimation model can be trained without updating the hyperparameters in the m feature extraction models and the fully connected mapping unit.

[0137] The following section provides an illustrative example of the functions of each model or unit in conjunction with the training process.

[0138] Image acquisition unit: This unit is used to acquire training images and send them to the first feature extraction model. For example, during training, it can be used to read training data, combine the read data into a batch, and send it to the first feature extraction model so that the first feature extraction model can extract features from the training images in the input batch.

[0139] The first feature extraction model is used to extract or sample spatial features from the training image and output a first feature map of the training image, which retains the spatial structure information of the image. This first feature extraction model typically has a convolutional neural network (CNN) structure. For example, this first feature extraction model may include operations such as convolution calculation, ReLU nonlinear activation function calculation, and pooling calculation.

[0140] Fully connected mapping unit: It can be used to map the first feature map extracted by the first feature extraction model into the first feature vector, which can be denoted as u.

[0141] A confidence estimation model is provided, which can be used to estimate the confidence level k of a first feature map. The confidence estimation model has a CNN structure. For example, the first feature extraction model may contain several fully connected layers of a residual (RESNET) neural network.

[0142] The computational unit is used to calculate a second loss value, based on the first class center vector, the confidence level of the first feature map, and the first feature vector, to assess the accuracy of the predicted identity category of the training image in representing the true identity category. The first class center vector is the class center vector among the n class center vectors used to characterize the true identity category of the training image, representing the class center of the c-th class sample. The first class center vector can be calculated by averaging all feature vectors in the first category, or by determining the first class center vector based on the classification weights of the trained feature extraction models corresponding to the first category among the m feature extraction models.

[0143] For example, the specific calculation formula is as follows:

[0144]

[0145] Where x represents the training image input to the first feature extraction model, k(x) represents the confidence level of the first feature map, u(x) represents the first feature vector, d represents the dimension of the first feature vector, r represents the radius of the hypersphere space in which the first feature map resides, and w x∈c Let I represent the center vector of the first kind, I be the Bessel function, and π represent pi.

[0146] Training Termination Judgment Unit: This unit is used to determine whether to terminate the training of the confidence estimation model. Specifically, training of the confidence estimation model is terminated when the number of calculations of the second loss value meets a set threshold or when the second loss value is less than or equal to a set function value threshold.

[0147] Optimization unit: When the termination training decision unit determines not to terminate the training of the confidence estimation model, it adjusts the confidence of the first feature map and the first feature vector using gradient descent, and recalculates the first loss value. For example, the gradient descent method includes, but is not limited to, stochastic gradient descent and stochastic gradient descent with a driving term.

[0148] For example, the confidence level of the first feature map and the first feature vector can be adjusted using the following formula:

[0149]

[0150] Where x represents the training image input to the first feature extraction model, k(x) represents the confidence level of the first feature map, u(x) represents the first feature vector, d represents the dimension of the first feature vector, r represents the radius of the hypersphere space in which the first feature map resides, and w x∈c Let I represent the center vector of the first kind, I be the Bessel function, and π represent pi.

[0151] Figure 7 This is a schematic flowchart of the image recognition method 300 provided in this application embodiment. The image recognition method 300 can be executed by any electronic device with data processing capabilities. For example, the electronic device can be implemented as a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations. For ease of description, the image recognition method provided in this application will be described below using an image recognition device as an example.

[0152] like Figure 7 As shown, the image recognition method 300 may include:

[0153] S310, acquire the image to be recognized;

[0154] S320: Using m feature extraction models, perform feature extraction on the image to be identified to obtain m feature maps of the image to be identified; m is an integer greater than 1.

[0155] S330, using the fully connected mapping units corresponding to the m feature extraction models, the m feature maps are transformed to obtain m feature vectors;

[0156] S340, Using the confidence estimation model, the confidence of the m feature maps is estimated to obtain the confidence of the m feature maps; the confidence estimation model is a confidence estimation model trained based on the m feature extraction models;

[0157] S350, Based on the confidence of the m feature maps, feature fusion is performed on the m feature vectors to obtain the fused feature vector;

[0158] S360 performs image recognition on the image to be recognized based on the fused feature vector.

[0159] In this embodiment, the m feature extraction models can be used to perform image recognition on the same image to be recognized. Compared with image recognition based on the sampling results of a single feature extraction model, image recognition based on the sampling results of m feature extraction models can increase the sampling rate of the image to be recognized without increasing the number of images to be recognized. This can solve the problem that the feature vector obtained when the image to be recognized is mapped to a feature vector has a certain probability of being insufficient in expression or capability, thereby improving the accuracy of image recognition.

[0160] Furthermore, by introducing the confidence levels of m feature maps, this application can quantify the ability or accuracy of the m feature maps to represent the image to be identified. Based on the confidence levels of the m feature maps, feature fusion is performed on the m feature vectors, which makes the distribution of the image to be identified in the feature space as close as possible to the actual distribution, thereby ensuring the ability or accuracy of the fused feature vectors to represent the image to be identified. As a result, when performing image recognition on the image to be identified based on the fused feature vectors, the accuracy of image recognition can be improved.

[0161] In addition, this application introduces the confidence scores of m feature maps to perform overall fusion of n feature vectors, and performs separate weighted fusion of features of each dimension of the n feature vectors. This can reduce the estimation error between the weighted fused features and the true weights, which is beneficial to improving the accuracy of image recognition.

[0162] In particular, the image recognition method provided in this application can improve the recognition accuracy of the image recognition system in various complex scenarios and reduce its false recognition alarm rate.

[0163] In some embodiments, S350 may include:

[0164] The confidence scores of the m feature maps are determined as the weights of the m feature vectors;

[0165] Based on the weights of the m feature vectors, a weighted average is calculated on the m feature vectors to obtain the fused feature vector.

[0166] Figure 8 This is another illustrative flowchart of the image recognition method provided in the embodiments of this application.

[0167] like Figure 8 As shown, the units and models involved in this image recognition method include, but are not limited to: m feature extraction models, confidence estimation models, fully connected mapping units, feature fusion units, similarity calculation units, and threshold judgment units.

[0168] The following section provides an illustrative explanation of the functions of each model or unit, using the process of image recognition methods as an example.

[0169] m feature extraction models: These are used to extract features from the image to be recognized to obtain m feature maps; and these m feature maps are then input into the confidence estimation model and the fully connected mapping unit, respectively.

[0170] Confidence estimation model: It is used to estimate the confidence of the m feature maps to obtain the confidence of the m feature maps.

[0171] Fully connected mapping unit: It is used to map the m feature maps into m feature vectors.

[0172] Feature fusion unit: It is used to perform feature fusion on the m feature vectors based on the confidence of the m feature maps to obtain a fused feature vector, and input the fused feature vector into the similarity calculation unit.

[0173] For example, the m feature vectors can be fused using the following formula:

[0174]

[0175] Where z is the fused feature vector, k m u represents the confidence level of the m feature maps. m This represents the m eigenvectors.

[0176] Similarity calculation unit: This unit calculates the similarity between the fused feature vector and features in the registry. For example, cosine similarity or Euclidean distance can be used as an equal distance metric to output the similarity between the fused feature vector and features in the registry.

[0177] Threshold judgment unit: It is used to determine whether two images are from the same person based on the similarity output by the similarity calculation unit.

[0178] For example, the output of the threshold determination unit can be determined by the following formula:

[0179]

[0180] Wherein, the threshold Lout represents the output of the threshold judgment unit, and th represents the threshold, the specific value of which can be determined according to the false alarm rate required in the actual application scenario.

[0181] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application.

[0182] It should also be understood that, in the various method embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0183] The methods provided in the embodiments of this application have been described above. The apparatus provided in the embodiments of this application will be described below.

[0184] Figure 9 This is a schematic block diagram of the model training device 400 provided in the embodiments of this application.

[0185] like Figure 9 As shown, the model training device 400 may include:

[0186] Image acquisition unit 410 is used to acquire training images;

[0187] The first feature extraction model 420 among the m feature extraction models is used to extract features from the training image to obtain the first feature map of the training image; m is an integer greater than 1.

[0188] The fully connected mapping unit 430 corresponding to the m feature extraction models is used to transform the first feature map to obtain the first feature vector of the training image;

[0189] The class center unit 440 is used to convert the first feature vector into n probability values ​​of the training image based on the vector representing the class centers of n identity categories; the n probability values ​​are used to represent the probability that the training image belongs to the n identity categories; n is an integer greater than 1;

[0190] The calculation unit 450 is used to calculate a first loss value, representing the difference between the predicted identity category of the training image and the real identity category of the training image, based on the n probability values ​​and the real identity category of the training image;

[0191] The first training unit 460 is used to train the first feature extraction model, the fully connected mapping unit, and the vector representing the class centers of the n identity categories based on the first loss value, so as to obtain the trained first feature extraction model, the trained fully connected mapping unit, and the trained n class center vectors representing the class centers of the n identity categories.

[0192] The second training unit 470 is used to train the remaining feature extraction model among the m feature extraction models based on the training image and the n class center vectors.

[0193] In some embodiments, the second training unit 470 is specifically used for:

[0194] While keeping the n class center vectors unchanged, the remaining feature extraction model is trained based on the training image.

[0195] In some embodiments, the initial values ​​of the hyperparameters of the residual feature extraction model are different from the initial values ​​of the hyperparameters of the first feature extraction model, and / or the hyperparameters in the loss function used by the first feature training model are different from the hyperparameters in the loss function used by the residual feature extraction model.

[0196] In some embodiments, the first feature map is a feature map distributed in a hyperspherical space.

[0197] In some embodiments, the class central unit 440 is specifically used for:

[0198] If the dimension of each vector in the vector representing the class centers of the n identity categories is the same as the dimension of the first feature vector, then the matrix formed by the vector representing the class centers of the n identity categories is obtained, and the value obtained by multiplying the first feature vector and the matrix formed by the vector representing the class centers of the n identity categories is determined as the n probability values.

[0199] In some embodiments, the second training unit 470 may also be used for:

[0200] Obtain the first class center vector from the n class center vectors that represents the true identity category of the training image;

[0201] The confidence level of the first feature map is estimated using a confidence estimation model.

[0202] Based on the first class center vector, the confidence of the first feature map, and the first feature vector, a second loss value is calculated to express the accuracy of the predicted identity category of the training image in representing the true identity category.

[0203] The confidence estimation model is trained based on this second loss value.

[0204] In some embodiments, the second training unit 470 is specifically used for:

[0205] The negative of the product of the ratio of the confidence level of the first feature map to the radius of the hypersphere where the first feature map is located, the first feature vector, and the first class center vector is determined as the first value; the negative of the product of the second value and the logarithm of the confidence level of the first feature map is determined as the fifth value; the second value is the value obtained by subtracting the fourth value from the ratio of the dimension of the first feature vector to the third value; the logarithm of the Bessel function with the confidence level of the first feature map as the independent variable and the second value as the order is determined as the sixth value; the product of the ratio of the dimension of the first feature vector to the third value and the logarithm of the seventh value is determined as the eighth value; the seventh value is the product of the third value, pi, and the square of the radius of the hypersphere; the sum of the first value, the fifth value, the sixth value, and the eighth value is determined as the second loss value.

[0206] In some embodiments, the second training unit 470 is specifically used for:

[0207] Using gradient descent, the confidence of the first feature map and the first feature vector are adjusted, and the second loss value is recalculated until the number of calculations of the second loss value meets the set threshold or the value of the second loss function is less than or equal to the set threshold, thus obtaining the trained confidence estimation model.

[0208] Figure 10 This is a schematic block diagram of the image recognition device 500 provided in the embodiments of this application.

[0209] like Figure 10 As shown, the image recognition device 500 may include:

[0210] Image acquisition unit 510 is used to acquire the image to be recognized;

[0211] m feature extraction models 520 are used to extract features from the image to be identified, resulting in m feature maps of the image to be identified; m is an integer greater than 1.

[0212] The fully connected mapping unit 530 is used to transform the m feature maps to obtain m feature vectors;

[0213] Confidence estimation model 540 is used to estimate the confidence of the m feature maps to obtain the confidence of the m feature maps; the confidence estimation model is based on the confidence estimation model trained by the m feature extraction models;

[0214] The feature fusion unit 550 is used to fuse the m feature vectors based on the confidence of the m feature maps to obtain a fused feature vector;

[0215] The recognition unit 560 is used to perform image recognition on the image to be recognized based on the fused feature vector.

[0216] In some embodiments, the feature fusion unit 550 is specifically used for:

[0217] The confidence scores of the m feature maps are determined as the weights of the m feature vectors;

[0218] Based on the weights of the m feature vectors, a weighted average is calculated on the m feature vectors to obtain the fused feature vector.

[0219] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details are omitted here. Specifically, the model training device 400 can correspond to the corresponding subject in executing the method 200 of this application embodiment, and each unit in the model training device 400 implements the corresponding process in method 200; similarly, the image recognition device 500 can correspond to the corresponding subject in executing the method 300 of this application embodiment, and each unit in the image recognition device 500 implements the corresponding process in method 300. For simplicity, further details are omitted here.

[0220] It should also be understood that the various units in the apparatus 400 or 500 involved in the embodiments of this application can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This can achieve the same operation without affecting the technical effect of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the apparatus 400 or 500 may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by multiple units working together. According to another embodiment of this application, the apparatus 400 or 500 involved in the embodiments of this application can be constructed by running a computer program (including program code) capable of performing the steps involved in the corresponding method on a general-purpose computing device including processing elements and storage elements such as a central processing unit (CPU), random access storage medium (RAM), and read-only storage medium (ROM), and the model training method or image recognition method provided in the embodiments of this application can be implemented. Computer programs can be recorded on, for example, a computer-readable storage medium, loaded into an electronic device via the computer-readable storage medium, and run therein to implement the corresponding methods of the embodiments of this application.

[0221] In other words, the units mentioned above can be implemented in hardware, in software instructions, or in a combination of hardware and software. Specifically, the steps of the method embodiments in this application can be completed by the integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software in the decoding processor. Optionally, the software can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads the information in the memory and completes the steps in the above method embodiments in conjunction with its hardware.

[0222] Figure 11 This is a schematic structural diagram of the electronic device 600 provided in the embodiments of this application.

[0223] like Figure 11As shown, the electronic device 600 includes at least a processor 610 and a computer-readable storage medium 620. The processor 610 and the computer-readable storage medium 620 can be connected via a bus or other means. The computer-readable storage medium 620 stores a computer program 621, which includes computer instructions. The processor 610 executes the computer instructions stored in the computer-readable storage medium 620. The processor 610 is the computing and control core of the electronic device 600, and is suitable for implementing one or more computer instructions, specifically for loading and executing one or more computer instructions to achieve a corresponding method flow or function.

[0224] As an example, processor 610 may also be referred to as a central processing unit (CPU). Processor 610 may include, but is not limited to: general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0225] As an example, the computer-readable storage medium 620 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device; optionally, it may also be at least one computer-readable storage medium located remotely from the aforementioned processor 610. Specifically, the computer-readable storage medium 620 includes, but is not limited to, volatile memory and / or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0226] like Figure 11 As shown, the electronic device 600 may also include a transceiver 630.

[0227] The processor 610 can control the transceiver 630 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 630 may include a transmitter and a receiver. The transceiver 630 may further include antennas, and the number of antennas may be one or more. It should be understood that the various components in the communication device 600 are connected through a bus system, which includes, in addition to a data bus, a power bus, a control bus, and a status signal bus.

[0228] In one implementation, the electronic device 600 can be any electronic device with data processing capabilities; the computer-readable storage medium 620 stores computer instructions; the processor 610 loads and executes the computer instructions stored in the computer-readable storage medium 620 to implement the corresponding steps in the model training method or image recognition method provided in the embodiments of this application; in a specific implementation, the computer instructions in the computer-readable storage medium 620 are loaded and executed by the processor 610 to implement the corresponding steps, which will not be described again here to avoid repetition.

[0229] According to another aspect of this application, embodiments of this application also provide a computer-readable storage medium (Memory), which is a memory device in electronic device 600 for storing programs and data. For example, computer-readable storage medium 620. It is understood that computer-readable storage medium 620 here may include both built-in storage media in electronic device 600 and extended storage media supported by electronic device 600. The computer-readable storage medium provides storage space that stores the operating system of electronic device 600. Furthermore, the storage space also stores one or more computer instructions suitable for loading and execution by processor 610, which may be one or more computer programs 621 (including program code).

[0230] According to another aspect of this application, embodiments of this application also provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. For example, computer program 621. In this case, the data processing device 600 may be a computer, and the processor 610 reads the computer instructions from the computer-readable storage medium 620. The processor 610 executes the computer instructions, causing the computer to perform the model training method or image recognition method provided in the various optional embodiments described above.

[0231] In other words, when implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes of the embodiments of this application are run or the functions of the embodiments of this application are implemented. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0232] Those skilled in the art will recognize that the units and process steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0233] Finally, it should be noted that the above content is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method applied to image classification, characterized in that, include: Acquire training images; The first feature extraction model among m feature extraction models is used to extract features from the training image to obtain the first feature map of the training image; m is an integer greater than 1. The first feature map is transformed using the fully connected mapping units corresponding to the m feature extraction models to obtain the first feature vector of the training image; Based on the class centers used to characterize n identity categories, the first feature vector is converted into n probability values ​​of the training image; the n probability values ​​are used to characterize the probability that the training image belongs to the n identity categories; n is an integer greater than 1; Based on the n probability values ​​and the true identity category of the training image, a first loss value is calculated to represent the difference between the predicted identity category and the true identity category of the training image; Based on the first loss value, the first feature extraction model, the fully connected mapping unit, and the vector representing the class centers of the n identity categories are trained to obtain the trained first feature extraction model, the trained fully connected mapping unit, and the trained n class center vectors representing the class centers of the n identity categories. Based on the training image and the n class center vectors, train the remaining feature extraction model among the m feature extraction models; The method further includes: Obtain the first class center vector from the n class center vectors, which represents the true identity category of the training image; The confidence of the first feature map is estimated using a confidence estimation model to obtain the confidence of the first feature map. Based on the first class center vector, the confidence of the first feature map, and the first feature vector, a second loss value is calculated to express the accuracy of the predicted identity category of the training image in representing the true identity category. The confidence estimation model is trained based on the second loss value; The step of calculating a second loss value based on the first class center vector, the confidence score of the first feature map, and the first feature vector to express the accuracy of the predicted identity category of the training image in representing the true identity category includes: The first value is determined by the negative of the product of the ratio of the confidence level of the first feature map to the radius of the hypersphere space in which the first feature map is located, the first feature vector, and the first class center vector. The negative of the product of the second value and the logarithm of the confidence level of the first feature map is determined as the fifth value; the second value is the value obtained by subtracting the fourth value from the ratio of the dimension of the first feature vector to the third value. The logarithm of the Bessel function, which has the confidence level of the first feature map as the independent variable and the second value as the order, is determined as the sixth value. The product of the ratio of the dimension of the first feature vector to the third value and the logarithm of the seventh value is determined as the eighth value; the seventh value is the product of the third value, pi, and the square of the radius of the hypersphere. The sum of the first value, the fifth value, the sixth value, and the eighth value is determined as the second loss value.

2. The method according to claim 1, characterized in that, The step of training the remaining feature extraction model among the m feature extraction models based on the training image and the n class center vectors includes: While keeping the n class center vectors unchanged, the remaining feature extraction model is trained based on the training images.

3. The method according to claim 1, characterized in that, The initial values ​​of the hyperparameters of the remaining feature extraction model are different from the initial values ​​of the hyperparameters of the first feature extraction model, and / or the hyperparameters in the loss function used by the first feature training model are different from the hyperparameters in the loss function used by the remaining feature extraction model.

4. The method according to claim 1, characterized in that, The first feature map is a feature map distributed in the hyperspherical space.

5. The method according to claim 1, characterized in that, The step of converting the first feature vector into n probability values ​​of the training image based on the class centers used to characterize n identity categories includes: If the dimension of each vector in the vectors used to characterize the class centers of the n identity categories is the same as the dimension of the first feature vector, then the matrix formed by the vectors used to characterize the class centers of the n identity categories is obtained, and the value obtained by multiplying the first feature vector and the matrix formed by the vectors used to characterize the class centers of the n identity categories is determined as the n probability values.

6. The method according to claim 1, characterized in that, The step of training the confidence estimation model based on the second loss value includes: Using gradient descent, the confidence of the first feature map and the first feature vector are adjusted, and the second loss value is recalculated until the number of calculations of the second loss value meets the set threshold or the second loss function value is less than or equal to the set function value threshold, thus obtaining the trained confidence estimation model.

7. An image recognition method, applied to the method according to any one of claims 1-6, characterized in that, include: Acquire the image to be recognized; Using m feature extraction models, features are extracted from the image to be identified to obtain m feature maps of the image to be identified; m is an integer greater than 1; Using the fully connected mapping units corresponding to the m feature extraction models, the m feature maps are transformed to obtain m feature vectors; The confidence of the m feature maps is estimated using a confidence estimation model to obtain the confidence of the m feature maps; the confidence estimation model is a confidence estimation model trained based on the m feature extraction models. Based on the confidence scores of the m feature maps, feature fusion is performed on the m feature vectors to obtain a fused feature vector; Image recognition is performed on the image to be recognized based on the fused feature vector.

8. The method according to claim 7, characterized in that, The step of fusing the m feature vectors based on the confidence scores of the m feature maps to obtain a fused feature vector includes: The confidence scores of the m feature maps are determined as the weights of the m feature vectors; Based on the weights of the m feature vectors, a weighted average is calculated on the m feature vectors to obtain the fused feature vector.

9. A model training device for image classification, characterized in that, include: Image acquisition unit, used to acquire training images; The first feature extraction model among m feature extraction models is used to extract features from the training image to obtain the first feature map of the training image; m is an integer greater than 1. The fully connected mapping units corresponding to the m feature extraction models are used to transform the first feature map to obtain the first feature vector of the training image; A class center unit is used to convert the first feature vector into n probability values ​​of the training image based on a vector representing the class centers of n identity categories; the n probability values ​​are used to represent the probability that the training image belongs to the n identity categories; n is an integer greater than 1; The calculation unit is configured to calculate a first loss value, representing the difference between the predicted identity category of the training image and the real identity category of the training image, based on the n probability values ​​and the real identity category of the training image; The first training unit is used to train the first feature extraction model, the fully connected mapping unit, and the vector representing the class centers of the n identity categories based on the first loss value, so as to obtain the trained first feature extraction model, the trained fully connected mapping unit, and the trained n class center vectors representing the class centers of the n identity categories. The second training unit is used to train the remaining feature extraction model among the m feature extraction models based on the training image and the n class center vectors; The device further includes: Obtain the first class center vector from the n class center vectors, which represents the true identity category of the training image; The confidence of the first feature map is estimated using a confidence estimation model to obtain the confidence of the first feature map. Based on the first class center vector, the confidence of the first feature map, and the first feature vector, a second loss value is calculated to express the accuracy of the predicted identity category of the training image in representing the true identity category. The confidence estimation model is trained based on the second loss value; The step of calculating a second loss value based on the first class center vector, the confidence score of the first feature map, and the first feature vector to express the accuracy of the predicted identity category of the training image in representing the true identity category includes: The first value is determined by the negative of the product of the ratio of the confidence level of the first feature map to the radius of the hypersphere space in which the first feature map is located, the first feature vector, and the first class center vector. The negative of the product of the second value and the logarithm of the confidence level of the first feature map is determined as the fifth value; the second value is the value obtained by subtracting the fourth value from the ratio of the dimension of the first feature vector to the third value. The logarithm of the Bessel function, which has the confidence level of the first feature map as the independent variable and the second value as the order, is determined as the sixth value. The product of the ratio of the dimension of the first feature vector to the third value and the logarithm of the seventh value is determined as the eighth value; the seventh value is the product of the third value, pi, and the square of the radius of the hypersphere. The sum of the first value, the fifth value, the sixth value, and the eighth value is determined as the second loss value.

10. An image recognition device, applied to the device of claim 9, characterized in that, include: An image acquisition unit is used to acquire the image to be recognized. m feature extraction models are used to extract features from the image to be identified, resulting in m feature maps of the image to be identified; m is an integer greater than 1. The fully connected mapping units corresponding to the m feature extraction models are used to transform the m feature maps to obtain m feature vectors; A confidence estimation model is used to estimate the confidence of the m feature maps to obtain the confidence of the m feature maps; the confidence estimation model is a confidence estimation model trained based on the m feature extraction models; The feature fusion unit is used to fuse the m feature vectors based on the confidence levels of the m feature maps to obtain a fused feature vector; The recognition unit is used to perform image recognition on the image to be recognized based on the fused feature vector.

11. An electronic device, characterized in that, include: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program that, when executed by the processor, implements the method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 8.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1 to 8.

Images