Feature extraction model training method and device, electronic equipment and storage medium

Abstract

The application discloses a feature extraction model training method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a sample mask image corresponding to a sample image containing a target object; performing feature extraction on an unmasked area in the sample mask image based on a to-be-trained feature extraction network to obtain a sample feature map; performing feature style coding on the sample feature map based on a to-be-trained feature style coding network to obtain a plurality of style hidden vectors representing different feature levels of the target object; performing image reconstruction based on the plurality of style hidden vectors; determining an image reconstruction loss and a feature reconstruction loss based on the sample image and the reconstructed image; adjusting network parameters of the to-be-trained feature extraction network and the to-be-trained feature style coding network based on the image reconstruction loss and the feature reconstruction loss; and taking the to-be-trained feature extraction network at the end of the training as a target feature extraction model. The application can obtain a feature extraction model with high feature extraction performance even if the training data is limited.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a feature extraction model training method, apparatus, electronic device, and storage medium. Background Technology

[0002] Object recognition (such as face recognition) is usually based on the feature representation of objects, so the quality of the feature representation will directly affect the performance of object recognition tasks.

[0003] In related technologies, large-scale pre-trained object data (such as face images) is required to train the feature extraction model in order to obtain high-quality feature representations. However, when object information (such as face information) is protected as privacy information, it is difficult for the scale of pre-trained object data to meet the data scale requirements of related technologies. As a result, the feature extraction performance of the feature extraction model trained by the pre-training method in related technologies is poor and cannot provide high-quality feature representations that meet the requirements of downstream object recognition tasks. Summary of the Invention

[0004] To address the problems of existing technologies, embodiments of this application provide a feature extraction model training method, apparatus, electronic device, and storage medium. The technical solution is as follows:

[0005] On the one hand, a method for training a feature extraction model is provided, the method comprising:

[0006] Obtain the sample occlusion image corresponding to the sample image; the sample image contains the target object, and the sample occlusion image is an image obtained by occluding a portion of the sample image;

[0007] Based on the feature extraction network to be trained, features are extracted from the unmasked regions in the sample occluded image to obtain a sample feature map; the sample feature map includes the features of the unmasked regions;

[0008] The sample feature map is encoded using a feature style coding network to be trained, resulting in multiple style latent vectors; the multiple style latent vectors represent the features of the target object at different feature levels.

[0009] Image reconstruction is performed based on the multiple style latent vectors to obtain a reconstructed image; based on the sample image and the reconstructed image, image reconstruction loss and feature reconstruction loss are determined respectively.

[0010] The network parameters are adjusted based on the image reconstruction loss and feature reconstruction loss until the preset training termination condition is met. The feature extraction network to be trained at the end of training is then used as the target feature extraction model. The network parameters include the network parameters of the feature extraction network to be trained and the feature style encoding network to be trained.

[0011] On the other hand, a feature extraction model training device is provided, the device comprising:

[0012] The occlusion image acquisition module is used to acquire the sample occlusion image corresponding to the sample image; the sample image contains a target object, and the sample occlusion image is an image obtained by occluding a portion of the sample image;

[0013] The feature extraction module is used to extract features from the unmasked regions in the sample occluded image based on the feature extraction network to be trained, to obtain a sample feature map; the sample feature map includes the features of the unmasked regions;

[0014] The feature style encoding module is used to perform feature style encoding on the sample feature map based on the feature style encoding network to be trained, to obtain multiple style latent vectors; the multiple style latent vectors represent the features of different feature levels of the target object;

[0015] The reconstruction and loss determination module is used to reconstruct the image based on the multiple style latent vectors to obtain the reconstructed image; and to determine the image reconstruction loss and feature reconstruction loss based on the sample image and the reconstructed image, respectively.

[0016] The parameter adjustment module is used to adjust the network parameters based on the image reconstruction loss and feature reconstruction loss until the preset training termination condition is met, and the feature extraction network to be trained at the end of training is used as the target feature extraction model; the network parameters include the network parameters of the feature extraction network to be trained and the feature style encoding network to be trained.

[0017] In some exemplary embodiments, the reconstruction and loss determination module includes:

[0018] The image reconstruction loss determination module is used to determine the image reconstruction loss based on the difference between the image patch corresponding to the occluded region in the sample image and the image patch corresponding to the occluded region in the reconstructed image;

[0019] A visual feature extraction module is used to extract the visual features of the sample image and the visual features of the reconstructed image based on a self-attention mechanism, respectively.

[0020] The feature reconstruction loss determination module is used to determine the feature reconstruction loss based on the distance between the visual features of the sample image and the visual features of the reconstructed image.

[0021] In some exemplary embodiments, the visual feature extraction module includes:

[0022] The first visual feature extraction submodule is used to input the sample image into a self-attention network, extract features from the sample image sequentially through multiple self-attention layers of the self-attention network, and use the output of at least one preset self-attention layer among the multiple self-attention layers as the visual features of the sample image.

[0023] The second visual feature extraction submodule is used to input the reconstructed image into the self-attention network, extract features from the reconstructed image sequentially through multiple self-attention layers of the self-attention network, and use the output of at least one preset self-attention layer as the visual features of the reconstructed image.

[0024] The feature reconstruction loss determination module includes:

[0025] The first determining module is used to determine the loss corresponding to each preset self-attention layer based on the distance between the output of each preset self-attention layer for the sample image and the output for the reconstructed image;

[0026] The second determining module is used to obtain the feature reconstruction loss based on the sum of the losses corresponding to each of the preset attention layers.

[0027] In some exemplary embodiments, the reconstruction and loss determination module includes:

[0028] The reconstruction module is used to input the multiple style latent vectors into the trained image generation network, and to perform image reconstruction on the style latent vectors corresponding to each feature level through the image generation sub-network corresponding to each feature level in the trained image generation network, so as to obtain the reconstructed sub-image of each feature level.

[0029] The fusion module is used to fuse the reconstructed sub-images of each feature level to obtain a reconstructed image;

[0030] The trained image generation network is obtained by training a style-based generative adversarial network based on a training image set of the target object.

[0031] In some exemplary embodiments, the feature style encoding module includes:

[0032] The intermediate feature extraction module is used to extract features from the sample feature map at different feature levels based on multiple convolutional neural networks in the feature style coding network to be trained, so as to obtain an intermediate feature map for each feature level.

[0033] The encoding processing module is used to encode the intermediate feature map of each feature level based on multiple encoding networks corresponding to the feature level to obtain multiple latent vectors corresponding to the feature level; wherein, the multiple encoding networks correspond one-to-one with multiple convolutional layers in the image generation subnetwork corresponding to the feature level in the trained image generation network.

[0034] The style latent vector determination module is used to concatenate multiple latent vectors corresponding to each feature level based on the hierarchical connection order of multiple convolutional layers in the image generation subnetwork corresponding to each feature level in the trained image generation network, so as to obtain the style latent vector of each feature level.

[0035] In some exemplary embodiments, the feature extraction module includes:

[0036] The unmasked region extraction module is used to extract unmasked regions from the sample masked image to obtain multiple unmasked image blocks;

[0037] The block feature extraction module is used to extract features from the multiple unmasked image blocks based on the feature extraction network to be trained, and obtain a one-dimensional feature vector corresponding to each unmasked image block;

[0038] A preset vector acquisition module is used to acquire a one-dimensional preset vector representing the occlusion region in the sample occlusion image;

[0039] The two-dimensional image reshaping module is used to perform two-dimensional image reshaping based on the one-dimensional preset vector, the one-dimensional feature vector corresponding to each unmasked image block, and the position in the sample masked image to obtain a sample feature map.

[0040] In some exemplary embodiments, the occlusion image acquisition module includes:

[0041] The image acquisition module is used to acquire the sample image and the preset mask image;

[0042] The masking processing module is used to perform dot multiplication on the sample image and the preset mask image to obtain the sample masking image.

[0043] On the other hand, an electronic device is provided, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the feature extraction model training method of any of the above aspects.

[0044] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the feature extraction model training method as described above.

[0045] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the feature extraction model training method of any of the above aspects.

[0046] This application embodiment extracts features from the unmasked regions of a sample occluded image using a feature extraction network to be trained, obtaining a sample feature map including features of the unmasked regions. Then, it encodes the sample feature map using a feature style encoding network to be trained, obtaining multiple style latent vectors. These style latent vectors indicate features at different feature levels of the target object. Finally, it reconstructs the image based on these style latent vectors to obtain a reconstructed image. Based on the original sample image and the reconstructed image, it determines the image reconstruction loss and feature reconstruction loss, respectively. Based on these losses, it adjusts the network parameters of the feature extraction network and the feature style encoding network to be trained. Finally, it uses the feature extraction network at the end of training as the target feature extraction model. This allows for a high-performance feature extraction model even with limited training data, providing high-quality object feature representations for downstream object recognition tasks. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating a feature extraction model training method provided in an embodiment of this application;

[0049] Figure 2 This is a flowchart illustrating another feature extraction model training method provided in an embodiment of this application;

[0050] Figure 3 This is an example of the correspondence between different feature levels of facial features and multiple convolutional layers in a face generation network, provided in the embodiments of this application.

[0051] Figure 4 This is a schematic diagram of the structure of the feature style coding network to be trained provided in the embodiments of this application;

[0052] Figure 5 This is a flowchart illustrating another feature extraction model training method provided in an embodiment of this application;

[0053] Figure 6 This is an example of the training process of the face feature extraction model provided in the embodiments of this application;

[0054] Figure 7 This is a structural block diagram of a feature extraction model training device provided in an embodiment of this application;

[0055] Figure 8 This is a hardware structure block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0057] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0058] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0059] It should be noted that the feature extraction model training method of this application embodiment can be applied to the feature extraction model training device of this application embodiment. The feature extraction model training device can be configured in an electronic device, which can be a terminal or a server.

[0060] Terminals include, but are not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft.

[0061] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0062] The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0063] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or computers-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0064] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes for target recognition and measurement, and further processes images to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0065] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0066] One specific scenario in which this application embodiment can be applied is the face recognition scenario. By adopting the feature extraction model training method of this application embodiment, and using existing public face image sets for face representation learning, a target feature extraction model with high feature extraction performance can be obtained. Based on this target feature extraction model, high-quality face feature representations can be provided for downstream face recognition tasks (such as face registration, face segmentation, etc.), which is conducive to improving the performance of downstream task models, shortening the training time of downstream task models, and reducing resource consumption.

[0067] Please see Figure 1 The diagram illustrates a flowchart of a feature extraction model training method provided in an embodiment of this application. It should be noted that while this specification provides the operational steps described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive methods. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 1 As shown, the method may include:

[0068] S101, Obtain the sample occlusion image corresponding to the sample image.

[0069] The sample image contains the target object, which can be determined based on the actual application scenario of the feature extraction model. For example, in a facial feature-based application scenario, the target object can be a face, that is, the sample image can be a face image, which can be obtained from an existing set of publicly available face images.

[0070] A sample occluded image is an image obtained by occluding a portion of a sample image. It can be understood that a sample occluded image includes occluded and unoccluded regions, where the unoccluded regions can be multiple discontinuous image patches, and these discontinuous image patches can be of the same or different sizes.

[0071] In some exemplary embodiments, obtaining the sample occlusion image corresponding to the sample image may be: obtaining the sample image and a preset mask image, and performing a dot product operation on the sample image and the preset mask image to obtain the sample occlusion image.

[0072] The preset mask image has the same size as the sample image. The preset mask image can use the pixel values ​​of pixels to represent the areas that need to be masked and the areas that do not need to be masked. For example, the pixel value of the pixel is set to 255 (i.e., white) for the areas that need to be masked, while the pixel value of the pixel is set to a slightly smaller value (i.e., gray) or 0 (i.e., black) for the areas that do not need to be masked.

[0073] For example, if I represents a sample image, M represents a preset mask image, and ⊙ represents a dot product, then the sample occlusion image corresponding to sample image I can be represented as I⊙M.

[0074] S103, Based on the feature extraction network to be trained, the unmasked areas in the sample occluded image are extracted to obtain the sample feature map.

[0075] The sample feature map includes features of the unmasked areas.

[0076] The feature extraction network to be trained can be an encoded neural network used to implement image encoding functions, such as the ViT (Vision Transformer) model.

[0077] In some exemplary implementations, in order to improve training efficiency, such as Figure 2 As shown, step S103 above may include the following when implemented:

[0078] S201, extract the unmasked region from the sample masked image to obtain multiple unmasked image blocks.

[0079] S203, Based on the feature extraction network to be trained, features are extracted from multiple unmasked image patches respectively to obtain a one-dimensional feature vector corresponding to each unmasked image patch.

[0080] Specifically, multiple unmasked image patches are input into the feature extraction network to be trained. The feature extraction network extracts features from each unmasked image patch to obtain a one-dimensional feature representation, i.e., a one-dimensional feature vector, for each unmasked image patch.

[0081] S205, obtain a one-dimensional preset vector representing the occluded region in the sample occluded image.

[0082] The one-dimensional preset vector can be set as needed to represent the occlusion area.

[0083] S207. Based on a one-dimensional preset vector, and the one-dimensional feature vector corresponding to each unmasked image block and its position in the sample masked image, a two-dimensional image reshaping is performed to obtain the sample feature map.

[0084] The sample feature map obtained after two-dimensional image reconstruction has the same size as the sample occlusion image. The sample feature map includes features corresponding to each unoccluded area and features corresponding to each occluded area. The features corresponding to each occluded area are represented by a one-dimensional preset vector.

[0085] Specifically, the position of each unmasked image patch in the sample masked image can be obtained, and then two-dimensional image reconstruction can be performed based on the position of each unmasked image patch in the sample masked image.

[0086] In the above implementation, by extracting features only from the unmasked areas and then reconstructing the sample feature map based on the extracted features corresponding to each unmasked area and the one-dimensional preset vector representing the masked area, the amount of data to be processed by the feature extraction model to be trained can be reduced, which is conducive to speeding up the training and improving the training efficiency.

[0087] S105, based on the feature style coding network to be trained, feature style coding is performed on the sample feature map to obtain multiple style latent vectors.

[0088] In this context, multiple style latent vectors represent features at different feature levels of the target object; that is, one style latent vector represents a feature at one feature level of the target object. Features at different feature levels correspond to different granularity attributes of the target object. In practical applications, different target objects may have different feature levels.

[0089] Taking a human face as an example, facial features can be divided into three feature levels, from low to high: global features, intermediate features, and fine features. Global features mainly include facial pose, hairstyle, and facial shape; intermediate features mainly include more refined facial features, hairstyle, and whether the eyes are open or closed; fine features mainly include texture and color details of the eyes, hair, and skin. By using a feature style encoding network to encode the feature maps of sample faces, we can obtain the style latent vectors of the global features, intermediate features, and fine features of the corresponding face.

[0090] In specific implementation, the feature style encoding network to be trained can include convolutional neural networks of different feature levels corresponding to the target object. Then, the convolutional neural network corresponding to each feature level can be used to encode the sample feature image to obtain the style latent vector of each feature level.

[0091] S107, based on the above multiple style latent vectors, image reconstruction is performed to obtain the reconstructed image.

[0092] Specifically, the image corresponding to each style latent vector can be reconstructed based on that style latent vector, and then the images corresponding to each style latent vector are fused to obtain the final reconstructed image. Since multiple style latent vectors represent features at different feature levels, high-quality reconstructed images can be obtained based on multiple style latent vectors.

[0093] In some exemplary embodiments, the above image reconstruction process can be implemented based on a trained image generation network, wherein the trained image generation network is obtained by training a style-based generative adversarial network on a training image set of the target object.

[0094] Style-based generative adversarial networks (GANs) achieve editing of attributes at different granularities through hierarchical control, thereby obtaining high-quality generated images. Specifically, this style-based GAN can be the generative network in StyleGAN, which is a progressively increasing resolution structure with a total of 17 convolutional layers. Except for the first convolutional layer, every two convolutional layers are upsampled by one scale, increasing the resolution from 4×4 to 1024×1024. Random noise is added after each convolutional layer.

[0095] Taking a face image generated by StyleGAN as an example, according to different feature levels of scale-based face features and multiple convolutional layers in the face generation network, there are... Figure 3 The correspondence shown is as follows: the resolution of convolutional layers S1 to S3 ranges from 4×4 to 8×8, corresponding to the global features of the face (mainly including facial pose, hairstyle, facial shape, etc.); the resolution of convolutional layers S3 to S7 ranges from 16×16 to 32×32, corresponding to the intermediate features of the face (mainly including more detailed facial features, hairstyle, eye opening or closing, etc.); and the resolution of convolutional layers S8 to S17 ranges from 64×64 to 1024×1024, corresponding to the fine features of the face (mainly including texture and color details of the eyes, hair, and skin).

[0096] Therefore, the trained image generation network in this application embodiment includes an image generation sub-network corresponding to each feature level of the target object. Each image generation sub-network may also include multiple convolutional layers connected in stages. Each image generation sub-network can be used to express the features of its corresponding feature level.

[0097] For details on training style-based generative adversarial networks using specific target object-based training image sets, please refer to existing technologies related to generative adversarial network training, which will not be elaborated here.

[0098] Typically, a Generative Adversarial Network (GAN) consists of a generator network and a discriminator network. When training a GAN, the generator network is trained to generate fake images that the discriminator network cannot distinguish, while the discriminator network is trained to determine whether an image comes from the generator network. Under these conditions, the latent space sampled during the training of the generator network will be endowed with rich semantic information. The latent space of the trained image generator network obtained by training the style-based GAN based on the training image set of the target object has the characteristic of decoupling the feature hierarchy of the target object. That is, the vectors in the latent space of the trained image generator network have been divided into feature levels.

[0099] Based on this, in some exemplary embodiments, the sample feature map can be encoded into the latent space of the trained image generation network based on the feature style encoding network to be trained, thereby obtaining style latent vectors representing different feature levels of the target object. Specifically, the feature style encoding network to be trained may include multiple convolutional neural networks, each corresponding to a feature level of the target object. Therefore, performing feature style encoding on the sample feature map based on the feature style encoding network to be trained to obtain multiple style latent vectors can be as follows:

[0100] Based on multiple convolutional neural networks in the feature style coding network to be trained, features of the target object at different feature levels are extracted from the sample feature map, and intermediate feature maps of each feature level are obtained.

[0101] For each intermediate feature map at each feature level, the intermediate feature map is encoded by multiple encoding networks corresponding to that feature level to obtain multiple latent vectors corresponding to that feature level. Among them, the multiple encoding networks correspond one-to-one with the multiple convolutional layers in the image generation subnetwork of the trained image generation network that corresponds to the feature level.

[0102] Based on the hierarchical connection order of multiple convolutional layers in the image generation subnetwork corresponding to each feature level in the trained image generation network, multiple latent vectors corresponding to each feature level are concatenated to obtain the style latent vector of each feature level.

[0103] Each feature level can be represented by multiple encoding networks, including a single convolutional layer and a linear mapping layer.

[0104] Please see Figure 4The diagram shows the structure of the feature style encoding network to be trained provided in the embodiment of this application. It includes multiple convolutional neural networks, each of which is used to extract features of different feature levels of the target object. Each convolutional neural network is followed by multiple encoding networks in parallel. The number of these multiple encoding networks is related to the image generation sub-network in the trained image generation network used to express the features of the corresponding feature level. Specifically, it is the number of multiple convolutional layers connected step by step in the image generation sub-network.

[0105] Assuming that the trained image generation network G1, which is used to express the features of feature level T1, contains three progressively connected convolutional layers, then the convolutional neural network F1 in the style coding network to be trained will be connected in parallel with three coding networks after the corresponding feature level T1. These three coding networks correspond one-to-one with the three convolutional layers in the image generation network G1. After feature extraction by the convolutional neural network F1, the sample feature map can obtain the intermediate feature map of the corresponding feature level T1. This intermediate feature map is used as the input of the three coding networks connected to the convolutional neural network F1 and is encoded and processed to obtain the three latent vectors corresponding to feature level T1. Then, the three latent vectors corresponding to feature level T1 are concatenated according to the connection order of the three convolutional layers in the image generation network G1 to obtain the style latent vector corresponding to feature level T1. This style latent vector will be expressed by the image generation network G1.

[0106] Based on this, in some exemplary embodiments, image reconstruction based on multiple style latent vectors to obtain a reconstructed image can be:

[0107] Multiple style latent vectors are input into a trained image generation sub-network. The style latent vectors corresponding to each feature level are reconstructed by the image generation sub-network corresponding to each feature level in the trained image generation sub-network, resulting in a reconstructed sub-image for each feature level.

[0108] The reconstructed sub-images at each feature level are fused to obtain the reconstructed image.

[0109] S109, based on the sample image and the reconstructed image, determine the image reconstruction loss and the feature reconstruction loss respectively.

[0110] Among them, image reconstruction loss is used to characterize the global image loss; feature reconstruction loss is used to characterize the image detail loss, which can make the feature extraction network under training pay more attention to style information such as image shape and texture. Specifically, the feature reconstruction loss can be calculated by comparing the convolution output of the sample image and the convolution output of the reconstructed image.

[0111] In some exemplary implementations, in order to improve training effectiveness, such as Figure 5As shown, determining the image reconstruction loss and feature reconstruction loss based on the sample image and the reconstructed image respectively can include:

[0112] S501, determine the image reconstruction loss based on the difference between the image patch corresponding to the occluded region in the sample image and the image patch corresponding to the occluded region in the reconstructed image.

[0113] In specific implementation, assuming that the sample occlusion image corresponding to sample image I is obtained by I⊙M processing through a preset mask image M, then the image reconstruction loss is... It can be represented as:

[0114]

[0115] in, This represents the reconstructed image corresponding to sample image I; This represents the inverted mask obtained by inverting a preset mask. For example, if the preset mask uses a pixel value of 0 to represent the occluded area and a pixel value of 1 to represent the unoccluded area, then the pixel value of each pixel in the inverted mask corresponding to the occluded area is 1, and the pixel value of each pixel in the unoccluded area is 0; ||·|2 represents the L2 norm.

[0116] S503 extracts visual features from sample images and reconstructs visual features from images based on a self-attention mechanism.

[0117] Considering that the effective information extracted from images solely based on convolution is limited, which may lead to incorrect gradient directions during optimization, this embodiment of the application uses a self-attention mechanism to extract visual features from the sample image and the reconstructed image separately. This allows for simultaneous attention to both global and local information, providing more effective information. Specifically, a Transformer-based feature extraction network (such as the ViT model) can be used to extract features from both the sample image and the reconstructed image to obtain their respective visual features.

[0118] S505, determine the feature reconstruction loss based on the distance between the visual features of the sample image and the visual features of the reconstructed image.

[0119] The distance can be Euclidean distance, Manhattan distance, cosine distance, etc.

[0120] For example, the feature reconstruction loss L percep It can be represented as: Where φ(I) represents the visual features of the sample image, This represents the visual features of the reconstructed image.

[0121] In some exemplary embodiments, in order to calculate the feature reconstruction loss more accurately, step S503 above may be implemented as follows: inputting the sample image into the self-attention network, extracting features from the sample image sequentially through multiple self-attention layers of the self-attention network, and using the output of at least one preset self-attention layer among the multiple self-attention layers as the visual features of the sample image; inputting the reconstructed image into the self-attention network, extracting features from the reconstructed image sequentially through multiple self-attention layers of the self-attention network, and using the output of the at least one preset self-attention layer as the visual features of the reconstructed image.

[0122] In this self-attention network, each self-attention layer can extract features from the input of that layer based on the self-attention mechanism.

[0123] The at least one preset self-attention layer can be set in combination with the number of self-attention layers in the self-attention network and actual needs. For example, if the self-attention network includes 15 self-attention layers connected in sequence, then the at least one preset self-attention layer can be the 2nd, 5th, 7th and 11th self-attention layers.

[0124] Based on this, the feature reconstruction loss can be determined by the distance between the visual features of the sample image and the visual features of the reconstructed image: the loss corresponding to each preset self-attention layer is determined by the distance between the output of each preset self-attention layer for the sample image and the output for the reconstructed image; the feature reconstruction loss is obtained by the sum of the losses corresponding to each preset attention layer, so as to effectively promote the image reconstruction quality and improve the training effect.

[0125] For example, the feature reconstruction loss L percep It can be represented as: Where, φ j (I) represents the output of the j-th preset self-attention layer corresponding to sample image I; Represents sample image The output of the corresponding j-th preset self-attention layer.

[0126] S111, adjust the network parameters based on image reconstruction loss and feature reconstruction loss until the preset training termination condition is met, and use the feature extraction network to be trained at the end of training as the target feature extraction model.

[0127] The adjusted network parameters include the network parameters of the feature extraction network to be trained and the feature style encoding network to be trained.

[0128] Specifically, the image reconstruction loss and feature reconstruction loss can be weighted and summed to obtain a comprehensive loss. Then, the network parameters of the feature extraction network and the feature style encoding network to be trained can be adjusted in reverse based on the comprehensive loss. The training can continue iteratively based on the adjusted network parameters until the preset training termination condition is met. The feature extraction network to be trained at the end of training is used as the target feature extraction model, which can be used for downstream object recognition tasks.

[0129] For example, the comprehensive loss L total It can be represented as: L total =λ1L2+λ2L percep Where L2 is the image reconstruction loss; L percep λ1 and λ2 are the weights corresponding to the feature reconstruction loss, and λ1 + λ2 = 1. The specific values ​​can be set according to actual needs.

[0130] The preset training termination condition can be that the number of iterations reaches a preset iteration threshold, the comprehensive loss reaches a preset loss threshold, or the difference between the comprehensive losses of two adjacent iterations reaches a preset difference threshold.

[0131] As can be seen from the above technical solutions of the embodiments of this application, the embodiments of this application extract features from the unmasked areas in the sample occluded image based on the feature extraction network to be trained to obtain a sample feature map including the features of the unmasked areas, and encode the sample feature map with feature style based on the feature style encoding network to be trained to obtain multiple style latent vectors. These multiple style latent vectors indicate the features of different feature levels of the target object. Then, image reconstruction is performed based on the multiple style latent vectors to obtain a reconstructed image. The image reconstruction loss and feature reconstruction loss are determined based on the original sample image and the reconstructed image, respectively. The network parameters of the feature extraction network to be trained and the feature style encoding network to be trained are adjusted based on the image reconstruction loss and the feature reconstruction loss. The feature extraction network to be trained at the end of training is used as the target feature extraction model. Thus, even with limited training data, a feature extraction model with high feature extraction performance can be obtained, providing high-quality object feature representation for downstream object recognition tasks.

[0132] To facilitate understanding of the technical solutions in the embodiments of this application, the following is combined with... Figure 6 This paper takes the training process of a face feature extraction model as an example, with the target object being a human face and the trained image generation network being StyleGAN, which is trained based on an existing set of face images.

[0133] like Figure 6As shown, the input sample face image I is randomly occluded to obtain the sample occluded face image I⊙M; then, the unoccluded image patches in the sample occluded face image are extracted to form the input image patch sequence, which is used as the feature extraction network E to be trained. attn The input is used to extract features to obtain the output image patch feature sequence; the output image patch feature sequence is reshaped into a sample feature map, in which a preset one-dimensional vector representation is sampled for the occluded region to mark the occluded region, and the sample feature map is further input into the Feature Style Coding Network (TSM) to be trained.

[0134] See also Figure 6 The Feature Style Encoding Network (TSM) to be trained encodes sample feature maps into the latent space of the face generation network G, based on the aforementioned... Figure 3 The facial features shown are configured with three convolutional neural networks (CNNs) {F1, F2, F3}, each corresponding to a feature level. Specifically, F1 corresponds to global features, F2 to intermediate features, and F3 to fine-grained features. After F1, three encoding networks {S1, S2, S3} are connected in parallel; after F2, four encoding networks {S4, S5, S6, S7} are connected in parallel; and after F3, ten encoding networks {S8, ..., S...} are connected in parallel. 17 In other words, each convolutional neural network and its subsequent parallel-connected multiple encoding networks constitute a processing path for the sample feature map in the style encoding network to be trained. After the sample feature map is input to the style encoding network to be trained, it undergoes processing through three paths, resulting in a 1x512 style latent vector output by each path. Each processing path corresponds to a style latent vector at a specific feature level. Furthermore, the style latent vectors at the global, intermediate, and fine-grained feature levels are obtained through the style encoding network to be trained. These three style latent vectors constitute the latent encoding w+ in the latent space of the face generation network: {w...} i ,i∈1,2,...,17}.

[0135] Next, the style latent vectors w+, comprising three feature levels, are input into the face generation network G. The face generation network G uses image generation subnetworks at the corresponding feature levels to reconstruct the style latent vectors at those feature levels, obtaining reconstructed sub-images for each feature level. Then, the reconstructed sub-images from each feature level are fused to output the reconstructed face image.

[0136] Furthermore, based on the aforementioned Calculate image reconstruction loss Furthermore, based on the trained ViT model, visual features are extracted from both the sample face images and the reconstructed face images. The outputs of the 2nd, 5th, 7th, and 11th self-attention layers are obtained as their respective visual features, and then based on the aforementioned... Calculate the feature reconstruction loss L percep .

[0137] Then, the image reconstruction loss and feature reconstruction loss L percep The weighted average is used to obtain the comprehensive loss L. total Based on this comprehensive loss L total Inversely adjust the feature extraction network E to be trained attn The network parameters of the Feature Style Encoding Network (TSM) to be trained are iteratively trained until the preset training termination condition is met. During this process, the features to be trained at the end of training are extracted into the network E. attn As a facial feature extraction model, it can be used for downstream facial recognition tasks (such as face segmentation, face registration, face reconstruction, etc.), thereby significantly improving the detection accuracy of downstream facial recognition tasks.

[0138] Corresponding to the feature extraction model training methods provided in the above embodiments, this application also provides a feature extraction model training device. Since the feature extraction model training device provided in this application corresponds to the feature extraction model training methods provided in the above embodiments, the implementation methods of the aforementioned feature extraction model training methods are also applicable to the feature extraction model training device provided in this embodiment, and will not be described in detail in this embodiment.

[0139] Please see Figure 7 The diagram shows a structural schematic of a feature extraction model training device provided in an embodiment of this application. This device has the function of implementing the feature extraction model training method described in the above-described method embodiments. This function can be implemented in hardware or by hardware executing corresponding software. Figure 7 As shown, the feature extraction model training device 700 may include:

[0140] The occlusion image acquisition module 710 is used to acquire a sample occlusion image corresponding to a sample image; the sample image contains a target object, and the sample occlusion image is an image obtained by occluding a portion of the sample image.

[0141] The feature extraction module 720 is used to extract features from the unmasked region in the sample occluded image based on the feature extraction network to be trained, and obtain a sample feature map; the sample feature map includes the features of the unmasked region;

[0142] The feature style encoding module 730 is used to perform feature style encoding on the sample feature map based on the feature style encoding network to be trained, to obtain multiple style latent vectors; the multiple style latent vectors represent the features of different feature levels of the target object;

[0143] The reconstruction and loss determination module 740 is used to perform image reconstruction based on the multiple style latent vectors to obtain a reconstructed image; and to determine the image reconstruction loss and feature reconstruction loss based on the sample image and the reconstructed image, respectively.

[0144] The parameter adjustment module 750 is used to adjust the network parameters based on the image reconstruction loss and feature reconstruction loss until the preset training termination condition is met, and the feature extraction network to be trained at the end of training is used as the target feature extraction model; the network parameters include the network parameters of the feature extraction network to be trained and the feature style encoding network to be trained.

[0145] In some exemplary embodiments, the reconstruction and loss determination module 740 includes:

[0146] The image reconstruction loss determination module is used to determine the image reconstruction loss based on the difference between the image patch corresponding to the occluded region in the sample image and the image patch corresponding to the occluded region in the reconstructed image;

[0147] A visual feature extraction module is used to extract the visual features of the sample image and the visual features of the reconstructed image based on a self-attention mechanism, respectively.

[0148] The feature reconstruction loss determination module is used to determine the feature reconstruction loss based on the distance between the visual features of the sample image and the visual features of the reconstructed image.

[0149] In some exemplary embodiments, the visual feature extraction module includes:

[0150] The first visual feature extraction submodule is used to input the sample image into a self-attention network, extract features from the sample image sequentially through multiple self-attention layers of the self-attention network, and use the output of at least one preset self-attention layer among the multiple self-attention layers as the visual features of the sample image.

[0151] The second visual feature extraction submodule is used to input the reconstructed image into the self-attention network, extract features from the reconstructed image sequentially through multiple self-attention layers of the self-attention network, and use the output of at least one preset self-attention layer as the visual features of the reconstructed image.

[0152] The feature reconstruction loss determination module includes:

[0153] The first determining module is used to determine the loss corresponding to each preset self-attention layer based on the distance between the output of each preset self-attention layer for the sample image and the output for the reconstructed image;

[0154] The second determining module is used to obtain the feature reconstruction loss based on the sum of the losses corresponding to each of the preset attention layers.

[0155] In some exemplary embodiments, the reconstruction and loss determination module 740 includes:

[0156] The reconstruction module is used to input the multiple style latent vectors into the trained image generation network, and to perform image reconstruction on the style latent vectors corresponding to each feature level through the image generation sub-network corresponding to each feature level in the trained image generation network, so as to obtain the reconstructed sub-image of each feature level.

[0157] The fusion module is used to fuse the reconstructed sub-images of each feature level to obtain a reconstructed image;

[0158] The trained image generation network is obtained by training a style-based generative adversarial network based on a training image set of the target object.

[0159] In some exemplary embodiments, the feature style encoding module 730 includes:

[0160] The intermediate feature extraction module is used to extract features from the sample feature map at different feature levels based on multiple convolutional neural networks in the feature style coding network to be trained, so as to obtain an intermediate feature map for each feature level.

[0161] The encoding processing module is used to encode the intermediate feature map of each feature level based on multiple encoding networks corresponding to the feature level to obtain multiple latent vectors corresponding to the feature level; wherein, the multiple encoding networks correspond one-to-one with multiple convolutional layers in the image generation subnetwork corresponding to the feature level in the trained image generation network.

[0162] The style latent vector determination module is used to concatenate multiple latent vectors corresponding to each feature level based on the hierarchical connection order of multiple convolutional layers in the image generation subnetwork corresponding to each feature level in the trained image generation network, so as to obtain the style latent vector of each feature level.

[0163] In some exemplary embodiments, the feature extraction module 720 includes:

[0164] The unmasked region extraction module is used to extract unmasked regions from the sample masked image to obtain multiple unmasked image blocks;

[0165] The block feature extraction module is used to extract features from the multiple unmasked image blocks based on the feature extraction network to be trained, and obtain a one-dimensional feature vector corresponding to each unmasked image block;

[0166] A preset vector acquisition module is used to acquire a one-dimensional preset vector representing the occlusion region in the sample occlusion image;

[0167] The two-dimensional image reshaping module is used to perform two-dimensional image reshaping based on the one-dimensional preset vector, the one-dimensional feature vector corresponding to each unmasked image block, and the position in the sample masked image to obtain a sample feature map.

[0168] In some exemplary embodiments, the occlusion image acquisition module 710 includes:

[0169] The image acquisition module is used to acquire the sample image and the preset mask image;

[0170] The masking processing module is used to perform dot multiplication on the sample image and the preset mask image to obtain the sample masking image.

[0171] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0172] This application provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement any of the feature extraction model training methods provided in the above method embodiments.

[0173] The method embodiments provided in this application can be executed in a computer terminal, server or similar computing device, that is, the above-mentioned electronic device may include a computer terminal, server or similar computing device. Figure 8 This is a hardware structure block diagram of an electronic device that runs a feature extraction model training method according to an embodiment of this application. This electronic device can participate in or include the feature extraction model training apparatus provided in the embodiment of this application. Figure 8As shown, the electronic device 80 may include one or more processors 802 (shown as 802a, 802b, ..., 802n in the figure) (processor 802 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 804 for storing data, and a transmission device 806 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 8 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 80 may also include... Figure 8 The more or fewer components shown, or having the same Figure 8 The different configurations shown.

[0174] It should be noted that the aforementioned one or more processors 802 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element within the electronic device 80 (or mobile device). As involved in the embodiments of this application, the data processing circuit serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0175] The memory 804 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of the present invention. The processor 802 executes various functional applications and data processing by running the software programs and modules stored in the memory 804, thereby realizing the feature extraction model training method described above. The memory 804 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 804 may further include memory remotely located relative to the processor 802, and these remote memories can be connected to the electronic device 80 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0176] The transmission device 806 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 80. In one example, the transmission device 806 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In one embodiment, the transmission device 806 may be a radio frequency (RF) module for wireless communication with the Internet.

[0177] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows a user to interact with the user interface of an electronic device 80 (or a mobile device).

[0178] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a feature extraction model training method. The at least one instruction or the at least one program is loaded and executed by the processor to implement any of the feature extraction model training methods provided in the above-described method embodiments.

[0179] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform any of the feature extraction model training methods provided in the above-described method embodiments.

[0180] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0181] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0182] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0183] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0184] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a feature extraction model, characterized in that, The method includes: Obtain the sample occlusion image corresponding to the sample image; the sample image contains the target object, and the sample occlusion image is an image obtained by occluding a portion of the sample image; Based on the feature extraction network to be trained, features are extracted from the unmasked regions in the sample occluded image to obtain a sample feature map; the sample feature map includes the features of the unmasked regions; The sample feature map is encoded using a feature style coding network to be trained, resulting in multiple style latent vectors; the multiple style latent vectors represent the features of the target object at different feature levels. Image reconstruction is performed based on the multiple style latent vectors to obtain a reconstructed image; based on the sample image and the reconstructed image, image reconstruction loss and feature reconstruction loss are determined respectively. The network parameters are adjusted based on the image reconstruction loss and feature reconstruction loss until the preset training termination condition is met. The feature extraction network to be trained at the end of training is then used as the target feature extraction model. The network parameters include the network parameters of the feature extraction network to be trained and the feature style encoding network to be trained.

2. The method according to claim 1, characterized in that, The step of determining image reconstruction loss and feature reconstruction loss based on the sample image and the reconstructed image, respectively, includes: The image reconstruction loss is determined based on the difference between the image patch corresponding to the occluded region in the sample image and the image patch corresponding to the occluded region in the reconstructed image; Visual features of the sample image and visual features of the reconstructed image are extracted based on a self-attention mechanism. The feature reconstruction loss is determined based on the distance between the visual features of the sample image and the visual features of the reconstructed image.

3. The method according to claim 2, characterized in that, The step of extracting visual features from the sample image and the reconstructed image based on a self-attention mechanism includes: The sample image is input into a self-attention network, and features are extracted from the sample image through multiple self-attention layers of the self-attention network in sequence. The output of at least one preset self-attention layer among the multiple self-attention layers is used as the visual features of the sample image. The reconstructed image is input into the self-attention network, and features are extracted from the reconstructed image through multiple self-attention layers of the self-attention network in sequence. The output of at least one preset self-attention layer is used as the visual features of the reconstructed image. The determination of feature reconstruction loss based on the distance between the visual features of the sample image and the visual features of the reconstructed image includes: Based on the distance between the output of each preset self-attention layer for the sample image and the output for the reconstructed image, the loss corresponding to each preset self-attention layer is determined; The feature reconstruction loss is obtained by summing the losses corresponding to each of the preset attention layers.

4. The method according to claim 1, characterized in that, The process of reconstructing the image based on the multiple style latent vectors to obtain the reconstructed image includes: The multiple style latent vectors are input into a trained image generation network. The style latent vectors corresponding to each feature level are reconstructed by the image generation sub-network corresponding to each feature level in the trained image generation network to obtain the reconstructed sub-image of each feature level. The reconstructed sub-images of each feature level are fused to obtain the reconstructed image; The trained image generation network is obtained by training a style-based generative adversarial network based on a training image set of the target object.

5. The method according to claim 4, characterized in that, The feature style encoding based on the feature style encoding network to be trained is used to encode the feature map of the sample to obtain multiple style latent vectors, including: Based on multiple convolutional neural networks in the feature style coding network to be trained, feature extraction is performed on the sample feature map at different feature levels to obtain an intermediate feature map for each feature level; For each intermediate feature map of the feature level, the intermediate feature map is encoded based on multiple encoding networks corresponding to the feature level to obtain multiple latent vectors corresponding to the feature level; wherein, the multiple encoding networks correspond one-to-one with multiple convolutional layers in the image generation subnetwork corresponding to the feature level in the trained image generation network. Based on the hierarchical connection order of multiple convolutional layers in the image generation subnetwork corresponding to each feature level in the trained image generation network, multiple latent vectors corresponding to each feature level are concatenated to obtain the style latent vector of each feature level.

6. The method according to any one of claims 1 to 5, characterized in that, The feature extraction of the unmasked region in the sample occluded image based on the feature extraction network to be trained, resulting in a sample feature map, includes: Extract the unmasked areas from the masked image of the sample to obtain multiple unmasked image blocks; Based on the feature extraction network to be trained, features are extracted from the multiple unmasked image blocks respectively to obtain a one-dimensional feature vector corresponding to each unmasked image block; Obtain a one-dimensional preset vector representing the occluded region in the sample occluded image; Based on the one-dimensional preset vector, and the one-dimensional feature vector corresponding to each unmasked image block and its position in the sample masked image, a two-dimensional image reshaping is performed to obtain the sample feature map.

7. The method according to any one of claims 1 to 5, characterized in that, The acquisition of the sample occlusion image corresponding to the sample image includes: Obtain the sample image and the preset mask image; The sample image and the preset mask image are multiplied by a dot to obtain the sample masking image.

8. A feature extraction model training device, characterized in that, The device includes: The occlusion image acquisition module is used to acquire the sample occlusion image corresponding to the sample image; the sample image contains a target object, and the sample occlusion image is an image obtained by occluding a portion of the sample image; The feature extraction module is used to extract features from the unmasked regions in the sample occluded image based on the feature extraction network to be trained, to obtain a sample feature map; the sample feature map includes the features of the unmasked regions; The feature style encoding module is used to perform feature style encoding on the sample feature map based on the feature style encoding network to be trained, to obtain multiple style latent vectors; the multiple style latent vectors represent the features of different feature levels of the target object; The reconstruction and loss determination module is used to reconstruct the image based on the multiple style latent vectors to obtain the reconstructed image; and to determine the image reconstruction loss and feature reconstruction loss based on the sample image and the reconstructed image, respectively. The parameter adjustment module is used to adjust the network parameters based on the image reconstruction loss and feature reconstruction loss until the preset training termination condition is met, and the feature extraction network to be trained at the end of training is used as the target feature extraction model; the network parameters include the network parameters of the feature extraction network to be trained and the feature style encoding network to be trained.

9. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the feature extraction model training method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the feature extraction model training method as described in any one of claims 1 to 7.

11. A computer program, characterized in that, When the computer program is executed by the processor, it implements the feature extraction model training method according to any one of claims 1 to 7.

Images