Feature extraction methods, model training methods, devices and electronic equipment
By dividing the target cluster centers and calculating the distance loss in self-supervised training, the problem of feature conflict in the self-supervised training mode is solved, thereby improving the training effect and accuracy of the feature extraction model.
Patent Information
- Application Number
- CN202211361033.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-02
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-11-02
AI Technical Summary
In the self-supervised training mode, the lack of labeled information leads to a decline in the training effect of the feature extraction model, and the feature conflict problem cannot be effectively resolved, affecting the accuracy of the feature extraction model.
By identifying multiple target cluster centers and dividing them into first and second center sets, calculating the distance between the sample data and each set, and training the feature extraction model using self-supervised loss and feature space reservation loss, the distribution of sample features is constrained to prevent feature conflicts during incremental training.
It improves the training performance of the feature extraction model under self-supervised incremental training, enhances the accuracy of feature extraction, and prevents feature conflicts caused by new sample data.
Smart Images

Figure CN117009776B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a feature extraction method, a model training method, an apparatus, and an electronic device. Background Technology
[0002] Currently, feature extraction models are widely used in various common scenarios. During incremental training of these models, as the number of training batches increases, feature conflicts may arise from the extracted sample data in different batches. In supervised learning, annotation information can be used to constrain the representation of features from different categories of samples, thus mitigating the impact of feature conflicts to some extent. However, in self-supervised training, the sample data lacks annotation information, making it impossible to resolve feature conflicts using annotations. This negatively impacts the training performance of the feature extraction model and reduces the accuracy of the features extracted. Summary of the Invention
[0003] The following is an overview of the subject matter described in detail in this application. This overview is not intended to limit the scope of the claims.
[0004] This application provides a feature extraction method, a model training method, an apparatus, and an electronic device, which can improve the effect of self-supervised incremental training of the feature extraction model and improve the accuracy of the features extracted by the feature extraction model.
[0005] On the one hand, embodiments of this application provide a feature extraction method, including:
[0006] Obtain sample media data from the current incremental training batch, extract sample features from the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features;
[0007] Multiple target cluster centers are identified, and the multiple target cluster centers are divided into a first center set and a second center set. A first distance between the sample media data and the first center set and a second distance between the sample media data and the second center set are determined. The feature space reservation loss is determined based on the first distance and the second distance.
[0008] The feature extraction model is trained based on the self-supervised loss and the feature space reservation loss;
[0009] Acquire target media data, and extract target features from the target media data based on the trained feature extraction model.
[0010] On the other hand, embodiments of this application also provide a model training method, including:
[0011] Obtain sample media data from the current incremental training batch, extract sample features from the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features;
[0012] Multiple target cluster centers are identified, and the multiple target cluster centers are divided into a first center set and a second center set. A first distance between the sample media data and the first center set and a second distance between the sample media data and the second center set are determined. The feature space reservation loss is determined based on the first distance and the second distance.
[0013] The feature extraction model is trained based on the self-supervised loss and the feature space reservation loss.
[0014] On the other hand, embodiments of this application also provide a feature extraction apparatus, including:
[0015] The first processing module is used to acquire sample media data in the current incremental training batch, extract sample features of the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features.
[0016] The second processing module is used to determine multiple target cluster centers, divide the multiple target cluster centers into a first center set and a second center set, determine a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set, and determine the feature space reservation loss based on the first distance and the second distance;
[0017] The first parameter adjustment module is used to train the feature extraction model based on the self-supervised loss and the feature space reservation loss;
[0018] The third processing module is used to acquire target media data and extract target features from the target media data based on the trained feature extraction model.
[0019] Furthermore, the aforementioned first processing module is specifically used for:
[0020] Obtain the first baseline feature in the current incremental training batch, obtain the comparison data of the sample media data, and extract the comparison features of the comparison data based on the feature extraction model;
[0021] A first similarity is determined between the sample feature and the first benchmark feature, and a second similarity is determined between the contrast feature and the first benchmark feature. A contrast loss is determined based on the first similarity and the second similarity, and the contrast loss is used as the self-supervised loss of the feature extraction model.
[0022] Furthermore, the aforementioned first processing module is specifically used for:
[0023] Obtain multiple second benchmark features from the previous incremental training batch, wherein the target cluster center is obtained by clustering multiple second benchmark features;
[0024] The target cluster center is determined as the sampling center, and the distribution variance of the second reference feature with the target cluster center as the cluster center is determined, and the distribution variance is used as the sampling variance;
[0025] Gaussian random sampling is performed based on the sampling center and the sampling variance to obtain the first baseline feature in the current incremental training batch.
[0026] Furthermore, the aforementioned first parameter adjustment module is specifically used for:
[0027] Determine the clustering loss when clustering multiple second benchmark features;
[0028] The target loss is determined based on the self-supervised loss, the feature space reservation loss, and the clustering loss, and the feature extraction model is trained based on the target loss.
[0029] Furthermore, the aforementioned first parameter adjustment module is specifically used for:
[0030] Determine multiple original cluster centers, and determine a third distance between the second benchmark feature and each of the original cluster centers;
[0031] Determine the target distance with the largest value from among the multiple third distances;
[0032] A first quantity of the second benchmark features is determined, and based on the first quantity and the distance to the target, a clustering loss is determined when clustering multiple second benchmark features based on the original cluster centers.
[0033] Furthermore, the aforementioned second processing module is specifically used for:
[0034] The distance between the sample media data and the nearest target cluster center in the first center set is taken as the first distance between the sample media data and the first center set.
[0035] The distance between the sample media data and the nearest target cluster center in the second center set is taken as the second distance between the sample media data and the second center set;
[0036] Determine a second quantity of the sample media data and the difference between the first distance and the second distance, and determine the feature space reservation loss based on the second quantity and the difference.
[0037] Furthermore, the aforementioned first parameter adjustment module is specifically used for:
[0038] At the start of the current incremental training batch, a copy of the feature extraction model is created to obtain the copy model;
[0039] Obtain an initialized encoding model, wherein the encoding model is used to extract features for the enhanced sample media data in the current incremental training batch;
[0040] Using the copy model and the encoding model as the teacher network and the feature extraction model as the student network, the distillation loss of the feature extraction model is determined.
[0041] The target loss is determined based on the self-supervised loss, the feature space reservation loss, and the distillation loss, and the feature extraction model is trained based on the target loss.
[0042] Furthermore, the aforementioned first parameter adjustment module is specifically used for:
[0043] The sample media data was enhanced in different ways to obtain first and second enhanced data.
[0044] The first enhancement feature of the first enhancement data is extracted based on the feature extraction model, the second enhancement feature of the first enhancement data is extracted based on the copy model, a third similarity between the first enhancement feature and the second enhancement feature is determined, and a first loss is determined based on the third similarity.
[0045] Based on the coding model, a third enhancement feature is extracted from the second enhancement data, a fourth similarity is determined between the first enhancement feature and the third enhancement feature, and a second loss is determined based on the fourth similarity.
[0046] The distillation loss of the feature extraction model is determined based on the first loss and the second loss.
[0047] Furthermore, the aforementioned first parameter adjustment module is specifically used for:
[0048] Get the initialized mapping model;
[0049] Based on the mapping model, the first enhanced feature is mapped to the feature space where the second enhanced feature is located to obtain the mapped feature;
[0050] The similarity between the mapped feature and the second enhanced feature is used as the third similarity between the first enhanced feature and the second enhanced feature.
[0051] Furthermore, the aforementioned second processing module is specifically used for:
[0052] The feature space segmentation surface is determined based on the midpoint between the two farthest target cluster centers;
[0053] The segmentation surface is adjusted according to a preset rotation angle step size until the difference in the number of target cluster centers on both sides of the segmentation surface is less than or equal to a preset number threshold. Then, the target cluster centers on one side of the segmentation surface are taken as the first center set, and the target cluster centers on the other side of the segmentation surface are taken as the second center set.
[0054] On the other hand, embodiments of this application also provide a model training apparatus, including:
[0055] The fourth processing module is used to acquire sample media data in the current incremental training batch, extract sample features of the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features.
[0056] The fifth processing module is used to determine multiple target cluster centers, divide the multiple target cluster centers into a first center set and a second center set, determine a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set, and determine the feature space reservation loss based on the first distance and the second distance;
[0057] The second parameter adjustment module is used to train the feature extraction model based on the self-supervised loss and the feature space reservation loss.
[0058] On the other hand, embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described feature extraction method or model training method.
[0059] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is executed by a processor to implement the above-described feature extraction method or model training method.
[0060] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the feature extraction method or model training method described above.
[0061] The embodiments of this application include at least the following beneficial effects: by determining multiple target cluster centers, dividing the multiple target cluster centers into a first center set and a second center set, determining a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set, determining the feature space reservation loss based on the first distance and the second distance, the feature space reservation loss can be used to constrain the distribution of sample features, achieving the effect of feature space reservation, preventing feature conflicts in newly added sample data during incremental training, and finally training the feature extraction model based on the self-supervised loss and the feature space reservation loss, which can improve the training effect of the feature extraction model in the scenario of incremental training combined with self-supervised training, and subsequently improving the accuracy of the features extracted by the feature extraction model when extracting target features of the target media data based on the trained feature extraction model.
[0062] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. Attached Figure Description
[0063] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.
[0064] Figure 1 A schematic diagram illustrating an optional implementation environment provided for an embodiment of this application;
[0065] Figure 2 A schematic diagram of an optional flow of the feature extraction method provided in an embodiment of this application;
[0066] Figure 3 A schematic diagram of an optional computational architecture for the contrast loss provided in an embodiment of this application;
[0067] Figure 4 An optional schematic diagram reserved for the feature space provided in the embodiments of this application;
[0068] Figure 5A schematic diagram of an optional confirmation process for target cluster centers provided in an embodiment of this application;
[0069] Figure 6 A schematic diagram of an optional process for obtaining a first reference feature provided in an embodiment of this application;
[0070] Figure 7 A schematic diagram of an optional training architecture for the feature extraction model provided in an embodiment of this application;
[0071] Figure 8 A schematic diagram of an optional calculation process for distillation loss provided in an embodiment of this application;
[0072] Figure 9 A schematic diagram of another optional training architecture for the feature extraction model provided in the embodiments of this application;
[0073] Figure 10 A schematic diagram of an optional model training method provided in an embodiment of this application;
[0074] Figure 11 A schematic diagram of an optional complete architecture of the feature extraction method provided in the embodiments of this application;
[0075] Figure 12 A schematic diagram of another optional complete architecture for the feature extraction method provided in the embodiments of this application;
[0076] Figure 13 This is a schematic diagram of an optional structure of the feature extraction device provided in the embodiments of this application;
[0077] Figure 14 A schematic diagram of an optional structure of the model training apparatus provided in the embodiments of this application;
[0078] Figure 15 This is a partial structural block diagram of a terminal provided in an embodiment of this application;
[0079] Figure 16 A partial structural block diagram of the server provided in an embodiment of this application. Detailed Implementation
[0080] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0081] It should be noted that in various specific embodiments of this application, when processing data related to the characteristics of the target object, such as target object attribute information or attribute information sets, is required, the permission or consent of the target object will be obtained first. Furthermore, the collection, use, and processing of this data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. The target object may be a user. In addition, when embodiments of this application need to obtain target object attribute information, separate permission or consent from the target object will be obtained through pop-ups or redirection to a confirmation page. Only after obtaining the target object's separate permission or consent will the necessary target object-related data for the normal operation of the embodiments of this application be obtained.
[0082] To facilitate understanding of the technical solutions provided in the embodiments of this application, some key terms used in the embodiments of this application will be explained below:
[0083] Catastrophic forgetting: In incremental training mode, if the task sequence is unlabeled, may be randomly switched, and the same task may not be reproduced for a long time, the model may suddenly lose its knowledge of previous tasks when learning the current task.
[0084] Self-supervised learning (SSL) involves training data without labels, where the supervision signals originate from the data itself.
[0085] Incremental learning (IL) is a learning system that continuously learns new knowledge from new samples while retaining previously learned knowledge.
[0086] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.
[0087] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.
[0088] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.
[0089] In related technologies, under supervised learning, labeled information can be used to constrain the representation of features of different categories of samples, thereby mitigating the impact of feature conflicts to some extent. However, under self-supervised training, the sample data lacks labeled information, making it impossible to use labeled information to resolve feature conflicts. This affects the training effect of the feature extraction model and reduces the accuracy of the features extracted by the model.
[0090] Based on this, embodiments of this application provide a feature extraction method, a model training method, an apparatus, and an electronic device, which can improve the effect of self-supervised incremental training of the feature extraction model and improve the accuracy of the features extracted by the feature extraction model.
[0091] Reference Figure 1 , Figure 1 This is a schematic diagram of an optional implementation environment provided in an embodiment of this application. The implementation environment includes a terminal 101 and a server 102, wherein the terminal 101 and the server 102 are connected through a communication network.
[0092] For example, server 102 can obtain sample media data in the current incremental training batch, extract sample features of the sample media data based on the feature extraction model, determine the self-supervised loss of the feature extraction model based on the sample features, determine multiple target cluster centers, divide the multiple target cluster centers into a first center set and a second center set, determine a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set, determine the feature space reservation loss based on the first distance and the second distance, train the feature extraction model based on the self-supervised loss and the feature space reservation loss, and then receive target media data sent by terminal 101, and extract target features of the target media data based on the trained feature extraction model.
[0093] In addition, after training the feature extraction model, server 102 can send the feature extraction model to terminal 101, which can then apply the model. Alternatively, terminal 101 can obtain sample media data from server 102 and complete the training and application of the feature extraction model locally, and so on.
[0094] The server 102 determines multiple target cluster centers, dividing them into a first center set and a second center set. It then determines a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set. Based on the first and second distances, it determines a feature space reservation loss. This feature space reservation loss can constrain the distribution of sample features, achieving the effect of feature space reservation and preventing feature conflicts from occurring in newly added sample data during incremental training. Finally, the feature extraction model is trained based on the self-supervised loss and the feature space reservation loss. This improves the training effect of the feature extraction model in a scenario combining incremental training and self-supervised training. Subsequently, when extracting target features from the target media data based on the trained feature extraction model, it can improve the accuracy of the features extracted by the feature extraction model.
[0095] Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Additionally, server 102 can also be a node server in a blockchain network.
[0096] Terminal 101 can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, vehicle terminal, etc., but is not limited to these. Terminal 101 and server 102 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the application does not impose any limitations.
[0097] The methods provided in this application can be applied to various technical fields, including but not limited to cloud technology, artificial intelligence, and other technical fields.
[0098] Reference Figure 2 , Figure 2 This is an optional flowchart of a feature extraction method provided in an embodiment of this application. The feature extraction method can be executed by a terminal, or by a server, or by a combination of both. The feature extraction method includes, but is not limited to, the following steps 201 to 204.
[0099] Step 201: Obtain sample media data from the current incremental training batch, extract sample features from the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features.
[0100] In incremental training mode, multiple training batches are typically performed, and the training samples are updated as the number of training batches increases, enabling the trained model to learn new knowledge.
[0101] In one possible implementation, the sample media data consists of training samples from the current incremental training batch. After extracting sample features from the sample media data using a feature extraction model, the self-supervised loss of the feature extraction model is determined based on these features. Specifically, a decoder can be introduced, where the processing of the decoder and the feature extraction model is inversely related. The sample features are input into the decoder to obtain the reconstructed data. The self-supervised loss of the feature extraction model is determined based on the difference between the sample media data and the reconstructed data, thus achieving the effect of self-supervised training. It is understood that the specific method for determining the difference between the sample media data and the reconstructed data can include methods such as mean squared error, and this application embodiment does not limit this. Alternatively, a self-supervised training architecture such as a generative adversarial network can also be used to determine the self-supervised loss of the feature extraction model.
[0102] In addition, in one possible implementation, when determining the self-supervised loss of the feature extraction model based on sample features, the first benchmark features in the current incremental training batch can be obtained, the comparison data of the sample media data can be obtained, and the comparison features of the comparison data can be extracted based on the feature extraction model; the first similarity between the sample features and the first benchmark features, and the second similarity between the comparison features and the first benchmark features can be determined, and the comparison loss can be determined based on the first similarity and the second similarity, and the comparison loss can be used as the self-supervised loss of the feature extraction model.
[0103] Here, the first baseline feature is a general feature used in the current incremental training batch to represent a class of sample data; it can also be called a prototype variable. The first baseline feature and the sample features belong to the same feature space. The first baseline feature can be pre-defined, and each incremental training batch can use the same first baseline feature. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of an optional computational architecture for the contrast loss provided in an embodiment of this application. The sample media data and the contrast data are a training data pair. The similarity between the sample media data and the contrast data is calculated with the first benchmark feature. The contrast loss is then determined by the first similarity and the second similarity, which can be used to train the feature extraction model. At this time, due to the existence of the first benchmark feature, there is no need to label the sample media data with information such as category, thereby achieving the effect of self-supervised incremental training.
[0104] Furthermore, after obtaining the sample features of two sample media data, the contrast loss can actually be calculated directly based on the sample features and the contrast features. However, this method only focuses on the local feature structure of the sample. In the embodiments of this application, the contrast loss is not calculated directly based on the sample features and the contrast features. Instead, a first benchmark feature is introduced, and the first similarity between the sample features and the first benchmark feature, as well as the second similarity between the contrast features and the first benchmark feature, are determined. The contrast loss is determined based on the first similarity and the second similarity, thereby distinguishing the differences in the local feature structure of the sample in the overall space and improving the robustness and transferability of the feature extraction model.
[0105] In one possible implementation, there are multiple first benchmark features. The calculated first and second similarities can be similarity vectors, where each element represents the similarity between a sample feature and each of the first benchmark features, or the similarity between a contrast feature and each of the first benchmark features. The contrast loss is determined based on the first and second similarities by calculating the cross-entropy loss between them and using that cross-entropy loss as the contrast loss.
[0106] In one possible implementation, the sample media data and the comparison data can be two similar media data, meaning the similarity between them can be greater than or equal to a preset similarity threshold. Alternatively, the comparison data can be obtained by enhancing the sample media data, or the sample media data and the comparison data can be obtained by enhancing the same media data using different methods. The sample media data can be image data, text data, or voice data; this application embodiment does not limit this. Correspondingly, the data type of the comparison data is the same as the data type of the sample media data.
[0107] For example, if the comparison data is obtained by enhancing the sample media data in different ways, when the sample media data is image data, the comparison data can be obtained by transforming the same image data through methods such as rotation, cropping, flipping, enlarging, and shrinking; when the sample media data is text data, the comparison data can be obtained by transforming the same text data through methods such as word replacement and word deletion; when the sample media data is speech data, the comparison data can be obtained by transforming the same speech data through methods such as speech frame replacement, speech frame deletion, and noise addition.
[0108] In one possible implementation, the feature extraction model is used to map sample media data or comparison data, thereby outputting sample features of the sample media data or comparison features of the comparison data. The specific structure of the feature extraction model can be determined according to the specific type of sample media data. For example, when the sample media data or comparison data is image data, the feature extraction model may mainly include structures such as convolutional layers; when the sample media data or comparison data is text data, the feature extraction model may mainly include structures such as Transformer layers; when the sample media data or comparison data is speech data, the feature extraction model may mainly include structures such as filtering layers. Of course, the embodiments of this application do not limit the specific structure of the feature extraction model.
[0109] Step 202: Determine multiple target cluster centers, divide the multiple target cluster centers into a first center set and a second center set, determine the first distance between the sample media data and the first center set, and the second distance between the sample media data and the second center set, and determine the feature space reserved loss based on the first distance and the second distance.
[0110] In this model, the target cluster centers and sample features reside in the same feature space. The target cluster centers can be pre-defined, and the distance between different target cluster centers can be used as a standard to ensure that multiple target cluster centers are evenly distributed in the feature space. Alternatively, the target cluster centers can be obtained by clustering multiple first benchmark features, in which case the first benchmark features remain unchanged in different incremental training batches.
[0111] In one possible implementation, when dividing multiple target cluster centers into a first center set and a second center set, a random partitioning method can be used. The first and second center sets can each contain a number of target cluster centers, and the number of target cluster centers in the first and second center sets can be equal or unequal. The first distance between the sample media data and the first center set is used to measure the similarity between the sample features and the features of the target cluster centers in the first center set. This can be achieved by clustering the target cluster centers in the first center set to obtain the set centers, and using the similarity between the sample features and the features of the set centers as the first distance. Alternatively, the first distance can be calculated by calculating the similarity between the sample features and the features of each target cluster center in the first center set, and then using the average of these similarities as the first distance. Another option is to calculate the similarity between the sample features and the features of the nearest target cluster center in the first center set, and use this as the first distance.
[0112] Similarly, the principle for calculating the second distance between the sample media data and the second center set is the same as that for calculating the aforementioned first distance, and will not be repeated here. The calculation of the first or second distance can be performed using cosine similarity, Euclidean distance, Manhattan distance, etc., and this application embodiment does not limit the method.
[0113] The feature space reservation loss is determined by the first distance and the second distance. The feature space reservation loss is used to constrain the relationship between the first distance and the second distance, such that the first distance is greater than the second distance, or the second distance is greater than the first distance. Therefore, when the feature space reservation loss converges, the sample features will be more likely to be distributed in the feature space close to the first center set or the second center set, thereby reducing the feature space occupied by the sample features obtained in the current incremental training batch, reserving feature space for subsequent incremental training batches, and reducing the probability of feature conflicts.
[0114] In one possible implementation, when determining the feature space reservation loss based on the first distance and the second distance, the distance between the sample media data and the nearest target cluster center in the first center set can be used as the first distance between the sample media data and the first center set; the distance between the sample media data and the nearest target cluster center in the second center set can be used as the second distance between the sample media data and the second center set; a second quantity of sample media data and the difference between the first distance and the second distance are determined, and the feature space reservation loss is determined based on the second quantity and the difference between the first distance and the second distance.
[0115] Specifically, it can be expressed as:
[0116]
[0117] Among them, L reserve Let f represent the feature space reservation loss, N represent the second quantity, C1 represent the first center set, C2 represent the second center set, and f represent the feature space reservation loss. i c represents the sample features. k The target cluster center is represented in the first set of centers. c represents the characteristics of the target cluster centers in the first center set. j Indicates the target cluster center in the second center set. The characteristics representing the target cluster centers in the second center set, This represents the distance between the sample media data and the target cluster center in the first center set. λ represents the distance between the sample media data and the target cluster center in the second center set, max represents the closest distance (highest similarity), ReLU is the activation function, i and j are positive integers, and λ is a preset coefficient.
[0118] Reference Figure 4 , Figure 4 This is an optional schematic diagram of feature space reservation provided in the embodiments of this application. Sim1 represents the distance between the sample media data and the nearest target cluster center in the first center set (i.e., the first distance), and Sim2 represents the distance between the sample media data and the nearest target cluster center in the second center set (i.e., the second distance). The feature space reservation loss is calculated by introducing the difference between the first distance and the second distance. When the feature space reservation loss converges, it will satisfy Sim1≥λSim2, so that the sample features will be more inclined to be distributed in the feature space close to the first center set. At the same time, it can effectively simplify the calculation of feature space reservation loss and improve the efficiency of training the feature extraction model.
[0119] In one possible implementation, when dividing multiple target cluster centers into a first center set and a second center set, the segmentation surface of the feature space can be determined based on the midpoint between the two farthest target cluster centers. The segmentation surface is adjusted according to a preset rotation angle step size until the difference in the number of target cluster centers on both sides of the segmentation surface is less than or equal to a preset number threshold. Then, the target cluster centers on one side of the segmentation surface are taken as the first center set, and the target cluster centers on the other side of the segmentation surface are taken as the second center set.
[0120] Specifically, the segmentation surface of the feature space is determined based on the midpoint between the two farthest target cluster centers. Generally, this can initially divide the feature space. On this basis, the segmentation surface is further adjusted according to the preset rotation angle step size until the difference in the number of target cluster centers on both sides of the segmentation surface is less than or equal to the preset number threshold. This indicates that the distribution of target cluster centers on both sides of the segmentation surface is relatively uniform. At this time, the target cluster centers on one side of the segmentation surface are taken as the first center set, and the target cluster centers on the other side of the segmentation surface are taken as the second center set, which can help improve the uniformity of the target cluster center division.
[0121] It is understood that the rotation angle step size can be set according to the actual situation, such as 5 degrees, 10 degrees, etc., and this application embodiment does not limit it.
[0122] Step 203: Train the feature extraction model using self-supervised loss and feature space reservation loss.
[0123] In one possible implementation, after determining the feature space reservation loss and the self-supervised loss, the feature extraction model can be trained based on the feature space reservation loss and the self-supervised loss respectively. Alternatively, the target loss can be determined based on the feature space reservation loss and the self-supervised loss, and the feature extraction model can be trained based on the target loss. Specifically, determining the target loss based on the feature space reservation loss and the self-supervised loss can be achieved by weighting the feature space reservation loss and the self-supervised loss, which can be expressed as follows:
[0124] L total =L ssl +βL reserve
[0125] Among them, L total L represents the target loss. ssl L represents the self-supervised loss (e.g., the contrastive loss mentioned above). reserve This represents the feature space reserved loss, where β is the weight of the feature space reserved loss, which can be set according to the actual situation.
[0126] Step 204: Obtain target media data and extract target features from the target media data based on the trained feature extraction model.
[0127] The target media data refers to the media data from which features are to be extracted. The target features of the target media data are extracted based on the trained feature extraction model, which can be applied to a variety of different scenarios.
[0128] For example, when the target media data is image data, the user can input image data using a terminal for image classification. The terminal then sends the image data to a server. The server extracts the target features of the image data based on a trained feature extraction model, obtains the image classification result based on the target features, and sends the image classification result to the terminal for display. Alternatively, the user can input image data using a terminal for image retrieval. The terminal sends the image data to a server. The server extracts the target features of the image data based on a trained feature extraction model, matches other image data similar to the target image data based on the target features, and sends the corresponding image retrieval result to the terminal for display. Alternatively, the terminal can extract the target features of the image data based on a trained feature extraction model and then match the corresponding image retrieval result within the local map database based on the target features.
[0129] For example, when the target media data is text data, the server can obtain the text data that the terminal has browsed in the past, extract the target features of the text data based on the trained feature extraction model, and then match other text data that are similar to the text data according to the target features and push them to the terminal as recommended content.
[0130] For example, when the target media data is voice data, the terminal can send the voice data to the server. The server can then extract the target features of the image data based on the trained feature extraction model, convert the voice data into text data according to the target features, and then send the text data to the terminal.
[0131] It is understood that the different scenarios described above are for illustrative purposes only and do not limit the application scenarios of the target features extracted by the feature extraction model.
[0132] This application embodiment determines multiple target cluster centers, divides these centers into a first center set and a second center set, determines a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set, and determines a feature space reservation loss based on the first and second distances. This feature space reservation loss can constrain the distribution of sample features, achieving the effect of feature space reservation and preventing feature conflicts in newly added sample data during incremental training. Finally, the feature extraction model is trained based on the self-supervised loss and the feature space reservation loss. This improves the training effect of the feature extraction model in a scenario combining incremental training and self-supervised training. Subsequently, when extracting target features from target media data based on the trained feature extraction model, the accuracy of the features extracted by the feature extraction model can be improved.
[0133] In one possible implementation, based on Figure 3 The architecture shown allows for updating the baseline features in each incremental training batch. This means the first baseline feature in the current incremental training batch is not fixed, thus improving the diversity of sample features. Specifically, when obtaining the first baseline feature in the current incremental training batch, multiple second baseline features from the previous incremental training batch can be obtained. The target cluster center is determined as the sampling center, and the variance of the distribution of the second baseline features with the target cluster center as the cluster center is determined. This variance is used as the sampling variance. Gaussian random sampling is then performed based on the sampling center and the sampling variance to obtain the first baseline feature in the current incremental training batch.
[0134] The target cluster center is obtained by clustering multiple second benchmark features. Specifically, multiple original cluster centers can be initialized, and multiple second benchmark features can be clustered according to the original cluster centers to obtain multiple clusters. The cluster center of the cluster is used as the target cluster center.
[0135] In this scenario, the target cluster centers in the current incremental training batch are based on those determined in the previous incremental training batch. The second baseline feature is the baseline feature from the previous incremental training batch, and its principle is similar to that of the first baseline feature, used to characterize the general features of a class of sample data. The original cluster centers are the initial cluster centers, which can be obtained through random initialization. The number of original cluster centers can also be determined according to the actual situation.
[0136] For example, refer to Figure 5 , Figure 5This is a schematic diagram of an optional confirmation process for the target cluster center provided in an embodiment of this application. It is assumed that there are three original cluster centers, namely cluster center A, cluster center B, and cluster center C, and nine second baseline features, namely category a1, category a2, category b1, category b2, category b3, category c1, category c2, category c3, and category c4. Next, the second baseline features are clustered. Finally, categories a1 and a2 form a cluster, with cluster center A as the cluster center; categories b1, b2, and b3 form a cluster, with cluster center B as the cluster center; and categories c1, c2, c3, and c4 form a cluster, with cluster center C as the cluster center. After obtaining the cluster centers, Gaussian random sampling can be performed based on each cluster center to obtain the first baseline features in the current incremental training batch, which are category a3, category a4, category b4, category b5, category b6, category c5, category c6, category c7 and category c8.
[0137] In one possible implementation, refer to Figure 6 , Figure 6 This is a schematic diagram of an optional process for obtaining the first benchmark feature according to an embodiment of this application. During training, a queue is first set up to store the second benchmark feature from the previous incremental training batch. After entering the current training batch, the second benchmark feature from the previous incremental training batch is read from the queue. Using the target cluster center of the second benchmark feature as the sampling center and the distribution variance of the second benchmark feature as the sampling variance, Gaussian random sampling is performed to obtain the first benchmark feature. The original second benchmark feature in the queue is replaced by the first benchmark feature. Similarly, in the next incremental training batch, the target cluster center of the first benchmark feature in the current incremental training batch is used as the sampling center, and Gaussian random sampling is performed in a similar manner to obtain the third benchmark feature. The third benchmark feature is used to calculate the contrastive loss in the next incremental training batch.
[0138] As can be seen, by performing Gaussian random sampling based on the sampling center and sampling variance, the first baseline feature in the current incremental training batch can be obtained, which can achieve the updating effect of the baseline features in each incremental training batch. Furthermore, by performing Gaussian random sampling based on the sampling center and sampling variance, the baseline features in each incremental training batch are randomly distributed, which can effectively enrich the representation of the baseline features, increase the number of categories indicated by the baseline features, and improve the accuracy of the contrast loss.
[0139] Furthermore, since the first baseline feature is updated with each incremental training batch, the target cluster center is also updated in each incremental training batch. Compared with the method of presetting a fixed target cluster center, the feature space reservation loss can be more flexible and reliable.
[0140] It is understandable that the process of clustering multiple second-benchmark features to obtain the target cluster centers can actually involve multiple clustering operations. During this process, the cluster centers continuously change based on the original cluster centers. Therefore, when clustering multiple second-benchmark features based on the original cluster centers, a clustering loss can be introduced. The original cluster centers at which the clustering loss converges are the target cluster centers. Correspondingly, when training the feature extraction model based on self-supervised loss and feature space reservation loss, the clustering loss for clustering multiple second-benchmark features can be specifically determined. The target loss is then determined based on the self-supervised loss, feature space reservation loss, and clustering loss, and the feature extraction model is trained based on the target loss.
[0141] By introducing clustering loss, the accuracy of clustering multiple second-benchmark features can be improved. At the same time, when determining the target loss, in addition to the loss reserved in the feature space and the extraction, clustering loss is further added, making the calculation of the target loss more diverse, improving the reliability of the target loss, and thus improving the training effect of the feature extraction model.
[0142] Based on this, the target loss can also be expressed as:
[0143] L total =L ssl +αL cluster +βL reserve
[0144] Among them, L total L represents the target loss. ssl L represents the self-supervised loss (e.g., the contrastive loss mentioned above). reserve L represents the feature space reserved loss. cluster The clustering loss is represented by α, where α is the weight of the clustering loss and β is the weight of the feature space reservation loss. These values can be set according to the actual situation.
[0145] In one possible implementation, when determining the clustering loss when clustering multiple second benchmark features, specifically, multiple original cluster centers can be determined, and a third distance between the second benchmark features and each original cluster center can be determined; the target distance with the largest value can be determined from the multiple third distances; a first number of second benchmark features can be determined, and the clustering loss when clustering multiple second benchmark features based on the original cluster centers can be determined according to the first number and the target distance.
[0146] The clustering loss can be specifically expressed as:
[0147]
[0148] Among them, L cluster Represents the clustering loss, n p Indicates the first quantity. Indicates the second reference feature, This represents the original cluster centers, and max represents the nearest cluster (highest similarity). Represents the third distance, n c This represents the number of original cluster centers, where i and j are positive integers.
[0149] In one possible implementation, in order to unify the number of target cluster centers in different incremental training stages, the number of original cluster centers can be fixed. Even if the number of original cluster centers is fixed, the clustering loss is calculated in the above way. During the formation of clusters in the clustering process, it is possible to have empty cluster centers or overlapping cluster centers, which is suitable for self-supervised training with uncertain number of categories, improves the flexibility of training, and expands the scope of training applicability.
[0150] As the incremental batches increase, the amount of sample media data also gradually increases. When there is insufficient storage space to store the sample media data from historical incremental training batches, using the feature extraction network trained in historical incremental training batches to extract sample features from the sample media data in the current incremental training batch will result in biased features for newly added sample media data, often leading to catastrophic forgetting and thus affecting the training effect of the feature extraction model. Based on this, in one possible implementation, when training the feature extraction model based on self-supervised loss and feature space reservation loss, a copy of the feature extraction model can be created at the beginning of the current incremental training batch to obtain the copy model; an initialized encoding model can be obtained; the copy model and the encoding model can be used as the teacher network, and the feature extraction model as the student network; the distillation loss of the feature extraction model can be determined; the target loss can be determined based on the self-supervised loss, feature space reservation loss, and distillation loss; and the feature extraction model can be trained based on the target loss.
[0151] Specifically, refer to Figure 7 , Figure 7This is a schematic diagram of an optional training architecture for the feature extraction model provided in the embodiments of this application. In the current incremental training batch, since the replica model is created at the beginning of the current incremental training batch, that is, the replica model is the feature extraction model in the previous incremental training batch, the feature extraction model is trained by distillation through the replica model. The replica model can provide distillation information to the feature extraction model in the current incremental training batch at the feature representation level, so that when the replica model and the feature extraction model are used to extract features from the same sample media data, the regions mapped in the feature space are similar.
[0152] Building upon this foundation, an encoding model is further introduced. This model is used to extract features from the enhanced sample media data in the current incremental training batch. The encoding model is used only in the current incremental training batch, is randomly initialized in each batch, and is trained together with the feature extraction model. Since the encoding model is used for feature extraction from the enhanced sample media data, it can provide distillation information to the feature extraction model in the current incremental training batch at the feature relationship level. This ensures that the mapped regions in the feature space are similar when extracting features from sample media data before and after enhancement, or from sample media data obtained through different enhancement methods. The distillation loss can be divided into two parts: a first loss between the feature extraction model and the copy model, and a second loss between the feature extraction model and the encoding model. For example, the distillation loss can be the sum of the first and second losses.
[0153] As can be seen, the training architecture of the feature extraction model provided in this application embodiment can perform distillation training at the feature representation level through the replica model and at the feature relation level through the encoding model, even when there is insufficient storage space to store the sample media data in the historical incremental training batches. This can alleviate the problem of catastrophic forgetting to a certain extent and improve the reliability of training the feature extraction model.
[0154] Based on this, the target loss can also be specifically expressed as:
[0155] L total =L ssl +βL reserve +γL distill
[0156] Among them, L total L represents the target loss. ssl L represents the self-supervised loss (e.g., the contrastive loss mentioned above). reserve L represents the feature space reserved loss. distillThe distillation loss is represented by γ, which is the weight of the distillation loss, and β is the weight of the feature space reserved loss. These values can be set according to the actual situation.
[0157] In one possible implementation, refer to Figure 8 , Figure 8 This is a schematic diagram of an optional calculation process for distillation loss provided in an embodiment of this application. Specifically, when determining the distillation loss of the feature extraction model, the following steps can be taken: first enhanced data and second enhanced data obtained by enhancing sample media data in different ways; first enhanced features of the first enhanced data are extracted based on the feature extraction model; second enhanced features of the first enhanced data are extracted based on the copy model; a third similarity is determined between the first and second enhanced features; a first loss is determined based on the third similarity; a third enhanced feature of the second enhanced data is extracted based on the encoding model; a fourth similarity is determined between the first and third enhanced features; a second loss is determined based on the fourth similarity; and the distillation loss of the feature extraction model is determined based on the first and second losses.
[0158] Similar to the principle of comparison data, when the sample media data is image data, the first or second enhanced data can be obtained from the same image data through methods such as rotation, cropping, flipping, enlarging, and shrinking; when the sample media data is text data, the first or second enhanced data can be obtained from the same text data through methods such as word replacement and word deletion; when the sample media data is speech data, the first or second enhanced data can be obtained from the same speech data through methods such as speech frame replacement, speech frame deletion, and noise addition. Of course, the enhancement methods for obtaining the first and second enhanced data are different. For example, if the first enhanced data is obtained through rotation, then the second enhanced data can be obtained through cropping.
[0159] Understandably, in Figure 3 Under the architecture shown, the comparison data and the first or second augmented data can be reused to improve the efficiency of data processing.
[0160] Specifically, the above distillation loss can be expressed as:
[0161]
[0162] in, Indicates the first loss. This represents the second loss, where N represents the number of sample media data, and f i t f represents the first enhanced feature. i t-1 This indicates the second enhancement feature. p represents the third similarity.i,k The fourth similarity is represented by i, k, and t, where i, k, and t are positive integers.
[0163] The fourth similarity mentioned above can be a probability distribution obtained through the Softmax function, specifically expressed as:
[0164]
[0165] Where, p i,k f represents the fourth similarity. i f represents the first enhanced feature. k ′ represents the third enhancement feature, τ is the temperature parameter, and i and j are positive integers.
[0166] In one possible implementation, refer to Figure 9 , Figure 9 This is a schematic diagram of another optional training architecture for the feature extraction model provided in the embodiments of this application. Figure 7 Based on the architecture shown, a mapping model is further introduced. In this case, when determining the third similarity between the first enhanced feature and the second enhanced feature, the initial mapping model can be obtained. Based on the mapping model, the first enhanced feature is mapped to the feature space where the second enhanced feature is located to obtain the mapped feature. The similarity between the mapped feature and the second enhanced feature is used as the third similarity between the first enhanced feature and the second enhanced feature.
[0167] Specifically, after extracting the first enhanced feature using the feature extraction model, the similarity between the first and second enhanced features is not directly determined. This is because training in this way forces the first and second enhanced features to be similar, which limits the feature extraction model's ability to learn new knowledge. Therefore, a mapping model can be further introduced. Based on this model, the first enhanced feature is mapped to the feature space containing the second enhanced feature, resulting in mapped features. The similarity between these mapped features and the second enhanced feature is used as the third similarity between the first and second enhanced features. In this case, when the first loss converges, it indicates that the mapping model can perfectly map the first enhanced feature to the feature space containing the second enhanced feature, thus indicating that the feature space containing the features extracted by the feature extraction model has good feature representation performance. Therefore, by further introducing a mapping model and calculating the third similarity, the scope of knowledge learned by the feature extraction model can be expanded, thereby improving the training effect of the feature extraction model.
[0168] In one possible implementation, by introducing distillation loss, the network structure of the feature extraction model in the current incremental training batch (serving as the student network) can be pruned during the training of the feature extraction model based on the target loss. For example, if the feature extraction model includes convolutional layers, the number of convolutional layers in the current incremental training batch can be reduced; or if the feature extraction model includes Transformer layers, the number of Transformer layers in the current incremental training batch can be reduced; or if the feature extraction model includes filtering layers, the number of filtering layers in the current incremental training batch can be reduced, and so on. By pruning the network structure of the feature extraction model, a distillation effect is achieved, thereby simplifying the network structure of the feature extraction model.
[0169] Reference Figure 10 , Figure 10 This is an optional flowchart of a model training method provided in an embodiment of this application. The model training method can be executed by a terminal, or by a server, or by a combination of a terminal and a server. The feature extraction method includes, but is not limited to, the following steps 1001 to 1003.
[0170] Step 1001: Obtain sample media data from the current incremental training batch, extract sample features from the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features;
[0171] Step 1002: Determine multiple target cluster centers, divide the multiple target cluster centers into a first center set and a second center set, determine the first distance between the sample media data and the first center set, and the second distance between the sample media data and the second center set, and determine the feature space reservation loss based on the first distance and the second distance;
[0172] Step 1003: Train the feature extraction model based on the self-supervised loss and the feature space reservation loss.
[0173] The aforementioned model training method and feature extraction method are based on the same inventive concept. By determining multiple target cluster centers, these centers are divided into a first center set and a second center set. A first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set are determined. Based on the first and second distances, a feature space reservation loss is determined. This feature space reservation loss can be used to constrain the distribution of sample features, achieving the effect of feature space reservation and preventing feature conflicts from occurring in newly added sample data during incremental training. Finally, the feature extraction model is trained based on the self-supervised loss and the feature space reservation loss, which can improve the training effect of the feature extraction model in the scenario of incremental training combined with self-supervised training.
[0174] The principles of steps 1001 to 1003 are similar to those of steps 201 to 203. For details, please refer to the previous explanation of the principles, which will not be repeated here.
[0175] The following uses a classification task as an example to illustrate the effect of the model training method provided in this application embodiment. Referring to Table 1, Table 1 shows the classification accuracy comparison data provided in this application embodiment. Table 1 compares the feature extraction model trained by the model training method provided in this application embodiment with the feature extraction model trained by the common self-supervised training method in related technologies on the experimental results of 5-tasks on the CIFAR100 and ImageNet100 datasets. The evaluation index is the top-1 classification accuracy after linear probing. It can be seen that the feature extraction model trained by the model training method provided in this application embodiment has a significantly higher accuracy on this classification task than the common self-supervised training method in related technologies.
[0176] method CIFAR100 ImageNet100 Common self-supervised training methods in related technologies 57.8 66.0 The model training method provided in this application embodiment 64.15 67.92
[0177] The principle of the feature extraction method provided in this application embodiment will be explained in detail below with an example.
[0178] The embodiments of this application are widely applicable to various common application scenarios. For example, in classification scenarios, it is often impossible to obtain data for all categories simultaneously, and the data volume is large, making manual annotation costly. Furthermore, servers that receive online data in real time can only acquire data for a portion of the categories at any given time, and the data volume is large. After training the feature extraction model, if data for new categories is acquired, retraining with all past data would be very costly, or storage space might be limited, making it impossible to acquire all past data. Secondly, when the data volume is very large, the cost of manual annotation becomes extremely high. However, the feature extraction method proposed in this application, based on incremental training combined with self-supervised training, requires no annotation information and can be applied to scenarios with large data volumes.
[0179] Reference Figure 11 , Figure 11 A schematic diagram of an optional complete architecture for the feature extraction method provided in the embodiments of this application is shown below:
[0180] At the start of the current incremental training batch, a copy of the feature extraction model is created to obtain the copy model. Simultaneously, the queue stores the second baseline features from the previous incremental training batch. These second baseline features are first retrieved from the queue, and clustering is performed on multiple second baseline features. During the clustering process, the clustering loss for clustering multiple second baseline features is determined, denoted as L. cluster The specific calculation principle can be found in the previous explanation, and will not be repeated here.
[0181] After clustering multiple second baseline features, multiple target cluster centers are obtained. Using the target cluster center as the sampling center, the distribution variance of the second baseline feature with the target cluster center as the cluster center is determined. The distribution variance is used as the sampling variance. Gaussian random sampling is performed according to the sampling center and the sampling variance to obtain the first baseline feature in the current incremental training batch. The first baseline feature replaces the original second baseline feature in the queue.
[0182] Next, sample data pairs are acquired, including sample media data and corresponding comparison data. Data augmentation is performed on the sample media data to obtain first augmented data. Comparison features are extracted from the comparison data based on the feature extraction model. The first similarity between the sample features and the first benchmark features, and the second similarity between the comparison features and the first benchmark features are determined. The comparison loss is determined based on the first and second similarities and is used as the self-supervised loss of the feature extraction model, denoted as L. ssl The specific calculation principle can be found in the previous explanation, and will not be repeated here.
[0183] Next, the multiple target cluster centers obtained by clustering multiple second benchmark features are divided into a first center set and a second center set. A first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set are determined. Based on the first and second distances, the feature space reservation loss is determined, denoted as L. reserve The specific calculation principle can be found in the previous explanation, and will not be repeated here.
[0184] Next, we obtain the initialized encoding model and the initialized mapping model. Using the copy model and encoding model as the teacher network and the feature extraction model as the student network, we perform a different type of data augmentation on the sample media data to obtain second-augmented data. Based on the feature extraction model, we extract the first augmented feature of the first augmented data; based on the copy model, we extract the second augmented feature of the first augmented data; based on the mapping model, we map the first augmented feature to the feature space containing the second augmented feature to obtain the mapped feature. We determine the third similarity between the mapped feature and the second augmented feature, and determine the first loss based on the third similarity. Based on the encoding model, we extract the third augmented feature of the second augmented data, determine the fourth similarity between the first and third augmented features, and determine the second loss based on the fourth similarity. Finally, we determine the distillation loss of the feature extraction model, denoted as L, based on the first and second losses. distill The specific calculation principle can be found in the previous explanation, and will not be repeated here.
[0185] The target loss is determined based on the aforementioned clustering loss, self-supervised loss, feature space reservation loss, and distillation loss. The parameters of the feature extraction network, encoding model, and mapping model are then adjusted according to the target loss. The target loss can be expressed as:
[0186] L total =L ssl +αL cluster +βL reserve +γL distill
[0187] Among them, L total L represents the target loss. ssl L represents self-monitored loss. cluster L represents the clustering loss. reserve L represents the feature space reserved loss. distill The values represent the distillation loss, α is the weight of the clustering loss, γ is the weight of the distillation loss, and β is the weight of the feature space reservation loss. These values can be set according to the actual situation.
[0188] At this point, the training of the current incremental training batch is complete. The queue currently stores the first baseline feature from this batch. At the start of the next incremental training batch, the first baseline feature from the current batch will become the second baseline feature from the previous batch, and so on, for the next batch to train using a similar process. This cycle continues until the number of incremental training batches reaches a preset batch threshold, completing the training of the feature extraction model. After training the feature extraction model, it can be used to extract features for downstream tasks.
[0189] As can be seen, the above training architecture determines multiple target cluster centers, dividing them into a first center set and a second center set. It then determines the first distance between the sample media data and the first center set, and the second distance between the sample media data and the second center set. Based on these first and second distances, it determines the feature space reservation loss. This feature space reservation loss can constrain the distribution of sample features, achieving the effect of feature space reservation and preventing feature conflicts in newly added sample data during incremental training. Furthermore, by introducing clustering loss to determine target cluster centers, Gaussian random sampling is performed based on the sampling center and sampling variance to obtain the first cluster center in the current incremental training batch. The baseline features can achieve the same update effect as the baseline features in each incremental training batch. At the same time, distillation loss is introduced, which can be used to perform distillation training at the feature representation level through the copy model, and at the feature relation level through the encoding model. This can alleviate the problem of catastrophic forgetting to some extent. Finally, the feature extraction model is trained based on self-supervised loss, clustering loss, feature space reservation loss and distillation loss. This can improve the training effect of the feature extraction model in the scenario of incremental training combined with self-supervised training. Subsequently, when extracting target features of target media data based on the trained feature extraction model, the accuracy of the features extracted by the feature extraction model can be improved.
[0190] Reference Figure 12 , Figure 12 A schematic diagram of another optional complete architecture for the feature extraction method provided in the embodiments of this application is shown below:
[0191] At the start of the current incremental training batch, a copy of the feature extraction model is created to obtain the copy model. Sample data pairs are acquired, including sample media data and corresponding contrast data. Data augmentation is performed on the sample media data to obtain the first augmented data. Contrast features are extracted from the contrast data based on the feature extraction model. The first similarity between the sample features and the first benchmark features, and the second similarity between the contrast features and the first benchmark features are determined. The contrast loss is determined based on the first and second similarities and is used as the self-supervised loss of the feature extraction model, denoted as L. ssl The specific calculation principle can be found in the previous explanation, and will not be repeated here.
[0192] Next, multiple preset target cluster centers are obtained, and these centers are divided into a first center set and a second center set. A first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set are determined. Based on the first and second distances, a feature space retention loss, denoted as L, is determined. reserve The specific calculation principle can be found in the previous explanation, and will not be repeated here.
[0193] Next, we obtain the initialized encoding model and the initialized mapping model. Using the copy model and encoding model as the teacher network and the feature extraction model as the student network, we perform a different type of data augmentation on the sample media data to obtain second-augmented data. Based on the feature extraction model, we extract the first augmented feature of the first augmented data; based on the copy model, we extract the second augmented feature of the first augmented data; based on the mapping model, we map the first augmented feature to the feature space containing the second augmented feature to obtain the mapped feature. We determine the third similarity between the mapped feature and the second augmented feature, and determine the first loss based on the third similarity. Based on the encoding model, we extract the third augmented feature of the second augmented data, determine the fourth similarity between the first and third augmented features, and determine the second loss based on the fourth similarity. Finally, we determine the distillation loss of the feature extraction model, denoted as L, based on the first and second losses. distill The specific calculation principle can be found in the previous explanation, and will not be repeated here.
[0194] The target loss is determined based on the aforementioned self-supervised loss, feature space reservation loss, and distillation loss. The parameters of the feature extraction network, decoder, encoding model, and mapping model are then adjusted according to the target loss. The target loss can be expressed as:
[0195] L total =L ssl +βL reserve +γL distill
[0196] Among them, L total L represents the target loss. ssl L represents self-monitored loss. reserve L represents the feature space reserved loss. distill The distillation loss is represented by γ, which is the weight of the distillation loss, and β is the weight of the feature space reserved loss. These values can be set according to the actual situation.
[0197] This completes the training of the current incremental training batch. This process continues until the number of incremental training batches reaches the preset batch threshold, thus completing the training of the feature extraction model. After training, the model can be used to extract features for downstream tasks.
[0198] As can be seen, the above training architecture determines multiple target cluster centers, dividing them into a first center set and a second center set. It then determines the first distance between the sample media data and the first center set, and the second distance between the sample media data and the second center set. Based on these first and second distances, it determines the feature space reservation loss. This feature space reservation loss constrains the distribution of sample features, achieving the effect of feature space reservation and preventing feature conflicts from newly added sample data during incremental training. Simultaneously, it introduces distillation loss, enabling distillation training at both the feature representation level (through the copy model) and the feature relationship level (through the encoding model), thus mitigating the catastrophic forgetting problem to some extent. Finally, it trains the feature extraction model using self-supervised loss, feature space reservation loss, and distillation loss. This improves the training effect of the feature extraction model in a scenario combining incremental training and self-supervised training. Subsequently, when extracting target features from target media data based on the trained feature extraction model, it enhances the accuracy of the extracted features.
[0199] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.
[0200] Reference Figure 13 , Figure 13 This is a schematic diagram of an optional structure of the feature extraction device provided in an embodiment of this application. The feature extraction device 1300 includes:
[0201] The first processing module 1301 is used to acquire sample media data in the current incremental training batch, extract sample features of the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features.
[0202] The second processing module 1302 is used to determine multiple target cluster centers, divide the multiple target cluster centers into a first center set and a second center set, determine a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set, and determine the feature space reservation loss based on the first distance and the second distance.
[0203] The first parameter adjustment module 1303 is used to train the feature extraction model based on the self-supervised loss and the feature space reservation loss;
[0204] The third processing module 1304 is used to acquire target media data and extract target features from the target media data based on the trained feature extraction model.
[0205] Furthermore, the aforementioned first processing module 1301 is specifically used for:
[0206] Obtain the first baseline feature in the current incremental training batch, obtain the comparative data of the sample media data, and extract the comparative features of the comparative data based on the feature extraction model;
[0207] Determine the first similarity between the sample features and the first benchmark features, and the second similarity between the contrast features and the first benchmark features. Determine the contrast loss based on the first and second similarities, and use the contrast loss as the self-supervised loss of the feature extraction model.
[0208] Furthermore, the aforementioned first processing module 1301 is specifically used for:
[0209] Obtain multiple second benchmark features from the previous incremental training batch, where the target cluster center is obtained by clustering multiple second benchmark features;
[0210] The target cluster center is determined as the sampling center, and the distribution variance of the second benchmark feature with the target cluster center as the cluster center is determined. The distribution variance is used as the sampling variance.
[0211] Gaussian random sampling is performed based on the sampling center and sampling variance to obtain the first baseline feature in the current incremental training batch.
[0212] Furthermore, the aforementioned first parameter adjustment module 1303 is specifically used for:
[0213] Determine the clustering loss when clustering multiple second baseline features;
[0214] The target loss is determined based on the self-supervised loss, feature space reservation loss, and clustering loss, and the feature extraction model is trained based on the target loss.
[0215] Furthermore, the aforementioned first parameter adjustment module 1303 is specifically used for:
[0216] Multiple initial cluster centers are identified, and a third distance between the second baseline feature and each initial cluster center is determined.
[0217] Determine the target distance with the largest value from multiple third distances;
[0218] Determine the first quantity of the second benchmark features, and based on the distance between the first quantity and the target, determine the clustering loss when clustering multiple second benchmark features based on the original cluster centers.
[0219] Furthermore, the aforementioned second processing module 1302 is specifically used for:
[0220] The distance between the sample media data and the nearest target cluster center in the first center set is taken as the first distance between the sample media data and the first center set.
[0221] The distance between the sample media data and the nearest target cluster center in the second center set is taken as the second distance between the sample media data and the second center set;
[0222] Determine the second quantity of sample media data and the difference between the first and second distances, and determine the feature space reserved loss based on the second quantity and the difference.
[0223] Furthermore, the aforementioned first parameter adjustment module 1303 is specifically used for:
[0224] At the start of the current incremental training batch, a copy of the feature extraction model is created, resulting in a copy model;
[0225] Obtain the initialized encoding model, which is used to extract features for the augmented sample media data in the current incremental training batch;
[0226] The copy model and encoding model are used as teacher networks, and the feature extraction model is used as student networks. The distillation loss of the feature extraction model is determined.
[0227] The target loss is determined based on the self-supervised loss, feature space reservation loss, and distillation loss, and the feature extraction model is trained based on the target loss.
[0228] Furthermore, the aforementioned first parameter adjustment module 1303 is specifically used for:
[0229] The first and second enhanced data were obtained by enhancing sample media data using different methods.
[0230] First augmentation features are extracted from the first augmented data based on the feature extraction model, second augmentation features are extracted from the first augmented data based on the copy model, a third similarity is determined between the first augmentation features and the second augmentation features, and a first loss is determined based on the third similarity.
[0231] The third enhancement feature of the second enhancement data is extracted based on the coding model, the fourth similarity between the first enhancement feature and the third enhancement feature is determined, and the second loss is determined based on the fourth similarity.
[0232] The distillation loss of the feature extraction model is determined based on the first loss and the second loss.
[0233] Furthermore, the aforementioned first parameter adjustment module 1303 is specifically used for:
[0234] Get the initialized mapping model;
[0235] Based on the mapping model, the first enhanced feature is mapped to the feature space where the second enhanced feature is located to obtain the mapped feature;
[0236] The similarity between the mapped feature and the second enhanced feature is used as the third similarity between the first and second enhanced features.
[0237] Furthermore, the aforementioned second processing module 1302 is specifically used for:
[0238] The feature space segmentation surface is determined based on the midpoint between the two farthest target cluster centers;
[0239] The segmentation plane is adjusted according to the preset rotation angle step size until the difference in the number of target cluster centers on both sides of the segmentation plane is less than or equal to the preset number threshold. Then, the target cluster centers on one side of the segmentation plane are taken as the first center set, and the target cluster centers on the other side of the segmentation plane are taken as the second center set.
[0240] The aforementioned feature extraction device 1300 and feature extraction method are based on the same inventive concept. Therefore, the feature extraction device 1300 determines multiple target cluster centers, divides these centers into a first center set and a second center set, determines a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set, and determines a feature space reservation loss based on the first and second distances. This feature space reservation loss can constrain the distribution of sample features, achieving the effect of feature space reservation and preventing feature conflicts in newly added sample data during incremental training. Finally, the feature extraction model is trained based on the self-supervised loss and the feature space reservation loss. This improves the training effect of the feature extraction model in a scenario combining incremental training and self-supervised training. Subsequently, when extracting target features from target media data based on the trained feature extraction model, the accuracy of the features extracted by the feature extraction model can be improved.
[0241] Reference Figure 14 , Figure 14 This is a schematic diagram of an optional structure of the model training apparatus provided in an embodiment of this application. The model training apparatus 1400 includes:
[0242] The fourth processing module 1401 is used to acquire sample media data in the current incremental training batch, extract sample features of the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features.
[0243] The fifth processing module 1402 is used to determine multiple target cluster centers, divide the multiple target cluster centers into a first center set and a second center set, determine a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set, and determine the feature space reservation loss based on the first distance and the second distance;
[0244] The second parameter adjustment module 1403 is used to train the feature extraction model based on the self-supervised loss and the feature space reservation loss.
[0245] The aforementioned model training device 1400 and model training method are based on the same inventive concept. Therefore, the model training device 1400 determines multiple target cluster centers, divides the multiple target cluster centers into a first center set and a second center set, determines a first distance between the sample media data and the first center set, and a second distance between the sample media data and the second center set, and determines the feature space reservation loss based on the first distance and the second distance. The feature space reservation loss can be used to constrain the distribution of sample features, thereby achieving the effect of feature space reservation and preventing feature conflicts from occurring in newly added sample data during incremental training. Finally, the feature extraction model is trained based on the self-supervised loss and the feature space reservation loss, which can improve the training effect of the feature extraction model in the scenario of incremental training combined with self-supervised training.
[0246] The electronic device provided in this application embodiment for performing the above-described feature extraction method or model training method can be a terminal, as shown below. Figure 15 , Figure 15 This is a partial structural block diagram of a terminal provided in an embodiment of this application. The terminal includes: a radio frequency (RF) circuit 1510, a memory 1520, an input unit 1530, a display unit 1540, a sensor 1550, an audio circuit 1560, a wireless fidelity (WiFi) module 1570, a processor 1580, and a power supply 1590, among other components. Those skilled in the art will understand that... Figure 15 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0247] The RF circuit 1510 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 1580; in addition, it transmits uplink data to the base station.
[0248] The memory 1520 can be used to store software programs and modules, and the processor 1580 executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory 1520.
[0249] The input unit 1530 can be used to receive input numeric or character information, and to generate key signal inputs related to the terminal's settings and function control. Specifically, the input unit 1530 may include a touch panel 1531 and other input devices 1532.
[0250] The display unit 1540 can be used to display input or provided information, as well as various menus of the terminal. The display unit 1540 may include a display panel 1541.
[0251] Audio circuitry 1560, speaker 1561, and microphone 1562 provide an audio interface.
[0252] In this embodiment, the processor 1580 included in the terminal can execute the feature extraction method or model training method of the previous embodiment.
[0253] The electronic device provided in this application embodiment for performing the above-described feature extraction method or model training method can also be a server, see below. Figure 16 , Figure 16 The diagram illustrates a partial structural block of a server provided in this application embodiment. The server 1600 can vary significantly due to different configurations or performance characteristics. It may include one or more Central Processing Units (CPUs) 1622 (e.g., one or more processors) and a memory 1632, and one or more storage media 1630 (e.g., one or more mass storage devices) for storing application programs 1642 or data 1644. The memory 1632 and storage media 1630 may be temporary or persistent storage. The program stored in the storage media 1630 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server 1600. Furthermore, the CPU 1622 may be configured to communicate with the storage media 1630 and execute the series of instruction operations in the storage media 1630 on the server 1600.
[0254] Server 1600 may also include one or more power supplies 1626, one or more wired or wireless network interfaces 1650, one or more input / output interfaces 1658, and / or one or more operating systems 1641, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.
[0255] The processor in the server 1600 can be used to execute feature extraction methods or model training methods.
[0256] This application also provides a computer-readable storage medium for storing program code, which is used to execute the feature extraction method or model training method of the foregoing embodiments.
[0257] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the feature extraction method or model training method described above.
[0258] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.
[0259] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0260] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.
[0261] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.
[0262] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0263] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0264] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0265] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.
[0266] The above provides a detailed description of the preferred embodiments of this application. However, this application is not limited to the above-described embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. A feature extraction method, characterized in that, include: Obtain sample media data from the current incremental training batch, extract sample features from the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features; Multiple target cluster centers are identified. A segmentation plane for the feature space is determined based on the midpoint between the two farthest target cluster centers. The segmentation plane is adjusted according to a preset rotation angle step size until the difference in the number of target cluster centers on both sides of the segmentation plane is less than or equal to a preset number threshold. The target cluster centers on one side of the segmentation plane are designated as a first center set, and the target cluster centers on the other side of the segmentation plane are designated as a second center set. The distance between the sample media data and the nearest target cluster center in the first center set is taken as the first distance between the sample media data and the first center set. The distance between the sample media data and the nearest target cluster center in the second center set is taken as the second distance between the sample media data and the second center set. A second quantity of sample media data and the difference between the first and second distances are determined. A feature space reservation loss is determined based on the second quantity and the difference. The feature extraction model is trained based on the self-supervised loss and the feature space reservation loss; Acquire target media data, and extract target features from the target media data based on the trained feature extraction model.
2. The feature extraction method according to claim 1, characterized in that, The step of determining the self-supervised loss of the feature extraction model based on the sample features includes: Obtain the first baseline feature in the current incremental training batch, obtain the comparison data of the sample media data, and extract the comparison features of the comparison data based on the feature extraction model; A first similarity is determined between the sample feature and the first benchmark feature, and a second similarity is determined between the contrast feature and the first benchmark feature. A contrast loss is determined based on the first similarity and the second similarity, and the contrast loss is used as the self-supervised loss of the feature extraction model.
3. The feature extraction method according to claim 2, characterized in that, The step of obtaining the first baseline feature in the current incremental training batch includes: Obtain multiple second benchmark features from the previous incremental training batch, wherein the target cluster center is obtained by clustering multiple second benchmark features; The target cluster center is determined as the sampling center, and the distribution variance of the second reference feature with the target cluster center as the cluster center is determined, and the distribution variance is used as the sampling variance; Gaussian random sampling is performed based on the sampling center and the sampling variance to obtain the first baseline feature in the current incremental training batch.
4. The feature extraction method according to claim 3, characterized in that, The step of training the feature extraction model based on the self-supervised loss and the feature space reservation loss includes: Determine the clustering loss when clustering multiple second benchmark features; The target loss is determined based on the self-supervised loss, the feature space reservation loss, and the clustering loss, and the feature extraction model is trained based on the target loss.
5. The feature extraction method according to claim 4, characterized in that, The determination of the clustering loss when clustering multiple second benchmark features includes: Determine multiple original cluster centers, and determine a third distance between the second benchmark feature and each of the original cluster centers; Determine the target distance with the largest value from among the multiple third distances; A first quantity of the second benchmark features is determined, and based on the first quantity and the distance to the target, a clustering loss is determined when clustering multiple second benchmark features based on the original cluster centers.
6. The feature extraction method according to any one of claims 1 to 5, characterized in that, The step of training the feature extraction model based on the self-supervised loss and the feature space reservation loss includes: At the start of the current incremental training batch, a copy of the feature extraction model is created to obtain the copy model; Obtain an initialized encoding model, wherein the encoding model is used to extract features for the enhanced sample media data in the current incremental training batch; Using the copy model and the encoding model as the teacher network and the feature extraction model as the student network, the distillation loss of the feature extraction model is determined. The target loss is determined based on the self-supervised loss, the feature space reservation loss, and the distillation loss, and the feature extraction model is trained based on the target loss.
7. The feature extraction method according to claim 6, characterized in that, Determining the distillation loss of the feature extraction model includes: The sample media data was enhanced in different ways to obtain first and second enhanced data. The first enhancement feature of the first enhancement data is extracted based on the feature extraction model, the second enhancement feature of the first enhancement data is extracted based on the copy model, a third similarity between the first enhancement feature and the second enhancement feature is determined, and a first loss is determined based on the third similarity. Based on the coding model, a third enhancement feature is extracted from the second enhancement data, a fourth similarity is determined between the first enhancement feature and the third enhancement feature, and a second loss is determined based on the fourth similarity. The distillation loss of the feature extraction model is determined based on the first loss and the second loss.
8. The feature extraction method according to claim 7, characterized in that, Determining the third similarity between the first enhanced feature and the second enhanced feature includes: Get the initialized mapping model; Based on the mapping model, the first enhanced feature is mapped to the feature space where the second enhanced feature is located to obtain the mapped feature; The similarity between the mapped feature and the second enhanced feature is used as the third similarity between the first enhanced feature and the second enhanced feature.
9. A model training method, characterized in that, include: Obtain sample media data from the current incremental training batch, extract sample features from the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features; Multiple target cluster centers are identified. A segmentation plane for the feature space is determined based on the midpoint between the two farthest target cluster centers. The segmentation plane is adjusted according to a preset rotation angle step size until the difference in the number of target cluster centers on both sides of the segmentation plane is less than or equal to a preset number threshold. The target cluster centers on one side of the segmentation plane are designated as a first center set, and the target cluster centers on the other side of the segmentation plane are designated as a second center set. The distance between the sample media data and the nearest target cluster center in the first center set is taken as the first distance between the sample media data and the first center set. The distance between the sample media data and the nearest target cluster center in the second center set is taken as the second distance between the sample media data and the second center set. A second quantity of sample media data and the difference between the first and second distances are determined. A feature space reservation loss is determined based on the second quantity and the difference. The feature extraction model is trained based on the self-supervised loss and the feature space reservation loss.
10. A feature extraction device, characterized in that, include: The first processing module is used to acquire sample media data in the current incremental training batch, extract sample features of the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features. The second processing module is used to determine multiple target cluster centers, determine a segmentation surface of the feature space based on the midpoint between the two farthest target cluster centers, adjust the segmentation surface according to a preset rotation angle step size until the difference in the number of target cluster centers on both sides of the segmentation surface is less than or equal to a preset number threshold, take the target cluster centers on one side of the segmentation surface as a first center set, take the target cluster centers on the other side of the segmentation surface as a second center set, take the distance between the sample media data and the nearest target cluster center in the first center set as the first distance between the sample media data and the first center set, take the distance between the sample media data and the nearest target cluster center in the second center set as the second distance between the sample media data and the second center set, determine the second number of sample media data, and the difference between the first distance and the second distance, and determine the feature space reservation loss based on the second number and the difference. The first parameter adjustment module is used to train the feature extraction model based on the self-supervised loss and the feature space reservation loss; The third processing module is used to acquire target media data and extract target features from the target media data based on the trained feature extraction model.
11. A model training device, characterized in that, include: The fourth processing module is used to acquire sample media data in the current incremental training batch, extract sample features of the sample media data based on the feature extraction model, and determine the self-supervised loss of the feature extraction model based on the sample features. The fifth processing module is used to determine multiple target cluster centers, determine a segmentation surface of the feature space based on the midpoint between the two farthest target cluster centers, adjust the segmentation surface according to a preset rotation angle step size until the difference in the number of target cluster centers on both sides of the segmentation surface is less than or equal to a preset number threshold, take the target cluster centers on one side of the segmentation surface as a first center set, take the target cluster centers on the other side of the segmentation surface as a second center set, take the distance between the sample media data and the nearest target cluster center in the first center set as the first distance between the sample media data and the first center set, take the distance between the sample media data and the nearest target cluster center in the second center set as the second distance between the sample media data and the second center set, determine the second number of sample media data, and the difference between the first distance and the second distance, and determine the feature space reservation loss based on the second number and the difference. The second parameter adjustment module is used to train the feature extraction model based on the self-supervised loss and the feature space reservation loss.
12. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the feature extraction method according to any one of claims 1 to 8, or the model training method according to claim 9.
13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the feature extraction method according to any one of claims 1 to 8, or the model training method according to claim 9.
14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the feature extraction method according to any one of claims 1 to 8, or the model training method according to claim 9.
