Information classification method and device, electronic equipment and computer readable storage medium

Through multiple rounds of clustering processing and feature vector analysis, the problem of information category division under the number of unknown categories is solved, and accurate information classification and efficient open set recognition are achieved.

CN120354186APending Publication Date: 2025-07-22TENCENT TECHNOLOGY (SHENZHEN) CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410084864.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-01-19
Publication Date
2025-07-22

AI Technical Summary

Technical Problem

The prior art cannot effectively realize the classification of information in the case of unknown number of categories, resulting in low accuracy in open set recognition.

Method used

By inputting the preset sample set into the information processing model to be trained for multiple rounds of clustering, multiple information categories and pre-trained information processing models are obtained. The pre-trained model extracts the feature vectors of the information to be classified, and the information category is determined based on the distance between the feature center point of the feature vector and the information category.

Benefits of technology

The precise classification of information is achieved in the case of unknown number of categories, which improves the effect of open set recognition and improves the efficiency of information classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120354186A_ABST
    Figure CN120354186A_ABST
Patent Text Reader

Abstract

The invention provides an information classification method and device, electronic equipment and a computer readable storage medium, which are at least applied to the field of artificial intelligence or information classification, and the method comprises the following steps: inputting a preset sample set into a to-be-trained information processing model, and through the to-be-trained information processing model, obtaining a to-be-trained information processing model; performing multi-round clustering processing on the unlabeled sample information in the preset sample set to obtain a plurality of information categories and a pre-trained information processing model; performing feature extraction on to-be-classified information by using the pre-trained information processing model to obtain a feature vector of the to-be-classified information; determining the distance between the feature vector and the feature center point of each information category in the plurality of information categories; and determining the information category of the to-be-classified information from the plurality of information categories based on the distance. According to the information classification method and device, information classification can be achieved under the condition that the number of classes is unknown, and the accuracy of class number estimation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to an information classification method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] With the development of machine recognition technology, in open-set recognition when classifying data, it is not only necessary to obtain the conclusion that the data of unknown classes does not come from known classes, but also necessary to further classify the unknown classes.

[0003] However, common open-set recognition methods generally require fixing the number of unknown classes, which does not meet the application requirements in actual scenarios. Therefore, in the related art, it is impossible to achieve the classification of information in the case of an unknown number of classes, and the accuracy of open-set recognition is not high. Summary of the Invention

[0004] Embodiments of this application provide an information classification method, apparatus, electronic device, and computer-readable storage medium, which can achieve the classification of information in the case of an unknown number of classes and improve the accuracy of estimating the number of classes.

[0005] The technical solution of the embodiments of this application is implemented as follows:

[0006] Embodiments of this application provide an information classification method, the method includes: inputting a preset sample set into an information processing model to be trained, and through the information processing model to be trained, performing multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information classes and a pre-trained information processing model; using the pre-trained information processing model to extract features from the information to be classified to obtain a feature vector of the information to be classified; determining the distance between the feature vector and the feature center point of each information class in the multiple information classes; based on the distance, determining the information class of the information to be classified from the multiple information classes.

[0007] Embodiments of this application provide an information classification apparatus, including: a clustering processing module, configured to input a preset sample set into an information processing model to be trained, and through the information processing model to be trained, perform multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information classes and a pre-trained information processing model; a feature extraction module, configured to use the pre-trained information processing model to extract features from the information to be classified to obtain a feature vector of the information to be classified; a distance determination module, configured to determine the distance between the feature vector and the feature center point of each information class in the multiple information classes; a class determination module, configured to determine the information class of the information to be classified from the multiple information classes based on the distance.

[0008] In some embodiments, the preset sample set includes a labeled sample set and an unlabeled sample set. The labeled sample set includes multiple labeled sample information and the known category of each labeled sample information. The unlabeled sample set includes multiple unlabeled sample information. The multiple labeled sample information and the multiple unlabeled sample information constitute the multiple sample information in the preset sample set. Each round of clustering processing in the multiple rounds of clustering processing corresponds to a training process of the information processing model to be trained. The clustering processing module is further configured to determine the unknown category of each unlabeled sample information, input the preset sample set into the information processing model to be trained, and use the known category and the unknown category as sample labels to perform multiple trainings on the information processing model to be trained, so as to obtain the pre-trained information processing model. Based on the sample feature vectors extracted by the information processing model to be trained during each training, perform one round of clustering processing on the multiple sample information to obtain the multiple information categories.

[0009] In some embodiments, for the first training process, the clustering processing module is further configured to determine each unlabeled sample information as a category respectively to obtain the unknown category of each unlabeled sample information. For the (i + 1)-th training process, obtain the clustering category obtained by each unlabeled sample information during the i-th round of clustering processing, and determine the clustering category as the unknown category of the unlabeled sample information during the (i + 1)-th training process, where i is an integer greater than or equal to 1.

[0010] In some embodiments, during the (i + 1)-th training process, the clustering processing module is further configured to use the information processing model to be trained to perform feature extraction on all sample information to obtain the sample feature vector of each sample information. The information processing model to be trained is the information processing model obtained after the i-th training, where i is an integer greater than or equal to 1. Using the known category and the unknown category as sample labels, update the model parameters in the information processing model to be trained based on the sample labels and the sample feature vector of each sample information to obtain the information processing model after the (i + 1)-th training. Determine the pre-trained information processing model from the multiple information processing models to be trained in all training processes.

[0011] In some embodiments, the clustering processing module is further configured to determine the contrast loss of the information processing model to be trained based on the sample labels and the sample feature vector of each sample information, and use the contrast loss to update the model parameters in the information processing model to be trained to obtain the information processing model after the (i + 1)-th training.

[0012] In some embodiments, the clustering processing module is further configured to, for each of the labeled sample information, determine a labeled positive sample feature set and a labeled negative sample feature set of the labeled sample information based on the known category; the labeled positive sample feature set includes sample feature vectors of labeled sample information belonging to the same known category as the labeled sample information, and the labeled negative sample feature set includes sample feature vectors of labeled sample information belonging to different known categories from the labeled sample information; for each of the unlabeled sample information, determine an unlabeled positive sample feature set and an unlabeled negative sample feature set of the unlabeled sample information based on the unknown category; the unlabeled positive sample feature set includes sample feature vectors of unlabeled sample information belonging to the same unknown category as the unlabeled sample information, and the unlabeled negative sample feature set includes sample feature vectors of unlabeled sample information belonging to different unknown categories from the unlabeled sample information; based on the sample feature vectors of each of the sample information, the labeled positive sample feature set and the labeled negative sample feature set of each of the labeled sample information, and the unlabeled positive sample feature set and the unlabeled negative sample feature set of each of the unlabeled sample information, determine the contrastive loss of the information processing model to be trained.

[0013] In some embodiments, the information classification device further includes an augmentation module configured to perform data augmentation on each of the sample information to obtain augmented sample information for each of the sample information; use the information processing model to be trained to perform feature extraction on each of the augmented sample information to obtain augmented sample feature vectors for each of the augmented sample information; for each of the sample information, determine the self-supervised positive sample feature set of the sample information from the multiple augmented sample feature vectors corresponding to the sample information, and determine the sample feature vectors of other sample information in the preset sample set and the multiple augmented sample feature vectors corresponding to the other sample information as the self-supervised negative sample feature set of the sample information; the clustering processing module is further configured to determine the first contrastive loss of the information processing model to be trained based on the sample feature vectors of each of the sample information, the labeled positive sample feature set and the labeled negative sample feature set of each of the labeled sample information, and the unlabeled positive sample feature set and the unlabeled negative sample feature set of each of the unlabeled sample information; determine the second contrastive loss of the information processing model to be trained based on the sample feature vectors of each of the sample information and the self-supervised positive sample feature set and the self-supervised negative sample feature set of each of the sample information; sum the first contrastive loss and the second contrastive loss to obtain the contrastive loss of the information processing model to be trained.

[0014] In some embodiments, the clustering processing module is further configured to, in the (i + 1)-th round of clustering processing, re-cluster the multiple clustering categories after the i-th round of clustering processing based on the sample feature vectors of each sample information extracted by the information processing model to be trained, where the information processing model to be trained is the information processing model obtained after the i-th training; determine the clustering index of each sample information in the (i + 1)-th round of clustering processing based on the multiple clustering categories after the (i + 1)-th round of clustering processing and the sample feature vectors of each sample information; if each clustering category in the multiple clustering categories after the K-th round of clustering processing is a known category after K rounds of clustering processing, stop the clustering processing and determine the multiple information categories from the multiple clustering categories in all clustering processing based on the clustering index of each sample information in each round of clustering processing; K is an integer greater than or equal to 1, and i is an integer between 1 and K.

[0015] In some embodiments, the clustering processing module is further configured to determine a category to be merged from the multiple clustering categories after the i-th round of clustering processing, where the number of sample information in the category to be merged is the smallest among the multiple clustering categories after the i-th round of clustering processing; for each clustering category, determine the center point vector of the clustering category based on the sample feature vectors of each sample information in the clustering category; determine the cosine distance between the center point vector of the category to be merged and the center point vectors of each of the other clustering categories; determine the clustering category corresponding to the minimum cosine distance as the target clustering category, and merge the category to be merged into the target clustering category to obtain new clustering categories; where the multiple clustering categories other than the category to be merged among the multiple clustering categories after the i-th round of clustering processing and the new clustering categories together constitute the multiple clustering categories after the (i + 1)-th round of clustering processing.

[0016] In some embodiments, the clustering processing module is further used to determine similar sample information and heterogeneous sample information of each of the sample information based on multiple clustering categories after the i+1 round of clustering processing; for each of the sample information, determine the heterogeneous category with the shortest distance from the clustering category to which the sample information belongs; based on the sample feature vector of the sample information and the sample feature vectors of all similar sample information of the sample information, determine the similar average distance between the sample information and all similar sample information; based on the sample feature vector of the sample information and the sample feature vectors of all heterogeneous sample information of the sample information, determine the heterogeneous average distance between the sample information and all heterogeneous sample information in the heterogeneous category; based on the sample feature vector of the sample information and the sample feature vectors of all heterogeneous sample information of the sample information, determine the clustering index of the sample information in the i+1 clustering processing process.

[0017] In some embodiments, the unlabeled sample information includes at least one unlabeled known class sample information; the clustering processing module is also used to determine the average clustering index of the unlabeled known class sample information in each round of clustering processing based on the clustering index of each sample information in each round of clustering processing; and determine the multiple clustering categories after the clustering processing process corresponding to the maximum average clustering index as the multiple information categories.

[0018] In some embodiments, the information classification device also includes a category breaking up module, which is used to determine the category clustering index of each of the multiple clustering categories after the i+1th round of clustering processing based on the clustering index of each of the sample information in the i+1th round of clustering processing after obtaining multiple clustering categories after the i+1th round of clustering processing; break up the clustering category corresponding to the minimum category clustering index, and determine each sample information in the clustering category corresponding to the minimum category clustering index as a new sample category; cluster each sample category and each unbroken merged cluster category again to obtain multiple clustering categories after the i+2th round of clustering processing.

[0019] In some embodiments, the information classification device also includes an initialization module, which is used to determine each of the labeled sample information as a category when the information processing model to be trained is trained for the first time, so as to obtain an initialized known category for each of the labeled sample information; input the preset sample set into the information processing model to be trained, and use the known category, the unknown category and the initialized known category as sample labels to perform a first training on the information processing model to be trained, so as to obtain the information processing model trained for the first time.

[0020] An embodiment of the present application provides an electronic device, which includes: a memory for storing computer-executable instructions or computer programs; a processor for implementing the information classification method provided by the embodiment of the present application when executing the computer-executable instructions or computer programs stored in the memory.

[0021] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the information classification method provided by the embodiment of the present application when being executed by a processor.

[0022] An embodiment of the present application provides a computer program product including a computer program or computer-executable instructions, and when the computer program or computer-executable instructions are executed by a processor, the information classification method provided by the embodiment of the present application is implemented.

[0023] The embodiment of the present application has the following beneficial effects:

[0024] When classifying information, first input a preset sample set into an information processing model to be trained. Through the information processing model to be trained, perform multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model. That is, in the model training process of the embodiment of the present application, multiple rounds of clustering processing are performed to adjust the number of information categories to be classified, so as to achieve an accurate estimation of the number of information categories. After obtaining multiple information categories and a pre-trained information processing model, the feature vector of the information to be classified can be extracted through the pre-trained information processing model. Based on the distance between the feature vector and the feature center point of each information category among the multiple information categories, determine the information category of the information to be classified. Therefore, the embodiment of the present application can achieve the classification of information categories in the case of unknown category numbers, effectively improving the effect of open-set recognition. In addition, the embodiment of the present application does not need to train and cluster the information processing model based on new information to be classified, improving the information classification efficiency. Description of the Drawings

[0025] Figure 1 is a schematic structural diagram of the information classification system architecture provided by the embodiment of the present application;

[0026] Figure 2 is a schematic structural diagram of the electronic device provided by the embodiment of the present application;

[0027] Figure 3 is a schematic flowchart of the information classification method provided by the embodiment of the present application;

[0028] Figure 4 is a schematic flowchart of the information classification method provided by the embodiment of the present application;

[0029] Figure 5 It is an example structure diagram of an information processing model to be trained involved in the embodiments of the present application;

[0030] Figure 6 It is a schematic flowchart of an information classification method provided by the embodiments of the present application;

[0031] Figure 7 It is a framework structure diagram provided by the embodiments of the present application;

[0032] Figure 8 It is a sample clustering process diagram of a bottom-up open clustering algorithm involved in the embodiments of the present application;

[0033] Figure 9 It is a curve graph of the change of clustering indexes during the merging process in the embodiments of the present application;

[0034] Figure 10 It is a comparison graph of the category quantity prediction results between the embodiments of the present application and other methods;

[0035] Figure 11 It is a comparison graph of the category prediction accuracy rates between the embodiments of the present application and other methods. Detailed implementation manners

[0036] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be construed as limitations on the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0037] In the following description, "some embodiments" are involved, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0038] In the following description, the terms "first / second / third" involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0039] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the function of that module or unit.

[0040] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0041] Before further elaborating on the embodiments of the present application, the nouns and terms involved in the embodiments of the present application are described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0042] 1) Supervised learning: A machine learning method that uses labeled data to train a model, where the labeled data represents the correspondence between inputs and outputs.

[0043] 2) Unsupervised learning: A machine learning method that does not require pre-labeled data as a training set, but instead automatically learns the inherent structure and patterns of the data from unlabeled data.

[0044] 3) Semi-Supervised Learning (SSL): A machine learning method that combines supervised learning and unsupervised learning. Semi-Supervised Learning uses a large amount of unlabeled data as well as labeled data simultaneously.

[0045] 4) Open-set recognition: A special type of semi-supervised learning. In conventional semi-supervised learning, there is a portion of unlabeled data in the training set, but the labeled data covers all the categories in the training set. In open-set recognition, there are some categories for which there is no corresponding labeled data, that is, the categories to be recognized may never appear in the labeled training set.

[0046] 5) Close-set recognition: Generally refers to a supervised image classification task. In such a task, the categories of the data used for testing are all included in the labeled training set, and no categories outside the training set will appear.

[0047] In related technologies, machine learning methods that do not fix the number of unknown categories generally use labeled data to supervised train a feature extraction model. Then, the feature extraction model is used to extract the features of all samples in the labeled data, and the k-means clustering algorithm (K-Means) is used to cluster the features of all samples. Since K-Means requires the number of clusters to be preset in advance, the above method needs to traverse all possible numbers of categories and select the number of categories with the best clustering effect according to the clustering index. Although the above supervised learning method can roughly estimate the number of categories, since the feature extraction model is trained with supervised samples, all data including unlabeled data is not fully utilized, the ability of the feature extraction model is limited, and the extracted features are not optimal, which affects the estimation effect of the number of categories. And inaccurate estimation of the number of categories will affect the learning effect of the subsequent open-set recognition model. Therefore, the accuracy of estimating the number of categories in related technologies is low, and the open-set recognition effect is not good.

[0048] Based on the problems existing in related technologies, the embodiments of the present application provide an information classification method, which is an end-to-end open-set recognition algorithm for the case where the number of new categories is unknown. The method is to randomly select a small amount of information for labeling, input a preset sample set constructed by the labeled sample information and the unlabeled sample information into the information processing model to be trained, and obtain multiple information categories and a pre-trained information processing model through multiple rounds of clustering processing on the unlabeled sample information. Furthermore, in the inference process, the information to be classified is directly input into the pre-trained information processing model to obtain a feature vector, and the information category of the information to be classified can be determined based on the distance between the feature vector and the feature center point of each information category among the multiple information categories. Compared with the supervised learning method, in the model training process of the embodiments of the present application, the number of information categories to be classified is adjusted through multiple rounds of clustering processing, so as to achieve accurate estimation of the number of information categories, thereby improving the accuracy of classifying the information to be classified and effectively improving the open-set recognition effect.

[0049] Specifically, in the information classification method provided by the embodiments of the present application, first, the preset sample set is input into the information processing model to be trained, and through the information processing model to be trained, multiple rounds of clustering processing are performed on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model; then, the pre-trained information processing model is used to extract the features of the information to be classified to obtain the feature vector of the information to be classified; then, the distance between the feature vector and the feature center point of each information category among the multiple information categories is determined; finally, based on the distance, the information category of the information to be classified is determined from the multiple information categories.

[0050] Here, the exemplary application of the information classification device according to the embodiments of the present application will be described first. The information classification device is an electronic device for implementing the information classification method. In one implementation, the information classification device (i.e., the electronic device) provided by the embodiments of the present application can be implemented as a terminal or as a server. In one implementation, the information classification device provided by the embodiments of the present application can be implemented as a notebook computer, a tablet computer, a desktop computer, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device, an intelligent robot, an intelligent home appliance, an intelligent vehicle-mounted device, or any other terminal with data processing and information classification functions; in another implementation, the information classification device provided by the embodiments of the present application can also be implemented as a server, where the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, which are not limited in the embodiments of the present application. Next, the exemplary application when the information classification device is implemented as a server will be described.

[0051] See Figure 1 , Figure 1 is an optional architecture diagram of the information classification system provided by the embodiments of the present application. To accurately classify the information to be classified, an information classification application can be provided. The information classification system 100 provided by the embodiments of the present application at least includes a terminal 400, a network 300, and a server 200. The information classification application runs on the terminal 400, where the server 200 is the server of the information classification application. The server 200 can constitute the information classification device according to the embodiments of the present application, that is, the information classification method according to the embodiments of the present application is implemented through the server 200. The terminal 400 is connected to the server 200 through the network 300, and the network 300 can be a wide area network, a local area network, or a combination of the two.

[0052] When performing information classification, the user can perform an interaction operation through the terminal 400 on the client side of the information recommendation and classification application. After receiving the user's interaction operation, the client side can encapsulate the terminal identifier or the client identifier into the information classification request, and send the information classification request to the server 200 through the terminal 400. When receiving the information classification request, the server 200, in response to the information classification request, inputs a preset sample set into the information processing model to be trained, and through the information processing model to be trained, performs multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model; the server 200 uses the pre-trained information processing model to extract features from the information to be classified to obtain the feature vector of the information to be classified; finally, the server 200 determines the distance between the feature vector and the feature center point of each information category in the multiple information categories; based on the distance, determines the information category of the information to be classified from the multiple information categories. After determining the information category of the information to be classified, the server 200 can send the information category of the information to be classified to the terminal 400 for the user to view.

[0053] In some embodiments, the information classification method of the embodiments of the present application can also be executed by the terminal 400 itself. That is to say, when the terminal 400 receives the interaction operation input by the user through the client side, the terminal 400 inputs the preset sample set into the information processing model to be trained, and through the information processing model to be trained, performs multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model; the terminal 400 uses the pre-trained information processing model to extract features from the information to be classified to obtain the feature vector of the information to be classified; finally, the terminal 400 determines the distance between the feature vector and the feature center point of each information category in the multiple information categories; based on the distance, determines the information category of the information to be classified from the multiple information categories. After determining the information category of the information to be classified, the information category of the information to be classified is displayed on the client interface of the terminal 400.

[0054] The information classification method provided by the embodiments of the present application can also be implemented based on a cloud platform and through cloud technology. For example, the above-mentioned server 200 can be a cloud server. The cloud server inputs the preset sample set into the information processing model to be trained, and through the information processing model to be trained, performs multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model; or, the cloud server uses the pre-trained information processing model to extract features from the information to be classified to obtain the feature vector of the information to be classified; or, the cloud server determines the distance between the feature vector and the feature center point of each information category in the multiple information categories; or, the cloud server based on the distance, determines the information category of the information to be classified from the multiple information categories.

[0055] In some embodiments, a cloud storage may also be provided. The model parameters of the pre-trained information processing model can be stored in the cloud storage, and multiple information categories, etc. can also be stored in the cloud storage. In this way, when an information classification request is received, the model parameters of the pre-trained information processing model can be directly obtained from the cloud storage to implement feature extraction of the information to be classified, thereby improving the information classification efficiency and saving training resources.

[0056] It should be noted here that cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data computing, storage, processing, and sharing. Cloud technology is the general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, be used on demand, and is flexible and convenient. Cloud computing technology will become an important support. The back-end services of technical network systems require a large amount of computing and storage resources, such as video websites, picture-based websites, and more portal websites. With the highly developed application of the Internet industry, in the future, each item may have its own identification mark and needs to be transmitted to the back-end system for logical processing. Data at different levels will be processed separately, and various industry data requires a powerful system back-end support, which can be achieved through cloud computing.

[0057] See Figure 2 , Figure 2 is a schematic structural diagram of the electronic device provided by the embodiment of the present application. Figure 2 The illustrated electronic device includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. Each component in the terminal 400 is coupled together through a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 2 all kinds of buses are labeled as the bus system 440.

[0058] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0059] The user interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, and other input buttons and controls.

[0060] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, etc. The memory 450 optionally includes one or more storage devices that are physically located remote from the processor 410.

[0061] The memory 450 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0062] In some embodiments, the memory 450 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are described below by way of example.

[0063] The operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks;

[0064] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wi-Fi (Wireless Fidelity), and Universal Serial Bus (USB), etc.;

[0065] The presentation module 453 is used to enable the presentation of information (such as a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with the user interface 430 (such as a display screen, speaker, etc.);

[0066] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one of the one or more input devices 432.

[0067] In some embodiments, the device provided by the embodiments of the present application may be implemented in software. Figure 2 The information classification device 455 stored in the memory 450 is shown, which may be software in the form of a program and a plug-in, etc., including the following software modules: a clustering processing module 4551, a feature extraction module 4552, a distance determination module 4553, and a category determination module 4554. These modules are logical, so they can be combined arbitrarily or further split according to the implemented functions. The functions of each module will be described below.

[0068] In other embodiments, the device provided by the embodiments of the present application may be implemented in hardware. As an example, the device provided by the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the information classification method provided by the embodiments of the present application. For example, a processor in the form of a hardware decoding processor may employ one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0069] The information classification method provided by each embodiment of the present application may be executed by an electronic device. Among them, the electronic device may be a server or a terminal, that is, the information classification method provided by each embodiment of the present application may be executed by a server, may be executed by a terminal, or may also be executed through the interaction between a server and a terminal.

[0070] Figure 3 is an optional flowchart of the information classification method provided by the embodiments of the present application. The following will be described in conjunction with Figure 3 the steps shown, as Figure 3 shown, taking the execution subject of the information classification method as a server as an example for description. The method includes the following steps S101 to step S104:

[0071] Step S101, input a preset sample set into the information processing model to be trained, and through the information processing model to be trained, perform multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model.

[0072] Here, the preset sample set includes a labeled sample set and an unlabeled sample set. The labeled sample set includes multiple labeled sample information and the known categories of each labeled sample information. The unlabeled sample set includes multiple unlabeled sample information. The multiple labeled sample information and the multiple unlabeled sample information constitute the multiple sample information in the preset sample set. The sample information in the preset sample set can have the same sample distribution as the information to be classified. Multiple sample information can be stored in advance, and a small part of the sample information is randomly selected from the multiple sample information for labeling of known categories to obtain multiple labeled sample information. The remaining sample information in the multiple sample information is used as the multiple unlabeled sample information to form the preset sample set. It should be noted that the categories of the unlabeled sample information in the unlabeled sample set are uncertain, and the number of categories is also uncertain. The unlabeled sample set includes at least some unlabeled sample information belonging to known categories. Exemplarily, the sample information can be a transmissible resource such as an image, audio, text, etc. Taking the sample information as an image as an example, the known categories can be categories for distinguishing images such as people, animals, landscapes, animations, etc.

[0073] The information processing model to be trained can be any feature extraction network model without a classification layer. Exemplarily, in the image classification scenario, the information processing model to be trained can select a common image classification network model. However, the image classification network model finally has a classification layer in its structure to map the image features to fixed categories. In order to eliminate the fixation of categories, the classification layer of the image classification network model can be removed to obtain the information processing model to be trained.

[0074] Performing multiple rounds of clustering processing on the unlabeled sample information in the preset sample set through the information processing model to be trained refers to the process of clustering and merging the unknown categories of the unlabeled sample information and training the information processing model based on the obtained clustering results. During one training process of the information processing model to be trained, the sample feature vectors of each sample information can be first obtained by using the feature extraction of the information processing model to be trained. Then, based on the sample feature vectors of each sample information, the labeled sample information and the known categories of the labeled sample information, the unlabeled sample information and the unknown categories of the unlabeled sample information, the information processing model is trained. Among them, the unknown categories of the unlabeled sample information are the results of the clustering processing of the unlabeled sample information in the previous round. During the clustering processing, category merging can be performed based on the sample feature vectors of each sample information and the clustering results obtained after the previous round of clustering processing to obtain the clustering categories obtained after the current round of clustering processing. During each round of clustering processing, the clustering index can also be calculated based on the sample feature vectors of each sample information and the clustering categories obtained after the clustering processing. Exemplarily, the clustering index can adopt the silhouette coefficient.

[0075] When a preset stop condition is satisfied, the above-mentioned clustering and multiple iterative processes of model training are stopped, and multiple information categories and a pre-trained information processing model can be determined from the results of each round of iteration based on preset metric conditions. Exemplarily, the preset stop condition can be that each clustering category among the multiple clustering categories after clustering processing is a known category. The preset metric condition is that the clustering metric reaches the maximum value.

[0076] Step S102: Use the pre-trained information processing model to extract features from the information to be classified, and obtain the feature vector of the information to be classified.

[0077] Here, the pre-trained information processing model is also a network model without a classification layer. After inputting the information to be classified into the pre-trained information processing model, the pre-trained information processing model can extract features from the information to be classified and output the feature vector of the information to be classified.

[0078] Step S103: Determine the distances between the feature vector and the feature center points of each information category among the multiple information categories.

[0079] Here, the feature center point of an information category is the average vector of the sample feature vectors of each sample information in that information category. The cosine distance between the feature vector of the information to be classified and the feature center point of each information category can be calculated.

[0080] Step S104: Based on the distances, determine the information category of the information to be classified from the multiple information categories.

[0081] Here, after obtaining the cosine distances between the feature vector of the information to be classified and the feature center points of each information category, the minimum value can be determined from all the cosine distances, and the information category corresponding to the minimum value is determined as the information category of the information to be classified.

[0082] The information classification method provided by the embodiments of the present application, when classifying information, can first input a preset sample set into the information processing model to be trained. Through the information processing model to be trained, multi-round clustering processing is performed on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model. That is, in the model training process of the embodiments of the present application, the number of information categories to be classified is adjusted through multi-round clustering processing, so as to achieve an accurate estimation of the number of information categories. After obtaining multiple information categories and a pre-trained information processing model, the feature vector of the information to be classified can be extracted through the pre-trained information processing model, and based on the distance between the feature vector and the feature center point of each information category in the multiple information categories, the information category of the information to be classified is determined. Therefore, the embodiments of the present application can achieve the classification of information categories in the case of unknown category numbers, effectively improving the effect of open-set recognition. In addition, the embodiments of the present application do not need to train and cluster the information processing model based on new information to be classified, improving the information classification efficiency.

[0083] Figure 4 is another optional process schematic diagram of the information classification method provided by the embodiments of the present application, as Figure 4 shown, the method includes the following steps S201 to step S210:

[0084] Step S201, the terminal receives the interaction operation of the user.

[0085] Here, the interaction operation can be a click confirmation operation, an information input operation to be classified, etc. The user can perform any one of the interaction operations on the client of the information classification application through the terminal.

[0086] Step S202, the terminal generates an information classification request in response to the interaction operation.

[0087] When the terminal receives the interaction operation of the user, it is determined that the information classification of a certain piece of information to be classified is currently required. Therefore, in response to the interaction operation, an information classification request is generated. The information recommendation request is used to request the server to first determine the number of unknown information categories, train multiple information categories and a pre-trained information processing model based on the preset sample set, and then determine the information category of the information to be classified from the multiple information categories based on the pre-trained information processing model.

[0088] Step S203, the terminal sends the information classification request to the server.

[0089] Step S204, the server obtains the preset sample set in response to the information classification request.

[0090] Here, the preset sample set includes the labeled sample set D l and the unlabeled sample set D u。The labeled sample set includes multiple labeled sample information and the known categories of each labeled sample information. The unlabeled sample set includes multiple unlabeled sample information. The multiple labeled sample information and the multiple unlabeled sample information constitute the multiple sample information in the preset sample set. The sample information in the preset sample set may have the same sample distribution as the information to be classified. Multiple sample information can be pre-stored, and a small part of the sample information is randomly selected from the multiple sample information for labeling of known categories to obtain multiple labeled sample information. The remaining sample information in the multiple sample information is used as the multiple unlabeled sample information to form the preset sample set. It should be noted that the categories of the unlabeled sample information in the unlabeled sample set are uncertain, and the number of categories is also uncertain. The unlabeled sample set includes at least some unlabeled sample information belonging to known categories. Exemplarily, the sample information can be a transmissible resource such as an image, audio, text, etc. Taking the sample information as an image as an example, the known categories can be categories for differentiating images such as people, animals, landscapes, animations, etc.

[0091] Exemplarily, D l ={(x j , y j ), j = 1, …, N}, that is, the number of labeled sample information in the labeled sample set is N. These N labeled sample information all come from T known categories. Both N and T are integers greater than or equal to 1. D u ={(x j , y j ), j = 1, …, M}, that is, the number of unlabeled sample information in the unlabeled sample set is M, and M is an integer greater than or equal to 1. The unlabeled sample information may come from known categories or unknown categories. The unlabeled sample set includes at least some unlabeled sample information belonging to known categories, denoted by D u / D l .

[0092] Step S205, the server inputs the preset sample set into the information processing model to be trained, and through the information processing model to be trained, performs multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information categories and the pre-trained information processing model.

[0093] Here, the information processing model to be trained can be any feature extraction network model with the classification layer removed. In the image classification scenario, the information processing model to be trained can be selected from common image classification network models. However, the image classification network model finally has a classification layer in its structure to map the image features to fixed categories. In order to eliminate the fixation on categories, the classification layer of the image classification network model can be removed to obtain the information processing model to be trained.

[0094] Performing multiple rounds of clustering processing on the unlabeled sample information in the preset sample set through the information processing model to be trained refers to the process of clustering and merging the unknown categories of the unlabeled sample information, and then training the information processing model based on the obtained clustering results. When the preset stop condition is met, the above-mentioned multiple iterative processes of clustering and model training are stopped, and multiple information categories and the pre-trained information processing model can be determined from the results of each round of iteration based on the preset metric conditions.

[0095] Figure 5 FIG. is an exemplary structural diagram of the information processing model to be trained. Exemplarily, refer to Figure 5 , the information processing model to be trained can be a plain deep neural network (such as a 34-layer plain network), a convolutional neural network (VGG, Visual Geometry Group Network, such as VGG-19), a deep residual network (ResNet, Residual network, such as a 34-layer residual network), etc. Figure 5 In which, conv is the convolutional layer and fc is the fully-connected layer. Figure 5 The fully-connected layer in the dashed box part 501 in FIG. is the classification layer. Removing this fully-connected layer and only retaining the network before the last pooling layer, the information processing model to be trained is obtained.

[0096] In some embodiments, each round of clustering processing in the multiple rounds of clustering processing corresponds to one training process of the information processing model to be trained. Refer to Figure 6 , Figure 6 FIG. shows that the multiple rounds of clustering processing on the unlabeled sample information in the preset sample set through the information processing model to be trained in step S205 can be implemented through the following steps S2051 to S2053:

[0097] Step S2051, determining the unknown category of each unlabeled sample information.

[0098] Here, each round of clustering processing corresponds to one training process of the information processing model to be trained. That is to say, in one training process of the information processing model to be trained, the information processing model can be trained based on the labeled sample information and the known categories of the labeled sample information, the unlabeled sample information and the unknown categories of the unlabeled sample information. Among them, the unknown category of the unlabeled sample information is the result of the previous round of clustering processing of the unlabeled sample information.

[0099] In the embodiments of the present application, determining the unknown category of each unlabeled sample information can be achieved through the following methods: For the first training process, each unlabeled sample information is determined as a category to obtain the unknown category of each unlabeled sample information; for the (i + 1)-th training process, obtain the clustering category obtained by each unlabeled sample information during the i-th clustering process, and determine the clustering category as the unknown category of the unlabeled sample information during the (i + 1)-th training process; i is an integer greater than or equal to 1.

[0100] Here, for the first training process, an initial information processing model can be obtained first, and the initial information processing model is used to extract features from all labeled sample information and all unlabeled sample information to obtain the feature vector of each labeled sample information and the sample feature vector of each unlabeled sample information. Then, based on the sample feature vector of each labeled sample information, the known category of each labeled sample information, the sample feature vector of each unlabeled sample information, and the unknown category of each unlabeled sample information, the initial information processing model is trained to obtain the information processing model to be trained during the second training process. During the above first training process, each unlabeled sample information in the unlabeled sample set can be initialized as an unknown category, and the initial information processing model is trained.

[0101] Exemplarily, if there are 3 unlabeled sample information in the preset sample set, namely unlabeled sample information 1, unlabeled sample information 2, and unlabeled sample information 3, then each unlabeled sample information can be determined as a category to obtain the corresponding unknown category 1 of unlabeled sample information 1, the corresponding unknown category 2 of unlabeled sample information 2, and the corresponding unknown category 3 of unlabeled sample information 3.

[0102] After obtaining the feature vector of each sample information, the first round of clustering process can be started. For the first clustering process, clustering can be performed on the known categories in the preset sample set and the unknown categories of each unlabeled sample information to obtain multiple clustering categories. If the number of known categories in the preset sample set is T and the number of unlabeled sample information is M, then during the first clustering process, the above M + T categories are merged to obtain multiple clustering categories.

[0103] Taking i = 1 as an example, during the second training process, multiple clustering categories obtained during the first clustering process can be obtained, and the clustering category corresponding to each unlabeled sample information can be obtained, and the clustering category corresponding to each unlabeled sample information is determined as the unknown category of the unlabeled sample information during the second training process.

[0104] In some embodiments, when performing the first training on the information processing model to be trained, first, each labeled sample information is determined as a category to obtain the initialized known category of each labeled sample information; then, the preset sample set is input into the information processing model to be trained, and the information processing model to be trained is first trained with the known category, unknown category, and initialized known category as sample labels to obtain the information processing model after the first training.

[0105] Here, since each round of clustering processing corresponds to a training process of the information processing model to be trained, in the first round of clustering processing, clustering processing can be performed on the known categories in the preset sample set, as well as the unknown categories of each unlabeled sample information and the initialized known categories of each labeled sample information to obtain multiple clustering categories. It should be noted that, in order to prevent the clustering result of the labeled sample information with known categories from conflicting with its true known category, the embodiments of the present application have a mandatory condition: that is, the category where the labeled sample information is located can only be clustered into the correct category, that is to say, the category containing the labeled sample information can only be clustered into other categories that have and only have labeled sample information of the same category as the labeled sample information.

[0106] Exemplarily, if the number of known categories in the preset sample set is T, the number of unlabeled sample information is M, and the number of labeled sample information is N, then in the first clustering processing, the above M + N + T categories are merged to obtain multiple clustering categories.

[0107] By determining each labeled sample information as a category to obtain the initialized known category of each labeled sample information, and using the initialized known category together with the known category and unknown category as the categories for clustering in the first clustering processing, the embodiments of the present application can prevent misclassification of unlabeled sample information with known categories, improve the accuracy of clustering, and further improve the performance of the pre-trained information processing model, thereby improving the accuracy of information classification.

[0108] Step S2052: Input the preset sample set into the information processing model to be trained, and use the known category and unknown category as sample labels to perform multiple trainings on the information processing model to be trained to obtain the pre-trained information processing model.

[0109] In the embodiments of the present application, taking known categories and unknown categories as sample labels, multiple trainings of the information processing model to be trained can be achieved in the following manner: First, in the (i + 1)-th training process, use the information processing model to be trained to extract features from all sample information to obtain the sample feature vector of each sample information; the information processing model to be trained is the information processing model obtained after the i-th training, and i is an integer greater than or equal to 1; then, taking known categories and unknown categories as sample labels, based on the sample labels and the sample feature vectors of each sample information, update the model parameters in the information processing model to be trained to obtain the information processing model after the (i + 1)-th training; finally, determine the pre-trained information processing model from multiple information processing models to be trained in all training processes.

[0110] Taking i = 1 as an example, in the second training process, the information processing model to be trained is the information processing model obtained after the first training. First, the information processing model to be trained can be used to extract features from all sample information to obtain the sample feature vector of each sample information. It should be noted that the embodiments of the present application do not limit the dimension of the sample feature vector, which can be determined by the model structure of the information processing model to be trained. As can be seen from the above embodiments, the unknown categories in the second training process are the clustering categories corresponding to each unlabeled sample information obtained in the first clustering process. Therefore, all known categories and the clustering categories corresponding to each unlabeled sample information obtained in the first clustering process can be used as sample labels. Then, based on the sample labels and the sample feature vectors of each sample information, the contrast loss of the information processing model to be trained can be calculated. Based on the contrast loss, the model parameters in the information processing model to be trained are updated to obtain the information processing model after the second training.

[0111] When the preset stop condition is satisfied, stop the multiple iteration processes of the above model training. The pre-trained information processing model can be determined from multiple information processing models to be trained obtained in all training processes based on the preset metric conditions.

[0112] In some embodiments, updating the model parameters in the information processing model to be trained based on the sample labels and the sample feature vectors of each sample information can be achieved in the following manner: First, determine the contrast loss of the information processing model to be trained based on the sample labels and the sample feature vectors of each sample information; then, use the contrast loss to update the model parameters in the information processing model to be trained to obtain the information processing model after the (i + 1)-th training.

[0113] Here, still taking i = 1 as an example, during the second training process, after obtaining the sample labels and the sample feature vectors of each sample information, the positive sample feature set and the negative sample feature set of the sample feature vectors of each sample information can be constructed based on the sample labels. Then, based on the sample feature vectors, the positive sample feature set, and the negative sample feature set of each sample information, the contrastive loss of the information processing model to be trained is determined. After obtaining the contrastive loss, the model parameters in the information processing model can be updated by using a preset gradient descent method for parameter update, and the information processing model after the second training is obtained. It should be noted that the embodiments of the present application do not specifically limit the calculation method of the contrastive loss and the preset gradient descent method. Exemplarily, the preset gradient descent method can use the gradient descent method based on the Adam optimization algorithm (Adaptive momentum, a stochastic optimization method with adaptive momentum). The contrastive loss can satisfy the following formula (1-1).

[0114]

[0115] Among them, L(x; P(x), N(x)) represents the contrastive loss, z represents the feature vector corresponding to the sample information x, and z T represents the transpose of z. z + ∈P(x) represents the positive sample feature set, while z - ∈N(x) represents the negative sample feature set, and τ is an adjustable parameter. τ can be adjusted according to different learning tasks, and the user can set it by experience. |P(x)| is the number of feature vectors in the positive sample feature set.

[0116] In some embodiments, the contrastive loss of the information processing model to be trained can be determined based on the sample tags and the sample feature vectors of each sample information in the following manner: For each labeled sample information, based on the known category, determine the labeled positive sample feature set and the labeled negative sample feature set of the labeled sample information; the labeled positive sample feature set includes the sample feature vectors of the labeled sample information that belong to the same known category as the labeled sample information, and the labeled negative sample feature set includes the sample feature vectors of the labeled sample information that belong to different known categories from the labeled sample information; for each unlabeled sample information, based on the unknown category, determine the unlabeled positive sample feature set and the unlabeled negative sample feature set of the unlabeled sample information; the unlabeled positive sample feature set includes the sample feature vectors of the unlabeled sample information that belong to the same unknown category as the unlabeled sample information, and the unlabeled negative sample feature set includes the sample feature vectors of the unlabeled sample information that belong to different unknown categories from the unlabeled sample information; based on the sample feature vectors of each sample information, the labeled positive sample feature set and the labeled negative sample feature set of each labeled sample information, and the unlabeled positive sample feature set and the unlabeled negative sample feature set of each unlabeled sample information, determine the contrastive loss of the information processing model to be trained.

[0117] For example, if a sample information x is labeled sample information and its known category is y. First, based on the known category y, multiple labeled sample information that belong to the same known category y as the sample information x can be obtained from the labeled sample information set. Then, the sample feature vectors of the above multiple labeled sample information form the labeled positive sample feature set z + ∈P(x). Based on the known category y, multiple labeled sample information that belong to different known categories from the sample information x can be obtained from the labeled sample information set. The sample feature vectors of the above multiple labeled sample information form the labeled negative sample feature set z - ∈N(x).

[0118] If a sample information x is unlabeled sample information, the unknown category y of the sample information x can be obtained based on the multiple clustering categories obtained from the previous round of clustering processing. Based on the unknown category y, multiple unlabeled sample information that belong to the same unknown category y as the sample information x can be obtained from the unlabeled sample information set. Then, the sample feature vectors of the above multiple unlabeled sample information form the unlabeled positive sample feature set z + ∈P(x). Based on the unknown category y, multiple unlabeled sample information that belong to different unknown categories from the sample information x can be obtained from the unlabeled sample information set. The sample feature vectors of the above multiple unlabeled sample information form the unlabeled negative sample feature set z - ∈N(x).

[0119] It should be noted that during the construction of the above-mentioned labeled positive sample feature set, labeled negative sample feature set, unlabeled positive sample feature set, and unlabeled negative sample feature set, data augmentation can be performed on the sample information. Data augmentation does not change the category of the sample information. Therefore, the feature vectors after data augmentation can also be added to the corresponding feature sets, which can improve the generalization of the information processing model.

[0120] During the calculation of the contrast loss, taking the sample information as the labeled sample information as an example, the contrast loss corresponding to the sample information can be determined based on the sample feature vector of the sample information, the transpose of the sample feature vector of the sample information, all the sample feature vectors in the labeled positive sample feature set of the sample information, all the sample feature vectors in the labeled negative sample feature set, and the number of sample feature vectors in the labeled positive sample feature set. If the number of sample information used in one training process is 10, then the contrast losses corresponding to these 10 sample information are added up to obtain the contrast loss of the information processing model to be trained during this training process.

[0121] In the embodiments of the present application, positive and negative sample feature sets are constructed for the labeled sample information using a supervised method, and positive and negative sample feature sets are constructed for the unlabeled sample information using an unsupervised method. This enriches the calculation parameters of the contrast loss, improves the accuracy of training the information processing model, and further improves the accuracy of information classification.

[0122] In some embodiments, after obtaining the preset sample set, the server can augment the preset sample set in advance. First, perform data augmentation on each sample information to obtain the augmented sample information of each sample information; use the information processing model to be trained to extract features from each augmented sample information to obtain the augmented sample feature vector of each augmented sample information; for each sample information, determine the self-supervised positive sample feature set of the sample information from the multiple augmented sample feature vectors corresponding to the sample information, and determine the self-supervised negative sample feature set of the sample information from the sample feature vectors of other sample information in the preset sample set and the multiple augmented sample feature vectors corresponding to other sample information. At this time, based on the sample labels and the sample feature vectors of each sample information, the contrastive loss of the information processing model to be trained can be achieved in the following manner: determine the first contrastive loss of the information processing model to be trained based on the sample feature vectors of each sample information, the labeled positive sample feature set and the labeled negative sample feature set of each labeled sample information, and the unlabeled positive sample feature set and the unlabeled negative sample feature set of each unlabeled sample information; determine the second contrastive loss of the information processing model to be trained based on the sample feature vectors of each sample information and the self-supervised positive sample feature set and the self-supervised negative sample feature set of each sample information; sum the first contrastive loss and the second contrastive loss to obtain the contrastive loss of the information processing model to be trained.

[0123] In the embodiment of the present application, the contrast loss is the sum of the first contrast loss and the second contrast loss. The calculation method of the first contrast loss can refer to the calculation methods of the labeled sample information and the unlabeled sample information in the above embodiments. The second contrast loss can be a self-supervised loss. For the self-supervised loss, the positive sample feature set P(x) of the sample information is the feature set extracted from the sample information itself after various data augmentations, and the negative sample feature set N(x) is the feature set obtained from other sample information after data augmentation. Therefore, after obtaining the preset sample set, data augmentation can be performed on each sample information in the preset sample set to obtain the augmented sample information of each sample information. It should be noted that the embodiment of the present application does not limit the data augmentation method, as long as the category of the sample information does not change after data augmentation. The information processing model to be trained is used to extract feature vectors from each augmented sample information to obtain the augmented sample feature vectors of each augmented sample information. Taking the sample information x as an example, the multiple augmented sample feature vectors corresponding to the sample information x itself can be determined as the self-supervised positive sample feature set of the sample information. Then, the sample feature vectors of all other sample information in the preset sample set and the augmented sample feature vectors of all other sample information are determined as the self-supervised negative sample feature set of the sample information x. After obtaining the self-supervised positive sample feature set and the self-supervised negative sample feature set, the method for calculating the second contrast loss based on the self-supervised positive sample feature set and the self-supervised negative sample feature set can refer to the calculation method of the first contrast loss.

[0124] The embodiment of the present application additionally calculates the first contrast loss, which can improve the accuracy of training the information processing model, and further improve the accuracy of information classification.

[0125] Step S2053: Based on the sample feature vectors extracted by the information processing model to be trained during each training, perform a round of clustering processing on multiple sample information to obtain multiple information categories.

[0126] In the embodiments of the present application, based on the sample feature vectors extracted by the information processing model to be trained each time during training, a round of clustering processing for multiple sample information can be achieved in the following manner: First, in the (i + 1)-th round of clustering processing, based on the sample feature vectors of each sample information extracted by the information processing model to be trained, re-clustering is performed on the multiple clustering categories after the i-th round of clustering processing to obtain multiple clustering categories after the (i + 1)-th round of clustering processing; the information processing model to be trained is the information processing model obtained after the i-th training; then, based on the multiple clustering categories after the (i + 1)-th round of clustering processing and the sample feature vectors of each sample information, the clustering index of each sample information in the (i + 1)-th round of clustering processing is determined; finally, after K rounds of clustering processing, if each clustering category in the multiple clustering categories after the K-th round of clustering processing is a known category, the clustering processing process is stopped, and based on the clustering indexes of each sample information in each round of clustering processing, multiple information categories are determined from the multiple clustering categories in all clustering processing processes; K is an integer greater than or equal to 1, and i is an integer between 1 and K.

[0127] For example, the method for clustering multiple sample information can adopt a bottom-up agglomerative clustering algorithm. Taking i = 1 as an example, in the second round of clustering processing, based on the sample feature vectors of each sample information extracted by the information processing model to be trained, re-clustering is performed on the multiple clustering categories after the first round of clustering processing to obtain multiple clustering categories after the second round of clustering processing. Then, based on the multiple clustering categories after the second round of clustering processing and the sample feature vectors of each sample information, the clustering index of each sample information is determined. Here, the clustering index can be the Silhouette index. For a sample information x, the similar sample information of the sample information x can be obtained based on multiple clustering categories, and based on the sample feature vector of the sample information x and the sample feature vectors of each similar sample information, the average distance from the sample information x to other similar sample information is determined. Based on multiple clustering categories, the clustering category closest to the clustering category where the sample information x is located can be obtained, and the average distance from the sample information x to the sample information in this closest clustering category is calculated. Finally, based on the average distance from the sample information x to other similar sample information and the average distance from the sample information x to the sample information in this closest clustering category, the Silhouette index of the sample information x is determined.

[0128] In the embodiments of the present application, the preset stop condition is that each of the multiple clustering categories after clustering processing is a known category. The preset index condition is the maximum clustering index. That is, after K rounds of clustering processing, if each of the multiple clustering categories after the Kth round of clustering processing is a known category, the clustering processing process and the iterative training process are stopped. The maximum clustering index can be selected from the clustering indexes obtained in the entire clustering processing process, and the multiple clustering categories corresponding to the maximum clustering index are determined as multiple information categories. The information processing model to be trained in the training process where the maximum clustering index is located is used as the pre-trained information processing model for information classification in the subsequent model inference process.

[0129] In some embodiments, re-clustering the multiple clustering categories after the ith round of clustering processing based on the sample feature vectors of each sample information extracted by the information processing model to be trained can be implemented in the following manner: Determine the category to be merged from the multiple clustering categories after the ith round of clustering processing, and the number of sample information in the category to be merged is the smallest among the multiple clustering categories after the ith round of clustering processing; for each clustering category, determine the center point vector of the clustering category based on the sample feature vectors of each sample information in the clustering category; based on the center point vector of the category to be merged and the center point vectors of each of the other clustering categories, determine the cosine distance between the category to be merged and each of the other clustering categories; determine the target clustering category as the clustering category corresponding to the minimum cosine distance, and merge the category to be merged into the target clustering category to obtain new clustering categories; where, the other clustering categories except the category to be merged among the multiple clustering categories after the ith round of clustering processing, and the new clustering categories together constitute the multiple clustering categories after the (i + 1)th round of clustering processing.

[0130] For example, in the second round of clustering processing, the multiple clustering categories after the first round of clustering processing and the number of sample information included in each clustering category can be obtained first. The clustering category with the smallest number of sample information is determined as the category to be merged. The number of categories to be merged may be one or multiple. For one category to be merged, the clustering category with the minimum cosine distance from the category to be merged is determined as the target clustering category, and the category to be merged is merged into the target clustering category. It should be noted that in the embodiments of the present application, known categories will not be used as categories to be merged, but only as target clustering categories. Here, the cosine distance can be calculated through the center point vectors of two clustering categories. The center point vector of a clustering category is the average value of the sample feature vectors of all sample information in the clustering category. The cosine distance can satisfy the following formula (1-2). The center point vector can satisfy the following formula (1-3).

[0131]

[0132]

[0133] Among them, D(μ i , μ j ) represents the cosine distance between the i-th clustering category and the j-th clustering category, μ i represents the center point vector of the i-th clustering category, that is, the average value of all sample feature vectors in this clustering category, μ j represents the center point vector of the j-th clustering category, represents the transpose of the center point vector of the i-th clustering category, and ‖·‖ represents the L2 norm. μ represents the center point vector, z i is the i-th sample feature vector in the clustering category, and n is the total number of sample feature vectors in this clustering category, that is, the total number of sample information.

[0134] In some embodiments, based on the multiple clustering categories after the (i + 1)-th round of clustering processing and the sample feature vectors of each sample information, determining the clustering index of each sample information in the (i + 1)-th round of clustering processing can be achieved through the following methods: Based on the multiple clustering categories after the (i + 1)-th round of clustering processing, determine the same-class sample information and different-class sample information of each sample information; for each sample information, determine the different-class category with the smallest distance from the clustering category to which the sample information belongs; based on the sample feature vector of the sample information and the sample feature vectors of all the same-class sample information of the sample information, determine the within-class average distance between the sample information and all the same-class sample information; based on the sample feature vector of the sample information and the sample feature vectors of all the different-class sample information of the sample information, determine the between-class average distance between the sample information and all the different-class sample information in the different-class category; based on the within-class average distance and the between-class average distance, determine the clustering index of the sample information in the (i + 1)-th clustering processing.

[0135] For example, in the second-round clustering process, after determining multiple clustering categories after the second-round clustering process, for a sample information x, the clustering category to which the sample information x belongs can be obtained, and then all other sample information in the clustering category to which the sample information x belongs is used as the same-class sample information of the sample information x, and all sample information in other clustering categories is used as the different-class sample information of the sample information x. Then, based on the center point vector of the clustering category to which the sample information x belongs and the center point vectors of each other clustering category, the cosine distance between the clustering category to which the sample information x belongs and each other clustering category can be determined. The clustering category with the smallest cosine distance is determined as the different-class category of the sample information x. Calculate the cosine distance between the sample feature vector of the sample information x and the sample feature vectors of each same-class sample information, and then average these multiple cosine distances to obtain the same-class average distance. Calculate the cosine distance between the sample feature vector of the sample information x and the sample feature vectors of each different-class sample information in the different-class category, and then average these multiple cosine distances to obtain the different-class average distance. Finally, the maximum value between the same-class average distance and the different-class average distance can be determined, and the difference between the different-class average distance and the same-class average distance is divided by this maximum value to obtain the clustering index of the sample information x in the second-round clustering process. The silhouette coefficient index can satisfy the following formula (1-4).

[0136]

[0137] Among them, (x) is the clustering index of the sample information x, a(x) represents the same-class average distance of the sample information x, and b(x) represents the different-class average distance of the sample information x. It should be noted that this clustering index is calculated separately for each sample information x. When a global clustering index is required, the average value of the clustering indices of all sample information can be used.

[0138] In some embodiments, the unlabeled sample information includes at least one unlabeled known-class sample information. Based on the clustering index of each sample information in each round of clustering process, to determine multiple information categories from multiple clustering categories in all clustering processes can be achieved in the following way: based on the clustering index of each sample information in each round of clustering process, determine the average clustering index of the unlabeled known-class sample information in each round of clustering process; the multiple clustering categories after the clustering process corresponding to the maximum average clustering index are determined as multiple information categories.

[0139] Here, at least one unlabeled known-class sample information in the unlabeled sample set of the preset sample set is provided. The unlabeled known-class sample information refers to the sample information whose true class is a known class but has not been labeled in advance. After obtaining the clustering index of each sample information, the clustering index of the unlabeled known-class sample information can be determined based on the clustering index of each sample information. Then, the average of the clustering indexes of each unlabeled known-class sample information is taken to obtain the average clustering index. That is, each round of clustering process has its corresponding average clustering index. The maximum value is selected from all the average clustering indexes as the maximum average clustering index. The multiple clustering categories obtained after the clustering process corresponding to the maximum average clustering index are determined as multiple information categories. At the same time, the information processing model to be trained during the training process corresponding to the maximum average clustering index is determined as the final pre-trained information processing model.

[0140] In some embodiments, after obtaining the multiple clustering categories after the (i + 1)-th round of clustering, based on the clustering index of each sample information during the (i + 1)-th round of clustering, the category clustering index of each clustering category among the multiple clustering categories after the (i + 1)-th round of clustering is determined; the clustering category corresponding to the minimum category clustering index is scattered, and each sample information in the clustering category corresponding to the minimum category clustering index is determined as a new sample category; each sample category and each non-scattered merged clustering category are clustered again to obtain the multiple clustering categories after the (i + 2)-th round of clustering.

[0141] For example, if 100 clustering categories are obtained after the second clustering, for each of the 100 clustering categories, the clustering index of each sample information in the clustering category can be determined, and the average of the clustering indexes of all the sample information in the clustering category is taken to obtain the category clustering index of the clustering category. After obtaining the category clustering indexes of the 100 clustering categories, the minimum value is determined as the minimum category clustering index. If the clustering category corresponding to the minimum category clustering index is the 21st clustering category, then the 21st clustering category is scattered, and each sample information in the 21st clustering category is determined as a new sample category. The remaining 99 clustering categories and all the newly obtained sample categories are clustered again to obtain the multiple clustering categories after the third round of clustering.

[0142] It should be noted that the category scattering method described above can be adopted in each round of clustering process in the embodiments of the present application, or can be used in combination with the clustering methods in other embodiments.

[0143] The embodiments of the present application scatter and re-cluster the clustering category with the minimum category clustering index, so as to separate the sample information that is wrongly clustered into other categories, improve the accuracy of the clustering process, and further improve the accuracy of information classification.

[0144] Step S206: The server uses a pre-trained information processing model to extract features from the information to be classified, obtaining a feature vector of the information to be classified.

[0145] Step S207: The server determines the distances between the feature vector and the feature centers of each information category among multiple information categories.

[0146] Here, the method for calculating the distances between the feature vector and the feature centers of each information category among multiple information categories can refer to the cosine calculation method in the above-mentioned other embodiments.

[0147] Step S208: The server determines the information category of the information to be classified from among multiple information categories based on the distances.

[0148] Here, the information category with the smallest distance can be used as the information category of the information to be classified.

[0149] Step S209: The server sends the information category of the information to be classified to the terminal.

[0150] Here, if the information category of the information to be classified is a known category, the specific category name, such as "animal", is sent to the terminal; if the information category of the information to be classified is a category that has not been labeled during the construction of a preset sample set, that is, an unknown category, the category serial number, such as "Category 1", is sent to the terminal.

[0151] Step S210: The terminal displays the information category of the information to be classified on the current interface.

[0152] In the embodiments of the present application, when classifying information, the preset sample set can be first input into the information processing model to be trained. Through the information processing model to be trained, multi-round clustering processing is performed on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model. That is, in the model training process of the embodiments of the present application, the number of information categories to be classified is adjusted through multi-round clustering processing, so as to achieve an accurate estimation of the number of information categories. After obtaining multiple information categories and a pre-trained information processing model, the information to be classified can be feature-extracted into a feature vector through the pre-trained information processing model, and based on the distances between the feature vector and the feature centers of each information category among multiple information categories, the information category of the information to be classified is determined. Therefore, the embodiments of the present application can achieve the classification of information in the case of an unknown number of categories, effectively improving the effect of open-set recognition. In addition, in the embodiments of the present application, there is no need to train and cluster the information processing model based on new information to be classified, improving the information classification efficiency.

[0153] In some embodiments, multiple information categories and pre-trained information processing models can be obtained through pre-training, and the multiple information categories and pre-trained information processing models can be stored in a database. After the terminal sends an information classification request to the server, the server can directly use the trained pre-trained information processing model to extract features from the new information to be classified in response to the information classification request, and then calculate the distances between the feature vectors of the information to be classified and the feature centroids of all information categories, and determine the information category with the closest distance as the information category of the information to be classified, without the need to perform another training and clustering operation, saving the resources and time required for training and improving the information classification efficiency.

[0154] Next, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0155] The embodiments of the present application propose an end-to-end open-set recognition algorithm for the case where the number of new categories is unknown. This algorithm is the above-mentioned information classification method, which can effectively estimate the number of new categories and at the same time realize the classification of known categories and new categories. The embodiments of the present application aim at the open-set recognition problem of unclassified categories, so it can be used in any information classification scenario (such as image classification scenario, text classification scenario, speech classification scenario, etc.). An exemplary description will be given below using the image classification scenario. The embodiments of the present application only need to select a small number of images from the training set for annotation to achieve image classification. Different from the related technologies where full supervision and semi-supervised classification need to select a part of all possible images for annotation respectively, the embodiments of the present application only need to randomly select some images, which do not necessarily cover all possible categories, greatly saving the annotation resources.

[0156] Figure 7 is the framework structure diagram provided by the embodiments of the present application. As Figure 7 shown, the process of the embodiments of the present application can be divided into two parts that are alternately carried out. One is to train the feature extraction network (the information processing model to be trained) through the true labels (annotated sample sets) of the labeled data (annotated sample information) and the pseudo-labels (unknown categories) of the unlabeled data (unlabeled sample information); the other is to update the pseudo-labels of the unlabeled data through the features extracted by the trained feature extraction network. The feature extraction network can be any image feature extraction network, such as VGG, ResNet, etc. Therefore, it is not shown in Figure 7As shown in. Open-set recognition is achieved by contrastive learning (unsupervised learning) of the pseudo-labels of the labeled sample set and the unlabeled sample set. That is to say, the feature vectors of the labeled data and the unlabeled data can be extracted by a trained feature extraction network, and a bottom-up clustering method can be performed based on the feature vectors to obtain clustering categories. After multiple rounds of clustering processing such as category shuffling and re-clustering, the classification result of the final unlabeled sample set can be obtained. The classification result includes prototypes, and the prototypes are used to represent each category. For example, the feature center point of the category is used to represent it.

[0157] The labeled sample set and the unlabeled sample set are specifically described below. In the open-set recognition problem, it can be assumed that the dataset D (preset sample set) contains two parts: the labeled part D l (labeled sample set) and the unlabeled part D u (unlabeled sample set). Among them, D l ={(x j , y j ), j = 1,..., N}, where x j and y j represent the image and the corresponding label respectively, and these N labeled data all come from T known categories. D u ={(x j , y j ), j = 1,..., M}, where M represents the number of unlabeled data, and the unlabeled data may come from known categories or unknown categories. D u / D l represents the unlabeled known-class data, and the types of these data should be the same as those in D l . The information classification method provided in the embodiments of the present application aims to divide the data in D u into the correct categories.

[0158] The training of the feature network is specifically described below. Common image classification networks can be used for feature extraction, such as ResNet, etc. However, these image classification networks generally have a classification layer at the end to map the image features to fixed categories. In order to eliminate the fixation on categories, the last classification layer in the image classification network can be removed, and instead, the features extracted from the last feature layer of the network are used for constraints. For example, for Figure 5For any one of the three image classification networks, removing the last fully connected layer 501, that is, only retaining the network before the last pooling, can be used as the feature extraction network in the embodiments of the present application. When training and learning the feature extraction network, the contrast loss can be used as the objective function. After obtaining the contrast loss, the gradient descent method based on the Adam optimization algorithm (Adaptive momentum, a stochastic optimization method with adaptive momentum) can be used to update the model parameters of the feature extraction network. Exemplarily, in the Adam optimization algorithm, weight decay is not used, and the initial learning rate is 0.0001, which is reduced to half of the original every 10 training epochs. The training stop condition is determined by the clustering result of the bottom-up open set clustering algorithm. The objective function satisfies the following formula (2-1).

[0159]

[0160] Among them, L(x; P(x), N(x)) represents the contrast loss, z represents the feature corresponding to the input image x, and z T represents the transpose of z. z + ∈P(x) represents the positive sample set, that is, the sample set of the same category as the image x, while z - ∈N(x) represents the negative sample set, that is, the sample set of different categories from the image x, and τ is an adjustable parameter. τ can be adjusted according to different learning tasks, and users can set it by themselves based on experience. |P(x)| is the number of samples in the positive sample set.

[0161] For the construction of the positive and negative sample sets, there are three types: self-supervised, supervised, and unsupervised. The construction of the positive and negative sample sets of the self-supervised loss is based on an assumption: some specific data augmentations, such as rotation, translation, flipping, etc., do not affect the high-level features of the samples. Therefore, for the self-supervised loss, the positive sample set P(x) of the image x is the feature set extracted after various data augmentations of itself, while the negative sample set N(x) is the feature set obtained after data augmentations of other images. The sample construction of the supervised loss is for the labeled dataset D l In this regard, for this part of the data, the true labels are directly used, and the features extracted from the images of the same category as x constitute the positive sample set P(x), while the features extracted from the images of different categories constitute the negative sample set N(x). In addition, in order to improve the generalization of the network, certain data augmentations can also be performed on the images here, which will not affect the construction of the positive and negative sample sets. The sample construction of the unsupervised loss is similar to that of the supervised loss, and both divide the positive and negative sample sets by the category of the images. However, the difference is that since it is for the unlabeled dataset D u, here pseudo - labels are used instead of real labels. The update process of pseudo - labels can be referred to the bottom - up open - set clustering algorithm below.

[0162] The bottom - up open - set clustering algorithm is specifically described below. To obtain pseudo - labels of unlabeled data for training the feature extraction network, it is necessary to classify the unlabeled data. However, since the number of classes is unknown, it is impossible to classify the unlabeled data through a classifier as in the conventional method. Therefore, the embodiment of this application proposes a bottom - up open - set clustering algorithm. Initially, all unlabeled data are regarded as separate classes, and together with the labeled known classes, there are M + T classes in total. During the training process, these M + T classes are gradually merged until the stopping criterion is met. During the class - merging process, the following two principles are followed: First, start from the smallest class, and second, start from the closest class. Starting from the smallest class means preferentially selecting the class with the smallest number of samples and merging it into other classes. Starting from the closest class has two meanings. One is that when there are multiple classes with the same small number of samples, we preferentially merge into the class closer to another class. The other is that the samples with the smallest number will be merged into the class closest to it. Here, the distance between two classes is represented by the cosine distance (cosine distance) between the central points of the two classes. The cosine distance can satisfy the following formula (2 - 2).

[0163]

[0164] Among them, D(μ i , μ j ) represents the cosine distance between the central points of the i - th class and the j - th class, μ i represents the central point of the i - th class, that is, the average feature of all features in this class, μ j represents the central point of the j - th class, represents the transpose of the central point of the i - th class, ‖·‖ represents the L2 norm. The central point can satisfy the following formula (2 - 3).

[0165]

[0166] Among them, μ represents the central point of the class, z i is the i - th feature in this class, and n is the total number of features in this class.

[0167] In the actual training of the feature extraction network, at the end of each epoch (that is, all samples have been used in training once), a class - merging operation is performed. The number of classes m participating in each merge is M I ×p, where M I is the current number of classes, and I represents the current epoch number, Represents rounding up. Additionally, it should be noted that known classes will not be merged into other classes, but other classes may be merged into known classes.

[0168] Regarding the stop condition for merging, due to the lack of prior knowledge about the unlabeled parts, forcibly setting an index is likely to fall into a local minimum. Therefore, in the embodiments of this application, merging and training are continuously performed, and the clustering index, data classification (clustering category), and feature extraction network after each merge are recorded until only known classes remain in the merged result. Then, the feature extraction network and clustering category with the best clustering index are selected as the final result. The clustering index can use the Silhouette index. The Silhouette index can satisfy the following formula (2-4).

[0169]

[0170] Among them, s(x) is the clustering index of sample x, a(x) represents the average distance of sample x to other samples in the same class, and b(x) represents the average distance of sample x to all samples in the nearest class to the class where it is located. It should be noted that this clustering index is calculated separately for each sample x. When a global clustering index is required, the average value of the clustering indexes of all samples can be used as the average index.

[0171] Figure 8 This is the sample clustering process of the above bottom-up open clustering algorithm. See Figure 8 , the number of clustering classes changes after each merge. For example, the numbers of clustering classes obtained in four merge processes are 35533, 569, 51, and 10 respectively. Figure 9 This is the curve graph of the change in the clustering index during the merging process. See Figure 9 , the left graph is the curve graph of the change in the clustering index when the final clustering result is 10 classes, and the right graph is the curve graph of the change in the clustering index when the final clustering result is 100 classes. When the average clustering index of the unlabeled known class data in the unlabeled sample set reaches the maximum, the corresponding feature extraction network and clustering category are used as the final result.

[0172] During the class merging process, class shuffling can also be performed. Since there is a problem in the above clustering process: when a sample is wrongly clustered into other classes, there is no way to distinguish it from samples in different classes. Therefore, to make up for this problem, a class shuffling process can be introduced. Specifically, after each epoch of training, first calculate the clustering index of each class, shuffle the class with the smallest clustering index, where each sample will be assigned a separate class label and then merged into other classes again, so as to correct the clustering errors.

[0173] During the category merging process, due to initialization, the known classes with labels are initially represented by the center points as the representatives of the categories, while the unlabeled samples are each considered as a separate class by themselves. This results in some unlabeled known classes potentially deviating from the center points of the known classes and being closer to the individual samples of other classes, leading to misclassification of the unlabeled known class samples. Therefore, during initialization, each labeled known class sample can also be regarded as an additional separate class and participate in the clustering. That is to say, the initial number of categories is M + N + T, not just M + T, so that the unlabeled known class samples may be clustered into the known similar labeled samples of the same category. Additionally, to prevent conflicts between the clustering results of the labeled known class samples and their true labels, the embodiments of this application can add a mandatory condition: namely, the class where the labeled known class sample is located can only be clustered into the correct category. That is, the class containing the known class sample x can only be clustered into the category of other known class samples that are of the same type as the sample x and only have such samples.

[0174] After obtaining the trained feature extraction network, the new samples (information to be classified) can be classified based on the trained feature extraction network. However, since the trained feature extraction network does not have a classification layer, it cannot directly obtain the classification results of the new samples. The embodiments of this application can use the trained feature extraction network to extract features from the new samples, and then calculate the distances between this feature and the feature center points of all categories. The category with the closest distance is the category to which this sample belongs. In the above category inference process, after adding new samples again, there is no need to retrain and cluster the feature extraction network, saving the time and resources required for training and improving the classification efficiency.

[0175] The embodiments of this application propose an end-to-end method framework that can adjust the number of categories to be classified during the training process, thereby achieving an accurate estimation of the number of categories and effectively improving the effect of open-set recognition. Figure 10 This is a comparison chart of the category number prediction results between the embodiments of this application and other methods. Among them, the number (size) + silhouette coefficient clustering index (S) represents the information classification method provided by the embodiments of this application, the number (size) represents clustering starting from the category with the smallest number of merges each time, the distance (Distance) represents the result of not starting clustering from the category with the smallest number, S represents the silhouette coefficient clustering index, and D and C represent other clustering indexes that are not the silhouette coefficient. Figure 11 This is a comparison chart of the category prediction accuracy rates between the embodiments of this application and other methods. Among them, the other methods are all results under the condition of the correct number of categories, and the embodiments of this application are results when estimating the number of categories. It can be found that through the proposed open-set clustering algorithm, the embodiments of this application can achieve the category division of data in the case of an unknown number of categories, and there is a significant improvement in accuracy compared with other related methods.

[0176] It should be noted that for the information classification method provided in the embodiments of the present application, in addition to being applied to the image field, if the feature extractor is replaced with other types, such as a text or speech feature extractor, and then modified to the corresponding data augmentation method, open-set recognition in other business scenarios can also be achieved.

[0177] It can be understood that in the embodiments of the present application, data related to user information and the like is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0178] Next, the exemplary structure of the information classification device 455 provided in the embodiments of the present application implemented as software modules will be continued. In some embodiments, as Figure 2 shown, the software modules in the information classification device 455 stored in the memory 450 may include: a clustering processing module 4551, configured to input a preset sample set into an information processing model to be trained, and through the information processing model to be trained, perform multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model. A feature extraction module 4552, configured to use the pre-trained information processing model to extract features from the information to be classified to obtain a feature vector of the information to be classified. A distance determination module 4553, configured to determine the distance between the feature vector and the feature center point of each information category among the multiple information categories. A category determination module 4554, configured to determine the information category of the information to be classified from the multiple information categories based on the distance.

[0179] In some embodiments, the preset sample set includes a labeled sample set and an unlabeled sample set. The labeled sample set includes multiple labeled sample information and the known category of each labeled sample information. The unlabeled sample set includes multiple unlabeled sample information; the multiple labeled sample information and the multiple unlabeled sample information constitute the multiple sample information in the preset sample set; each round of clustering processing in the multiple rounds of clustering processing corresponds to a training process of the information processing model to be trained; the clustering processing module 4551 is further configured to determine the unknown category of each unlabeled sample information; input the preset sample set into the information processing model to be trained, and use the known category and the unknown category as sample labels to perform multiple trainings on the information processing model to be trained to obtain a pre-trained information processing model; based on the sample feature vectors extracted by the information processing model to be trained during each training, perform one round of clustering processing on the multiple sample information to obtain multiple information categories.

[0180] In some embodiments, the clustering processing module 4551 is further configured to, for the first training process, determine each unlabeled sample information as a category respectively, to obtain the unknown category of each unlabeled sample information; for the (i + 1)-th training process, obtain the clustering category obtained by each unlabeled sample information during the i-th round of clustering processing, and determine the clustering category as the unknown category of the unlabeled sample information during the (i + 1)-th training process; i is an integer greater than or equal to 1.

[0181] In some embodiments, the clustering processing module 4551 is further configured to, during the (i + 1)-th training process, use the information processing model to be trained to extract features from all sample information, to obtain the sample feature vector of each sample information; the information processing model to be trained is the information processing model obtained after the i-th training, i is an integer greater than or equal to 1; using the known category and the unknown category as sample labels, based on the sample labels and the sample feature vector of each sample information, update the model parameters in the information processing model to be trained, to obtain the information processing model after the (i + 1)-th training; determine the pre-trained information processing model from multiple information processing models to be trained during all training processes.

[0182] In some embodiments, the clustering processing module 4551 is further configured to determine the contrastive loss of the information processing model to be trained based on the sample labels and the sample feature vector of each sample information; use the contrastive loss to update the model parameters in the information processing model to be trained, to obtain the information processing model after the (i + 1)-th training.

[0183] In some embodiments, the clustering processing module 4551 is further configured to, for each labeled sample information, determine the labeled positive sample feature set and the labeled negative sample feature set of the labeled sample information based on the known category; the labeled positive sample feature set includes the sample feature vectors of the labeled sample information belonging to the same known category as the labeled sample information, and the labeled negative sample feature set includes the sample feature vectors of the labeled sample information belonging to different known categories from the labeled sample information; for each unlabeled sample information, determine the unlabeled positive sample feature set and the unlabeled negative sample feature set of the unlabeled sample information based on the unknown category; the unlabeled positive sample feature set includes the sample feature vectors of the unlabeled sample information belonging to the same unknown category as the unlabeled sample information, and the unlabeled negative sample feature set includes the sample feature vectors of the unlabeled sample information belonging to different unknown categories from the unlabeled sample information; determine the contrastive loss of the information processing model to be trained based on the sample feature vector of each sample information, the labeled positive sample feature set and the labeled negative sample feature set of each labeled sample information, and the unlabeled positive sample feature set and the unlabeled negative sample feature set of each unlabeled sample information.

[0184] In some embodiments, the information classification device 455 further includes an augmentation module configured to perform data augmentation on each sample information to obtain augmented sample information for each sample information; extract feature vectors of each augmented sample information by using the information processing model to be trained to obtain augmented sample feature vectors of each augmented sample information; for each sample information, determine the self-supervised positive sample feature set of the sample information from the multiple augmented sample feature vectors corresponding to the sample information, and determine the self-supervised negative sample feature set of the sample information from the sample feature vectors of other sample information in the preset sample set and the multiple augmented sample feature vectors corresponding to the other sample information; the clustering processing module 4551 is further configured to determine a first contrast loss of the information processing model to be trained based on the sample feature vector of each sample information, the labeled positive sample feature set and the labeled negative sample feature set of each labeled sample information, and the unlabeled positive sample feature set and the unlabeled negative sample feature set of each unlabeled sample information; determine a second contrast loss of the information processing model to be trained based on the sample feature vector of each sample information, and the self-supervised positive sample feature set and the self-supervised negative sample feature set of each sample information; sum the first contrast loss and the second contrast loss to obtain the contrast loss of the information processing model to be trained.

[0185] In some embodiments, the clustering processing module 4551 is further configured to, in the (i + 1)-th round of clustering processing, re-cluster the multiple clustering categories after the i-th round of clustering processing based on the sample feature vectors of each sample information extracted by the information processing model to be trained to obtain multiple clustering categories after the (i + 1)-th round of clustering processing; the information processing model to be trained is the information processing model obtained after the i-th training; determine the clustering index of each sample information in the (i + 1)-th round of clustering processing based on the multiple clustering categories after the (i + 1)-th round of clustering processing and the sample feature vector of each sample information; if each clustering category in the multiple clustering categories after the K-th round of clustering processing is a known category after K rounds of clustering processing, stop the clustering processing, and determine multiple information categories from the multiple clustering categories in all the clustering processing based on the clustering index of each sample information in each round of clustering processing; K is an integer greater than or equal to 1, and i is an integer between 1 and K.

[0186] In some embodiments, the clustering processing module 4551 is also used to determine a category to be merged from multiple clustering categories after the i-th round of clustering processing, and the number of sample information in the category to be merged is the smallest among the multiple clustering categories after the i-th round of clustering processing; for each clustering category, based on the sample feature vector of each sample information in the clustering category, determine the center point vector of the clustering category; based on the center point vector of the category to be merged and the center point vector of each other clustering category, determine the cosine distance between the category to be merged and each other clustering category; determine the clustering category corresponding to the minimum cosine distance as the target clustering category, and merge the category to be merged into the target clustering category to obtain a new clustering category; wherein, the other clustering categories except the category to be merged in the multiple clustering categories after the i-th round of clustering processing, and the new clustering category, together constitute the multiple clustering categories after the i+1-th round of clustering processing.

[0187] In some embodiments, the clustering processing module 4551 is also used to determine the same type of sample information and the different type of sample information for each sample information based on multiple clustering categories after the i+1 round of clustering processing; for each sample information, determine the different type of category with the smallest distance from the clustering category to which the sample information belongs; based on the sample feature vector of the sample information and the sample feature vectors of all the same type of sample information of the sample information, determine the same type average distance between the sample information and all the same type of sample information; based on the sample feature vector of the sample information and the sample feature vectors of all the different type of sample information of the sample information, determine the different type average distance between the sample information and all the different type of sample information in the different type; based on the sample feature vector of the sample information and the sample feature vectors of all the different type of sample information of the sample information, determine the clustering index of the sample information in the i+1 clustering processing process.

[0188] In some embodiments, the unlabeled sample information includes at least one unlabeled known class sample information; the clustering processing module 4551 is also used to determine the average clustering index of the unlabeled known class sample information in each round of clustering processing based on the clustering index of each sample information in each round of clustering processing; and determine multiple clustering categories after the clustering processing process corresponding to the maximum average clustering index as multiple information categories.

[0189] In some embodiments, the information classification device 455 also includes a category breaking up module, which is used to determine the category clustering index of each clustering category in the multiple clustering categories after the i+1th round of clustering processing based on the clustering index of each sample information in the i+1th round of clustering processing after obtaining multiple clustering categories after the i+1th round of clustering processing; break up the clustering category corresponding to the minimum category clustering index, and determine each sample information in the clustering category corresponding to the minimum category clustering index as a new sample category; cluster each sample category and each merged cluster category that has not been broken up again to obtain multiple clustering categories after the i+2th round of clustering processing.

[0190] In some embodiments, the information classification device 455 further includes an initialization module, which is configured to, when performing the first training on the information processing model to be trained, determine each of the labeled sample information as a category, so as to obtain the initialized known category of each labeled sample information; input the preset sample set into the information processing model to be trained, and use the known category, unknown category, and initialized known category as sample labels to perform the first training on the information processing model to be trained, so as to obtain the information processing model after the first training.

[0191] The embodiments of the present application provide a computer program product, which includes a computer program or computer-executable instructions, and the computer program or computer-executable instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer-executable instructions or computer program from the computer-readable storage medium, and the processor executes the computer-executable instructions or computer program, so that the electronic device executes the information classification method described above in the embodiments of the present application.

[0192] The embodiments of the present application provide a computer-readable storage medium storing computer-executable instructions, where computer-executable instructions or a computer program are stored, and when the computer-executable instructions or the computer program are executed by a processor, the processor will be caused to execute the information classification method provided by the embodiments of the present application. For example, Figure 3 the information classification method shown.

[0193] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0194] In some embodiments, the computer-executable instructions may be in the form of a program, software, software module, script, or code, and may be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0195] As an example, the computer-executable instructions may or may not correspond to a file in the file system, and may be stored as part of a file that stores other programs or data. For example, they may be stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program being discussed, or stored in multiple cooperating files (for example, files that store one or more modules, subroutines, or code portions).

[0196] As an example, the computer-executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected through a communication network.

[0197] In summary, through the embodiments of the present application, an end-to-end open-set recognition algorithm for the case where the number of new categories is unknown is implemented. This method can effectively estimate the number of new categories, and at the same time realize the classification of known categories and new categories. Only a small number of images need to be selected and labeled in the training set to achieve image classification, greatly saving labeling resources and improving classification efficiency. The embodiments of the present application can adjust the number of categories to be classified during the training process, so as to achieve an accurate estimate of the number of categories and effectively improve the effect of open-set recognition.

[0198] The above is only the embodiments of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An information classification method, characterized in that, The method includes: Inputting a preset sample set into an information processing model to be trained, and through the information processing model to be trained, performing multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model; Using the pre-trained information processing model to extract features from the information to be classified to obtain a feature vector of the information to be classified; Determining the distances between the feature vector and the feature center points of each information category among the multiple information categories; Based on the distances, determining the information category of the information to be classified from the multiple information categories.

2. The method according to claim 1, characterized in that, The preset sample set includes a labeled sample set and an unlabeled sample set. The labeled sample set includes multiple labeled sample information and the known categories of each labeled sample information. The unlabeled sample set includes multiple unlabeled sample information. The multiple labeled sample information and the multiple unlabeled sample information constitute the multiple sample information in the preset sample set. Each round of clustering processing in the multiple rounds of clustering processing corresponds to a training process of the information processing model to be trained; The step of inputting the preset sample set into the information processing model to be trained, and through the information processing model to be trained, performing multiple rounds of clustering processing on the unlabeled sample information in the preset sample set to obtain multiple information categories and a pre-trained information processing model includes: Determining the unknown category of each unlabeled sample information; Inputting the preset sample set into the information processing model to be trained, and using the known category and the unknown category as sample labels to perform multiple trainings on the information processing model to be trained to obtain the pre-trained information processing model; Based on the sample feature vectors extracted by the information processing model to be trained during each training, performing one round of clustering processing on the multiple sample information to obtain the multiple information categories.

3. The method according to claim 2, wherein The step of determining the unknown category of each unlabeled sample information includes: For the first training process, determining each unlabeled sample information as a category to obtain the unknown category of each unlabeled sample information; For the (i + 1)-th training process, obtaining the clustering category of each unlabeled sample information during the i-th round of clustering processing, and determining the clustering category as the unknown category of the unlabeled sample information during the (i + 1)-th training process; i is an integer greater than or equal to 1.

4. The method according to claim 2, characterized in that, The step of inputting the preset sample set into the information processing model to be trained, and using the known category and the unknown category as sample labels to perform multiple trainings on the information processing model to be trained to obtain the pre-trained information processing model includes: During the (i + 1)-th training process, Using the information processing model to be trained to extract features from all sample information to obtain a sample feature vector of each sample information; the information processing model to be trained is the information processing model obtained after the i-th training, and i is an integer greater than or equal to 1; Using the known classes and the unknown classes as sample labels, based on the sample labels and the sample feature vectors of each sample information, update the model parameters in the information processing model to be trained to obtain the information processing model after the (i + 1)-th training; Determine the pre-trained information processing model from multiple information processing models to be trained during the entire training process.

5. The method according to claim 4, characterized in that The step of updating the model parameters in the information processing model to be trained based on the sample labels and the sample feature vectors of each sample information to obtain the information processing model after the (i + 1)-th training includes: Determine the contrastive loss of the information processing model to be trained based on the sample labels and the sample feature vectors of each sample information; Use the contrastive loss to update the model parameters in the information processing model to be trained to obtain the information processing model after the (i + 1)-th training.

6. The method according to claim 5, wherein The step of determining the contrastive loss of the information processing model to be trained based on the sample labels and the sample feature vectors of each sample information includes: For each labeled sample information, determine the labeled positive sample feature set and the labeled negative sample feature set of the labeled sample information based on the known classes; the labeled positive sample feature set includes the sample feature vectors of the labeled sample information belonging to the same known class as the labeled sample information, and the labeled negative sample feature set includes the sample feature vectors of the labeled sample information belonging to different known classes from the labeled sample information; For each unlabeled sample information, determine the unlabeled positive sample feature set and the unlabeled negative sample feature set of the unlabeled sample information based on the unknown classes; the unlabeled positive sample feature set includes the sample feature vectors of the unlabeled sample information belonging to the same unknown class as the unlabeled sample information, and the unlabeled negative sample feature set includes the sample feature vectors of the unlabeled sample information belonging to different unknown classes from the unlabeled sample information; Determine the contrastive loss of the information processing model to be trained based on the sample feature vectors of each sample information, the labeled positive sample feature set and the labeled negative sample feature set of each labeled sample information, and the unlabeled positive sample feature set and the unlabeled negative sample feature set of each unlabeled sample information.

7. The method according to claim 6, wherein The method further includes: Perform data augmentation on each sample information to obtain the augmented sample information of each sample information; Use the information processing model to be trained to extract features from each augmented sample information to obtain the augmented sample feature vectors of each augmented sample information; For each sample information, determine the self-supervised positive sample feature set of the sample information as the multiple augmented sample feature vectors corresponding to the sample information, and determine the self-supervised negative sample feature set of the sample information as the sample feature vectors of other sample information in the preset sample set and the multiple augmented sample feature vectors corresponding to other sample information; Determining the contrastive loss of the information processing model to be trained based on the sample feature vectors of each of the sample information, the labeled positive sample feature sets and labeled negative sample feature sets of each of the labeled sample information, and the unlabeled positive sample feature sets and unlabeled negative sample feature sets of each of the unlabeled sample information, includes: Determining the first contrastive loss of the information processing model to be trained based on the sample feature vectors of each of the sample information, the labeled positive sample feature sets and labeled negative sample feature sets of each of the labeled sample information, and the unlabeled positive sample feature sets and unlabeled negative sample feature sets of each of the unlabeled sample information; Determining the second contrastive loss of the information processing model to be trained based on the sample feature vectors of each of the sample information, and the self-supervised positive sample feature sets and self-supervised negative sample feature sets of each of the sample information; Summing the first contrastive loss and the second contrastive loss to obtain the contrastive loss of the information processing model to be trained.

8. The method according to claim 2, wherein Performing a round of clustering processing on the multiple sample information based on the sample feature vectors extracted by the information processing model to be trained during each training, to obtain the multiple information categories, includes: During the (i + 1)-th round of clustering processing, re-clustering the multiple clustering categories after the i-th round of clustering processing based on the sample feature vectors of each of the sample information extracted by the information processing model to be trained; the information processing model to be trained is the information processing model obtained after the i-th training; Determining the clustering index of each of the sample information during the (i + 1)-th round of clustering processing based on the multiple clustering categories after the (i + 1)-th round of clustering processing and the sample feature vectors of each of the sample information; After K rounds of clustering processing, if each of the multiple clustering categories after the K-th round of clustering processing is a known category, stop the clustering processing, and determine the multiple information categories from the multiple clustering categories during all clustering processing based on the clustering indices of each of the sample information during each round of clustering processing; K is an integer greater than or equal to 1, and i is an integer between 1 and K.

9. The method according to claim 8, characterized in that Re-clustering the multiple clustering categories after the i-th round of clustering processing based on the sample feature vectors of each of the sample information extracted by the information processing model to be trained to obtain the multiple clustering categories after the (i + 1)-th round of clustering processing, includes: Determining the categories to be merged from the multiple clustering categories after the i-th round of clustering processing, where the number of sample information in the categories to be merged is the smallest among the multiple clustering categories after the i-th round of clustering processing; For each of the clustering categories, determining the center point vector of the clustering category based on the sample feature vectors of each of the sample information in the clustering category; Determining the cosine distance between the category to be merged and each of the other clustering categories based on the center point vector of the category to be merged and the center point vectors of each of the other clustering categories; The cluster category corresponding to the minimum cosine distance is determined as the target cluster category, and the category to be merged is merged into the target cluster category to obtain a new cluster category; wherein, the other cluster categories except the category to be merged in the multiple cluster categories after the i-th round of clustering processing, and the new cluster category, together constitute the multiple cluster categories after the i+1-th round of clustering processing.

10. The method according to claim 8, characterized in that, The determining of the clustering index of each of the sample information in the i+1 round of clustering processing based on the multiple clustering categories after the i+1 round of clustering processing and the sample feature vector of each of the sample information includes: Determine similar sample information and heterogeneous sample information for each of the sample information based on the multiple clustering categories after the (i+1) round of clustering processing; For each piece of sample information, determining a heterogeneous category having the smallest distance from the cluster category to which the sample information belongs; Determine, based on a sample feature vector of the sample information and sample feature vectors of all sample information of the same type as the sample information, a similar average distance between the sample information and all sample information of the same type; Determining a heterogeneous average distance between the sample information and all heterogeneous sample information in the heterogeneous category based on a sample feature vector of the sample information and sample feature vectors of all heterogeneous sample information of the sample information; Based on the same-class average distance and the different-class average distance, a clustering index of the sample information in the (i+1)th clustering processing process is determined.

11. The method according to claim 8, wherein The unlabeled sample information includes at least one unlabeled known class sample information; The step of determining the multiple information categories from the multiple clustering categories in the entire clustering process based on the clustering index of each sample information in each round of clustering process includes: Based on the clustering index of each sample information in each round of clustering processing, determining the average clustering index of the unlabeled known class sample information in each round of clustering processing; The plurality of clustering categories after the clustering process corresponding to the maximum average clustering index are determined as the plurality of information categories.

12. The method according to claim 8, wherein The method further comprises: After obtaining a plurality of clustering categories after the i+1th round of clustering processing, determining a category clustering index of each of the clustering categories in the plurality of clustering categories after the i+1th round of clustering processing based on the clustering index of each of the sample information in the i+1th round of clustering processing; The clustering categories corresponding to the minimum category clustering index are scattered, and each sample information in the clustering category corresponding to the minimum category clustering index is determined as a new sample category; Each sample category and each unbroken merged cluster category are clustered again to obtain multiple cluster categories after the i+2th round of clustering processing.

13. The method according to claim 2, characterized in that, The method further comprises: When the information processing model to be trained is trained for the first time, each of the labeled sample information is determined as a category to obtain an initial known category for each of the labeled sample information; Input the preset sample set into the information processing model to be trained, and use the known category, the unknown category, and the initialized known category as sample labels to perform the first training on the information processing model to be trained, and obtain the information processing model of the first training.

14. An information classification device, characterized in that, The device includes: A clustering processing module, configured to input a preset sample set into an information processing model to be trained, and perform multiple rounds of clustering processing on the unlabeled sample information in the preset sample set through the information processing model to be trained, so as to obtain multiple information categories and a pre-trained information processing model; A feature extraction module, configured to use the pre-trained information processing model to extract features from the information to be classified, so as to obtain a feature vector of the information to be classified; A distance determination module, configured to determine the distance between the feature vector and the feature center point of each information category in the multiple information categories; A category determination module, configured to determine the information category of the information to be classified from the multiple information categories based on the distance.

15. An electronic device, characterized in that, The electronic device includes: A memory, configured to store computer-executable instructions or computer programs; A processor, configured to implement the information classification method according to any one of claims 1 to 13 when executing the computer-executable instructions or computer programs stored in the memory.

16. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, The computer-executable instructions or computer programs, when executed by the processor, implement the information classification method according to any one of claims 1 to 13.

17. A computer program product, comprising computer-executable instructions or a computer program, characterized in that, The computer-executable instructions or computer programs, when executed by the processor, implement the information classification method according to any one of claims 1 to 13.