Method and related device for fine-grained text category discovery based on noise reduction neighborhood aggregation

Abstract

The present invention belongs to the field of natural language processing, and discloses a method and related device for discovering fine-grained categories of texts based on denoising neighborhood aggregation, including: iteratively updating for a preset number of times, and using the final text feature extractor as the optimized text feature extractor; using the optimized text feature extractor to extract the features of each text in sequence to obtain a number of optimized text features; clustering the number of optimized text features to obtain a number of clustering clusters, and obtaining the cluster labels of each clustering cluster and using the cluster labels of each clustering cluster as the fine-grained categories of each text in each clustering cluster. According to the principle that texts with similar features have the same fine-grained category, in the updating step, three kinds of neighborhood constraints, namely coarse-grained label constraint, mutual neighbor constraint and rank statistic constraint, are used to obtain the denoising neighborhood set of text features, that is, the final neighborhood set, and multi-positive sample contrast learning is used to make the text features and their neighbor momentum features have similar representations, effectively improving the accuracy of the fine-grained categories of texts.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and relates to a method for discovering fine-grained text categories based on noise reduction neighborhood aggregation and related devices. Background Art

[0002] Fine-grained category discovery is mainly used for fine-grained analysis of text. Since the acquisition cost of fine-grained labels for text is relatively high and the quality of the labels is relatively poor, fine-grained category discovery aims to use relatively easily obtainable coarse-grained labels (such as sports) as supervision signals, and automatically discover the potential fine-grained categories (such as basketball) to which the text belongs through representation learning and clustering, so as to achieve the purpose of reducing the annotation cost.

[0003] At present, fine-grained category discovery models all model the features of input text based on contrastive learning of a single text. For example, some scholars use the hierarchical modeling features of pre-trained language models and weighted self-contrastive learning to regulate the feature representation of samples, so that different fine-grained categories are distinguishable. Some scholars use the prior information contained in the coarse-grained classifier as a constraint to adjust the feature representation of input samples. There are also scholars who use the label names of fine-grained categories to generate pseudo-labels for input samples, and use the self-training method to learn the feature representation of input samples.

[0004] Although the above methods have achieved good results in the fine-grained category discovery task, they still have the following disadvantages: the above methods only focus on the feature representation learning of a single text, while ignoring the modeling of semantic feature relationships between multiple texts, which makes the above methods unable to learn the relative relationships between texts and compact fine-grained category representations, resulting in poor distinguishability between different fine-grained categories, thus affecting the performance of the final clustering, and ultimately resulting in the text not being accurately divided into the correct fine-grained categories. Summary of the Invention

[0005] The purpose of the present invention is to overcome the above-mentioned disadvantages of the prior art, and provide a method for discovering fine-grained text categories based on noise reduction neighborhood aggregation and related devices.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions:

[0007] In the first aspect of the present invention, a method for discovering fine-grained text categories based on noise reduction neighborhood aggregation is provided, including:

[0008] Iteratively update the preset number of times, and use the final text feature extractor as the optimized text feature extractor;

[0009] Use an optimized text feature extractor to extract the features of each text in turn, obtaining a number of optimized text features; and cluster the number of optimized text features to obtain a number of clusters, as well as obtain the cluster labels of each cluster and use the cluster labels of each cluster as the fine-grained categories of each text in each cluster.

[0010] Among them, the update steps include:

[0011] Extract the momentum features of all texts through a momentum feature extractor to obtain a momentum set; and use a text feature extractor to extract the features of each text in turn to obtain a number of text features.

[0012] Traverse each text feature, select the momentum features with the top preset number of high similarities to the current text feature from the momentum set and combine them to obtain the initial neighbor set of each text feature; use the coarse-grained label constraint to filter out the momentum features with different coarse-grained labels from the text feature in the initial neighbor set to obtain the first filtered neighbor set of each text feature; use the mutual neighbor constraint to filter out the momentum features that are not mutual neighbors with the text feature in the first filtered neighbor set to obtain the second filtered neighbor set of each text feature; use the rank statistic constraint to filter out the momentum features with different n-order rank statistics of the text corresponding to the text feature in the second filtered neighbor set to obtain the final neighbor set of each text feature; where n is the order of the rank statistic.

[0013] According to the momentum set and the final neighbor set of each text feature, use multi-positive sample contrast learning to update the parameters of the text feature extractor and the parameters of the momentum feature extractor.

[0014] Optionally, the selecting the momentum features with the top preset number of high similarities to the current text feature from the momentum set includes:

[0015] Obtain the similarity between the current text feature and each momentum feature in the momentum set through the following formula:

[0016]

[0017] Where, sim(q i ,h l ) is the similarity between q i and h l , sim represents the cosine similarity function, q i is the text feature of the i-th text, h l is the momentum feature of the l-th text, |||| represents the L2 normalization operation, and T represents the transpose of the vector.

[0018] According to the similarity between the current text feature and each momentum feature in the momentum set, select the momentum features with the top preset number of high similarities to the current text feature from the momentum set.

[0019] Optionally, the step of filtering momentum features with different coarse-grained labels from the text feature in the initial nearest neighbor set by using the coarse-grained label constraint includes:

[0020] Filter the momentum features with different coarse-grained labels from the text feature in the initial nearest neighbor set by the following formula:

[0021]

[0022] where is the first-filtered nearest neighbor set of the text feature of the i-th text, h j is the momentum feature of the j-th text, is the initial nearest neighbor set of the text feature of the i-th text, c i is the coarse-grained label of the text feature of the i-th text, c j is the coarse-grained label of the momentum feature of the j-th text, the ∈ symbol indicates that an element belongs to a certain set, and the ∧ symbol represents the intersection of two sets;

[0023] The step of filtering momentum features that are not mutual nearest neighbors with the text feature from the first-filtered nearest neighbor set to obtain the second-filtered nearest neighbor set of each text feature includes:

[0024] Filter the momentum features that are not mutual nearest neighbors with the text feature from the first-filtered nearest neighbor set by the following formula:

[0025]

[0026] where is the second-filtered nearest neighbor set of the text feature of the i-th text, h i is the momentum feature of the i-th text, is the first-filtered nearest neighbor set of the text feature of the j-th text;

[0027] The step of filtering momentum features with different n-th order rank statistics of the text corresponding to the text feature from the second-filtered nearest neighbor set by using the rank statistic constraint includes:

[0028] Filter the momentum features with different n-th order rank statistics of the text corresponding to the text feature from the second-filtered nearest neighbor set by the following formula:

[0029]

[0030] where is the final nearest neighbor set of the text feature of the i-th text, top n is the n-th order rank statistic function, and the n-th order rank statistic function is used to return the index numbers corresponding to the largest n-dimensional elements of the input features.

[0031] Optionally, updating the parameters of the text feature extractor and the momentum feature extractor according to the momentum set and the final nearest neighbor set of each text feature includes:

[0032] Using multi-positive sample contrast learning as the self-supervised loss function, traversing each text feature, taking all the momentum features in the final nearest neighbor set of the current text feature as positive texts, and taking all the momentum features in the momentum set as negative texts for contrast learning, using the backpropagation algorithm to update the parameters of the text feature extractor, and using the exponential moving average method to update the parameters of the momentum feature extractor.

[0033] Optionally, the self-supervised loss function is specifically:

[0034]

[0035] Among them, is the self-supervised loss function value, N is the number of texts, q i is the text feature of the i-th text, is the set composed of all text features, is the final nearest neighbor set of the text feature of the i-th text, is the number of momentum features included in the final nearest neighbor set of the text feature of the i-th text, τ is a preset temperature hyperparameter, ∑ is the summation operation, exp is the exponential function, h j is the momentum feature of the j-th text, is the momentum set, h k is the momentum feature of the k-th text, and T represents the transpose of the vector.

[0036] Optionally, using the exponential moving average method to update the parameters of the momentum feature extractor includes:

[0037] Updating the parameters of the momentum feature extractor through the following formula:

[0038]

[0039] Among them, is the parameter of the (t + 1)-th update of the momentum feature extractor, α is a weighting coefficient with a value between 0 and 1, is the parameter of the t-th update of the momentum feature extractor, θ t+1 is the parameter of the (t + 1)-th update of the text feature extractor.

[0040] Optionally, clustering several optimized text features includes:

[0041] Using the KMeans clustering method to cluster several optimized text features.

[0042] In a second aspect of the present invention, there is provided a text fine-grained category discovery system based on noise reduction neighborhood aggregation, including:

[0043] A model update module, configured to iteratively update for a preset number of times and use the final text feature extractor as the optimized text feature extractor;

[0044] A fine-grained category determination module, configured to use the optimized text feature extractor to extract the features of each text in turn to obtain a number of optimized text features; cluster the number of optimized text features to obtain a number of clusters, and obtain the cluster labels of each cluster and use the cluster labels of each cluster as the fine-grained categories of each text in each cluster;

[0045] Wherein, the update steps include:

[0046] Extract the momentum features of all texts through the momentum feature extractor to obtain a momentum set; and use the text feature extractor to extract the features of each text in turn to obtain a number of text features;

[0047] Traverse each text feature, select and combine the momentum features with the top preset number of similarities to the current text feature from the momentum set to obtain the initial neighborhood set of each text feature; use the coarse-grained label constraint to filter the momentum features with different coarse-grained labels from the text feature in the initial neighborhood set to obtain the first filtered neighborhood set of each text feature; use the mutual neighborhood constraint to filter the momentum features that are not mutual neighbors of the text feature in the first filtered neighborhood set to obtain the second filtered neighborhood set of each text feature; use the rank statistic constraint to filter the momentum features with different n-order rank statistics of the text corresponding to the text feature in the second filtered neighborhood set to obtain the final neighborhood set of each text feature; wherein, n is the order of the rank statistic;

[0048] According to the momentum set and the final neighborhood set of each text feature, use multi-positive sample contrast learning to update the parameters of the text feature extractor and the parameters of the momentum feature extractor.

[0049] In a third aspect of the present invention, there is provided a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned text fine-grained category discovery method based on noise reduction neighborhood aggregation are implemented.

[0050] In a fourth aspect of the present invention, there is provided a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned text fine-grained category discovery method based on noise reduction neighborhood aggregation are implemented.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The text fine-grained category discovery method based on noise reduction neighborhood aggregation of the present invention optimizes the text feature extractor by performing an update step. According to the principle that texts with similar features have the same fine-grained category, in the update step, momentum features with different coarse-grained labels from the text features in the initial neighbor set are filtered using the coarse-grained label constraint, momentum features that are not mutual neighbors with the text features in the first-filtered neighbor set are filtered using the mutual neighbor constraint, and momentum features with different n-order rank statistics of the text corresponding to the text features in the second-filtered neighbor set are filtered using the rank statistic constraint. By using these three neighbor constraints, namely the coarse-grained label constraint, the mutual neighbor constraint, and the rank statistic constraint, a noise reduction neighbor set of the text features, that is, the final neighbor set, is obtained, and multi-positive sample contrast learning is used to make the text features and their neighbor momentum features have similar representations, effectively improving the accuracy of subsequent clustering to obtain the fine-grained categories of the texts. This method can effectively enhance the expression ability of the text feature extractor and has made great breakthroughs in the field of fine-grained category discovery. Brief Description of the Drawings

[0053] Figure 1 It is a flowchart of the text fine-grained category discovery method based on noise reduction neighborhood aggregation according to an embodiment of the present invention.

[0054] Figure 2 It is a schematic diagram of the principle of the text fine-grained category discovery method based on noise reduction neighborhood aggregation according to an embodiment of the present invention.

[0055] Figure 3 It is a block diagram of the text fine-grained category discovery system based on noise reduction neighborhood aggregation according to an embodiment of the present invention. Detailed Embodiment

[0056] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0057] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0058] The present invention will be further described in detail below with reference to the drawings:

[0059] See Figure 1 and 2 In an embodiment of the present invention, a method for discovering fine-grained text categories based on denoising neighborhood aggregation is provided, which can model the semantic relationships between different input features, thereby learning compact fine-grained category representations and enhancing the expressive power of the model.

[0060] Specifically, the method for discovering fine-grained text categories based on denoising neighborhood aggregation includes the following steps:

[0061] S1: Iteratively update the step a preset number of times, and use the final text feature extractor as the optimized text feature extractor.

[0062] S2: Use the optimized text feature extractor to extract the features of each text in turn to obtain a number of optimized text features; cluster the number of optimized text features to obtain a number of clusters, and obtain the cluster labels of each cluster and use the cluster labels of each cluster as the fine-grained categories of each text in each cluster.

[0063] Among them, the update step includes:

[0064] S11: Extract the momentum features of all texts through the momentum feature extractor to obtain a momentum set; and use the text feature extractor to extract the features of each text in turn to obtain a number of text features.

[0065] S12: Traverse each text feature, select and combine the momentum features with the top preset number of similarities to the current text feature from the momentum set to obtain the initial neighbor set for each text feature; use the coarse-grained label constraint to filter out the momentum features with different coarse-grained labels from the text feature in the initial neighbor set to obtain the first-filtered neighbor set for each text feature; use the mutual-neighbor constraint to filter out the momentum features that are not mutual neighbors with the text feature in the first-filtered neighbor set to obtain the second-filtered neighbor set for each text feature; use the rank statistic constraint to filter out the momentum features with different nth-order rank statistics of the text corresponding to the text feature in the second-filtered neighbor set to obtain the final neighbor set for each text feature; where n is the order of the rank statistic.

[0066] S13: According to the momentum set and the final neighbor set of each text feature, use multi-positive sample contrast learning to update the parameters of the text feature extractor and the parameters of the momentum feature extractor.

[0067] The initial text feature extractor and momentum feature extractor can both be the pre-trained language model BERT.

[0068] The method for discovering fine-grained text categories based on denoising neighborhood aggregation in the present invention optimizes the text feature extractor by performing an update step. According to the principle that texts with similar features have the same fine-grained category, in the update step, use the coarse-grained label constraint to filter out the momentum features with different coarse-grained labels from the text feature in the initial neighbor set, use the mutual-neighbor constraint to filter out the momentum features that are not mutual neighbors with the text feature in the first-filtered neighbor set, use the rank statistic constraint to filter out the momentum features with different nth-order rank statistics of the text corresponding to the text feature in the second-filtered neighbor set, use the three neighbor constraints of the coarse-grained label constraint, the mutual-neighbor constraint, and the rank statistic constraint to obtain the denoised neighbor set of the text feature, that is, the final neighbor set, and use multi-positive sample contrast learning to make the text feature and its neighbor momentum feature have similar representations, effectively improving the accuracy of subsequent clustering to obtain the fine-grained categories of the text. This method can effectively enhance the expression ability of the text feature extractor and has a great breakthrough in the field of fine-grained category discovery.

[0069] In a possible implementation, use the momentum feature extractor to extract momentum features for all texts and store the extracted momentum features in the momentum set where θ m is the parameter of the momentum feature extractor x l is the l-th input text, h l is the momentum feature of the text x l and N is the number of texts; use the text feature extractor F θ for the input text xi Extract features to obtain the text feature q of the input text i = F θ (x i ) and calculate the text feature q i Calculate the similarity between the text feature q and each momentum feature in the set Retrieve v momentum features with the highest similarities from them to form the initial neighbor set of the text feature q i of the text feature q where θ is the parameter of the text feature extractor F θ , and v is the number of initial neighbors for each text feature

[0070] Specifically, the specific operation of retrieving k momentum features with the highest similarities is as follows

[0071] For the given text feature q i = F θ (x i ), calculate the similarity between q i and each momentum feature h in the set l :

[0072]

[0073] where sim represents the cosine similarity function, |||| represents the L2 normalization operation, and T represents the transpose of the vector. After obtaining the similarities between the text feature and the momentum features in the set , select k momentum features with the highest similarities from as the initial neighbor set of the text feature q i of the text feature q

[0074]

[0075] where argtopv represents the v elements with the largest values, and v is the number of initial neighbors for each text feature, which is set to 120 in this embodiment

[0076] In a possible implementation, the filtering of the neighbor set includes three times, specifically

[0077] The first filtering of the neighbor set: Use the coarse-grained label constraint to filter out the momentum features in the initial neighbor set that have different coarse-grained labels from the text feature q i . Denote the neighbor set after the first filtering as the once-filtered neighbor set

[0078] Specifically, by comparing the text feature q i with its initial neighbor set The coarse-grained labels corresponding to the momentum features in and filter out those related to q in i Momentum features with different coarse-grained labels, only retain and those related to q in i Momentum features with the same coarse-grained label as the nearest neighbor set after the first filtering

[0079]

[0080] Among them, is the first-filtered nearest neighbor set of the text features of the i-th text, h j is the momentum feature of the j-th text, is the initial nearest neighbor set of the text features of the i-th text, c i is the coarse-grained label of the text features of the i-th text, c j is the coarse-grained label of the momentum feature of the j-th text. The ∈ symbol indicates that an element belongs to a certain set, and the ∧ symbol represents the intersection of two sets.

[0081] Second nearest neighbor set filtering: Use the mutual nearest neighbor constraint to filter out the momentum features in that are not mutual nearest neighbors with the text feature q i Denote the nearest neighbor set after the second filtering as the second-filtered nearest neighbor set

[0082] Specifically, by comparing the nearest neighbor relationship between the text feature q i and the momentum features in the nearest neighbor set after the first filtering filter out the momentum features in that are not mutual nearest neighbors with the momentum feature h i corresponding to q i Retain the momentum features in that are mutual nearest neighbors with the momentum feature h i corresponding to q i as the nearest neighbor set after the second filtering

[0083]

[0084] Among them, is the second-filtered nearest neighbor set of the text features of the i-th text, h i is the momentum feature of the i-th text, is the first-filtered nearest neighbor set of the text features of the j-th text.

[0085] Third nearest neighbor set filtering: Use the order statistic constraint to filter out the ones in that are related to the text xi The momentum features of the n-th order sorting statistic that are different, and the nearest neighbor set after the third filtering is denoted as the final nearest neighbor set

[0086] By comparing the text feature q i with its nearest neighbor set after the second filtering in terms of the n-th order sorting statistic of the momentum features, filter out the momentum features in that are different from the n-th order sorting statistic of q i , and only retain the momentum features in that are the same as the n-th order sorting statistic of q i as the final nearest neighbor set after the third filtering

[0087]

[0088] Among them, is the final nearest neighbor set of the text features of the i-th text, top n is the n-th order sorting statistic function, and the n-th order sorting statistic function is used to return the index numbers corresponding to the largest n-dimensional elements of the input features. In this embodiment, n is set to 5.

[0089] Among them, the n-th order sorting statistic refers to arranging the elements in the feature vector in descending order, selecting the largest n elements among them, and returning the index values of these elements in the original feature vector, that is, the positions of these elements in the original feature vector.

[0090] In a possible implementation manner, the updating the parameters of the text feature extractor and the parameters of the momentum feature extractor according to the momentum set and the final nearest neighbor sets of each text feature includes: using multi-positive sample contrast learning as the self-supervised loss function, traversing each text feature, taking all the momentum features in the final nearest neighbor set of the current text feature as positive texts, and taking all the momentum features in the momentum set as negative texts for contrast learning, using the backpropagation algorithm to update the parameters of the text feature extractor, and using the exponential moving average method to update the parameters of the momentum feature extractor.

[0091] Specifically, using multi-positive sample contrast learning as the self-supervised loss function, making the text feature q i and the momentum features in its nearest neighbor set as close as possible, and at the same time as far away as possible from other non-nearest neighbor momentum features in the set ; for the text features of other input texts, calculate the loss function value in the same way, and then use the backpropagation algorithm to continuously update the parameters θ of the text feature extractor F θ and use the exponential moving average method to update the momentum feature extractor parameter θ m 。

[0092] Among them, the self-supervised loss function is specifically:

[0093]

[0094] Among them, is the self-supervised loss function value, N is the number of texts, q i is the text feature of the i-th text, is the set composed of all text features, is the final nearest neighbor set of the text feature of the i-th text, is the number of momentum features included in the final nearest neighbor set of the text feature of the i-th text, τ is a preset temperature hyperparameter, which is 0.07 in this embodiment, ∑ is the summation operation, exp is the exponential function, h j is the momentum feature of the j-th text, is the momentum set, h k is the momentum feature of the k-th text.

[0095] According to the loss Update the parameters θ of the text feature extractor F through backpropagation θ and update the parameters of the momentum feature extractor using the exponential moving average method parameter θ m 。

[0096] Specifically, at the (t + 1)-th parameter update, first update the parameters of the text feature extractor F through backpropagation θ to obtain the parameter θ t+1 , then combine the parameter updated by the momentum feature extractor at the t-th time with the parameter θ t+1 for weighted combination to obtain the parameter of θ m updated at the (t + 1)-th time

[0097]

[0098] Among them, α is a weighting coefficient between 0 and 1, which is set to 0.99 in this embodiment.

[0099] In a possible implementation manner, the clustering of several optimized text features includes: clustering several optimized text features using the KMeans clustering method

[0100] Specifically, the fine-grained categories to which all texts belong are obtained using KMeans clustering: after the text feature extractor is updated, the re-extracted text feature set is used for unsupervised KMeans clustering to obtain the fine-grained categories to which each text belongs

[0101]

[0102] where KMeans is the unsupervised KMeans clustering function.

[0103] After clustering, each text is assigned to a different cluster, and the cluster label of the cluster to which each text belongs is used as the fine-grained category to which each text belongs.

[0104] In a possible implementation manner, three data sets in Table 1 are used to measure the specific performance of the method for discovering fine-grained text categories based on denoising neighborhood aggregation of the present invention.

[0105] Table 1

[0106]

[0107] where |C| represents the number of coarse-grained categories in the data set, represents the number of fine-grained categories in the data set, and #Train and #Test respectively represent the number of samples in the training set and the test set in the data set.

[0108] Referring to Table 2, the accuracy ACC, adjusted Rand index ARI, and normalized mutual information NMI of the method for discovering fine-grained text categories based on denoising neighborhood aggregation of the present invention and existing methods on different data sets are shown.

[0109] Table 2

[0110]

[0111]

[0112] where Improvement is the performance improvement value of the method proposed by the present invention compared to the previous optimal method. It can be seen that the method for discovering fine-grained text categories based on denoising neighborhood aggregation of the present invention has a large improvement in the application of different data sets.

[0113] The following is an apparatus embodiment of the present invention, which can be used to execute the method embodiment of the present invention. For details not disclosed in the apparatus embodiment, please refer to the method embodiment of the present invention.

[0114] See Figure 3, in another embodiment of the present invention, a text fine-grained category discovery system based on noise reduction neighborhood aggregation is provided, which can be used to implement the above-mentioned text fine-grained category discovery method based on noise reduction neighborhood aggregation. Specifically, the text fine-grained category discovery system based on noise reduction neighborhood aggregation includes a model update module and a fine-grained category determination module.

[0115] Among them, the model update module is used to iteratively update the preset number of times and use the final text feature extractor as the optimized text feature extractor; the fine-grained category determination module is used to use the optimized text feature extractor to extract the features of each text in turn to obtain a number of optimized text features; and cluster the number of optimized text features to obtain a number of clusters, and obtain the cluster labels of each cluster and use the cluster labels of each cluster as the fine-grained categories of each text in each cluster.

[0116] Among them, the update steps include: extracting the momentum features of all texts through the momentum feature extractor to obtain a momentum set; and using the text feature extractor to extract the features of each text in turn to obtain a number of text features; traversing each text feature, selecting the momentum features with the top preset number of similarities with the current text feature from the momentum set and combining them to obtain the initial neighbor set of each text feature; using the coarse-grained label constraint to filter the momentum features with different coarse-grained labels from the text feature in the initial neighbor set to obtain the first-filtered neighbor set of each text feature; using the mutual neighbor constraint to filter the momentum features that are not mutual neighbors with the text feature in the first-filtered neighbor set to obtain the second-filtered neighbor set of each text feature; using the rank statistic constraint to filter the momentum features with different nth-order rank statistics of the text corresponding to the text feature in the second-filtered neighbor set to obtain the final neighbor set of each text feature; where n is the order of the rank statistic; according to the momentum set and the final neighbor set of each text feature, use multi-positive sample contrast learning to update the parameters of the text feature extractor and the parameters of the momentum feature extractor.

[0117] All the relevant contents of each step involved in the embodiment of the foregoing text fine-grained category discovery method based on noise reduction neighborhood aggregation can be cited in the function description of the corresponding functional modules of the text fine-grained category discovery system in the embodiment of the present invention, and will not be repeated here.

[0118] The division of modules in the embodiments of the present invention is illustrative, only a logical function division. In actual implementation, there may be other division methods. In addition, in each embodiment of the present invention, each functional module can be integrated in one processor, or can exist physically alone, or two or more modules can be integrated in one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0119] In another embodiment of the present invention, a computer device is provided. The computer device includes a processor and a memory. The memory is used to store a computer program, and the computer program includes program instructions. The processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function. The processor described in the embodiment of the present invention can be used for the operation of the method for discovering fine-grained text categories based on noise reduction neighborhood aggregation.

[0120] In another embodiment of the present invention, a storage medium is also provided, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a computer device and is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and, of course, the extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space that stores the operating system of the terminal. And in this storage space, one or more instructions suitable for being loaded and executed by the processor are also stored. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the method for discovering fine-grained text categories based on noise reduction neighborhood aggregation in the above embodiments.

[0121] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0122] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0123] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realizes the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, such that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, so that the instructions executed on the computer or other programmable devices provide steps for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that it is still possible to modify the specific implementation manners of the present invention or make equivalent replacements. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A method for discovering fine-grained text categories based on noise reduction neighborhood aggregation, characterized in that Including: Iteratively update the preset number of steps, and use the final text feature extractor as the optimized text feature extractor; Use the optimized text feature extractor to extract the features of each text in turn, obtaining a number of optimized text features; Cluster the number of optimized text features to obtain a number of clusters, and obtain the cluster labels of each cluster and use the cluster labels of each cluster as the fine-grained categories of each text in each cluster; Among them, the update step includes: Extract the momentum features of all texts through the momentum feature extractor to obtain a momentum set; and use the text feature extractor to extract the features of each text in turn to obtain a number of text features; Traverse each text feature, select the momentum features with the top preset number of high similarities with the current text feature from the momentum set and combine them to obtain the initial neighbor set of each text feature; use the coarse-grained label constraint to filter out the momentum features with different coarse-grained labels from the text feature in the initial neighbor set to obtain the first-filtered neighbor set of each text feature; use the mutual neighbor constraint to filter out the momentum features that are not mutual neighbors with the text feature in the first-filtered neighbor set to obtain the second-filtered neighbor set of each text feature; use the rank statistic constraint to filter out the momentum features with different n-order rank statistics of the text corresponding to the text feature in the second-filtered neighbor set to obtain the final neighbor set of each text feature; where n is the order of the rank statistic; According to the momentum set and the final neighbor set of each text feature, use multi-positive sample contrast learning to update the parameters of the text feature extractor and the parameters of the momentum feature extractor.

2. The method for discovering fine-grained text categories based on noise reduction neighborhood aggregation according to claim 1, wherein The selecting the momentum features with the top preset number of high similarities with the current text feature from the momentum set includes: Obtain the similarity between the current text feature and each momentum feature in the momentum set through the following formula: where, sim(q i , h l ) is the similarity between q i and h l , sim represents the cosine similarity function, q i is the text feature of the i-th text, h l is the momentum feature of the l-th text, |||| represents the L2 normalization operation, and T represents the transpose of the vector; According to the similarity between the current text feature and each momentum feature in the momentum set, select the momentum features with the top preset number of high similarities with the current text feature from the momentum set.

3. The method for discovering fine-grained text categories based on noise reduction neighborhood aggregation according to claim 1, wherein The using the coarse-grained label constraint to filter out the momentum features with different coarse-grained labels from the text feature in the initial neighbor set includes: Filter out the momentum features with different coarse-grained labels from the text feature in the initial neighbor set through the following formula: Among them, is the set of one-time filtered neighbors of the text features of the i-th text, h j is the momentum feature of the j-th text, is the initial neighbor set of the text features of the i-th text, c i is the coarse-grained label of the text features of the i-th text, c j is the coarse-grained label of the momentum feature of the j-th text, the ∈ symbol indicates that an element belongs to a certain set, and the ∧ symbol indicates the intersection of two sets; The using the mutual neighbor constraint to filter out the momentum features that are not mutual neighbors with the text feature in the first-filtered neighbor set to obtain the second-filtered neighbor set of each text feature includes: Filter out the momentum features that are not mutual neighbors with the text feature in the first-filtered neighbor set through the following formula: Among them, is the set of second-filtered nearest neighbors of the text features of the i-th text, h i is the momentum feature of the i-th text, is the set of first-filtered nearest neighbors of the text features of the j-th text; The using the rank statistic constraint to filter out the momentum features with different n-order rank statistics of the text corresponding to the text feature in the second-filtered neighbor set includes: Filter out the momentum features with different n-order rank statistics of the text corresponding to the text feature in the second-filtered neighbor set through the following formula: Among them, is the final nearest neighbor set of the text features of the i-th text, top n is the n-th order sorting statistic function, and the n-th order sorting statistic function is used to return the index numbers corresponding to the n-dimensional elements with the largest input features.

4. The method for discovering fine-grained text categories based on noise reduction neighborhood aggregation according to claim 1, characterized in that The updating the parameters of the text feature extractor and the parameters of the momentum feature extractor according to the momentum set and the final neighbor set of each text feature includes: Multi-positive sample contrast learning is used as the self-supervised loss function. Each text feature is traversed, and all momentum features in the final nearest neighbor set of the current text feature are used as positive texts, and all momentum features in the momentum set are used as negative texts for contrast learning. The parameters of the text feature extractor are updated using the backpropagation algorithm, and the parameters of the momentum feature extractor are updated using the exponential moving average method.

5. The method for discovering text fine-grained categories based on noise reduction neighborhood aggregation according to claim 4, wherein The specific self-supervised loss function is as follows: Among them, is the self-supervised loss function value, N is the number of texts, q i is the text feature of the i-th text, is the set composed of all text features, is the final nearest neighbor set of the text feature of the i-th text, is the number of momentum features included in the final nearest neighbor set of the text feature of the i-th text, τ is a preset temperature hyperparameter, ∑ is the summation operation, exp is the exponential function, h j is the momentum feature of the j-th text, is the momentum set, h k is the momentum feature of the k-th text, and T represents the transpose of the vector.

6. The method for discovering fine-grained text categories based on noise reduction neighborhood aggregation according to claim 4, wherein The method of using the exponential moving average to update the parameters of the momentum feature extractor includes: The parameters of the momentum feature extractor are updated by the following formula: Among them, is the parameter updated for the (t + 1)-th time of the momentum feature extractor, α is a weighting coefficient with a value between 0 and 1, is the parameter updated for the t-th time of the momentum feature extractor, θ t+1 is the parameter updated for the (t + 1)-th time of the text feature extractor.

7. The method for discovering fine-grained text categories based on noise reduction neighborhood aggregation according to claim 1, wherein The clustering of several optimized text features includes: The KMeans clustering method is used to cluster several optimized text features.

8. A text fine-grained category discovery system based on noise reduction neighborhood aggregation, characterized in that It includes: A model update module, which is used to iterate and update for a preset number of times, and use the final text feature extractor as the optimized text feature extractor; A fine-grained category determination module, which is used to use the optimized text feature extractor to extract the features of each text in turn to obtain several optimized text features; Several optimized text features are clustered to obtain several clusters, and the cluster labels of each cluster are obtained and used as the fine-grained categories of each text in each cluster; Among them, the update steps include: The momentum features of all texts are extracted through the momentum feature extractor to obtain a momentum set; and the features of each text are extracted in turn using the text feature extractor to obtain several text features; Each text feature is traversed, and the momentum features with the top preset number of similarities to the current text feature are selected from the momentum set and combined to obtain the initial nearest neighbor set of each text feature; the momentum features with different coarse-grained labels from the text feature are filtered out from the initial nearest neighbor set using the coarse-grained label constraint to obtain the first filtered nearest neighbor set of each text feature; the momentum features that are not mutual nearest neighbors with the text feature are filtered out from the first filtered nearest neighbor set using the mutual nearest neighbor constraint to obtain the second filtered nearest neighbor set of each text feature; the momentum features with different nth-order rank statistics of the text corresponding to the text feature are filtered out from the second filtered nearest neighbor set using the rank statistic constraint to obtain the final nearest neighbor set of each text feature; where n is the order of the rank statistic; According to the momentum set and the final nearest neighbor set of each text feature, the parameters of the text feature extractor and the parameters of the momentum feature extractor are updated using multi-positive sample contrast learning.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method for discovering fine-grained text categories based on noise reduction neighborhood aggregation according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method for discovering fine-grained text categories based on noise reduction neighborhood aggregation according to any one of claims 1 to 7 are implemented.

Images