Streaming news clustering method and apparatus, and computer device

Abstract

The present disclosure relates to the technical field of data analysis, and specifically discloses a streaming news clustering method and device and computer equipment. The method comprises: receiving news to be clustered and extracting the news title of the news to be clustered; representing the news title of the news to be clustered as a title vector through a title representation model; the title representation model is obtained through supervised contrast learning training based on a plurality of news title samples; searching for a candidate vector having an associated relationship with the title vector in a preset title vector library according to the title vector; and clustering according to the title vector and the candidate vector to obtain a clustering result. When the news title is vectorized, the title representation model trained through supervised contrast learning is used for vectorization output, which greatly improves the accuracy of semantic representation of the news title and further improves the accuracy of streaming news clustering.

Description

Technical Field

[0001] This disclosure relates to the field of data analysis technology, and in particular to a streaming news clustering method, apparatus, and computer equipment. Background Technology

[0002] With the rapid development and widespread adoption of internet technology, more and more users are accessing the latest information online. News is a major form of information resource on websites. Faced with a massive amount of streaming news, users often need to find the information they need. However, the scattered and redundant information in large volumes of streaming news makes it more difficult for users to find the information they need.

[0003] In related technologies, text clustering can be used to cluster streaming news to reduce the display of redundant news to users. However, current clustering algorithms are often trained based on common, publicly known information, which limits the accuracy of news clustering. Summary of the Invention

[0004] Therefore, it is necessary to provide a streaming news clustering method, apparatus, computer equipment, storage medium, and computer program product to address the aforementioned technical problems.

[0005] Firstly, this disclosure provides a streaming news clustering method. The method includes:

[0006] Receive news articles to be clustered and extract their titles;

[0007] The news titles of the news to be clustered are represented as title vectors using a title representation model; the title representation model is obtained through supervised contrastive learning training based on several news title samples.

[0008] Based on the title vector, retrieve candidate vectors that are related to the title vector from a preset title vector library;

[0009] Clustering is performed based on the title vector and candidate vectors to obtain the clustering results.

[0010] In one embodiment, the step of obtaining the title representation model includes:

[0011] A pre-trained language model is built based on news headline samples, and the pre-trained language model is used to represent the news headline samples as sample vectors.

[0012] Self-supervised contrastive learning is performed on the pre-trained language model based on the sample vectors to obtain a self-supervised language model.

[0013] Enhanced positive samples and enhanced negative samples are generated based on the news headline samples;

[0014] The title representation model is obtained by performing supervised comparative learning on the self-supervised language model based on the enhanced positive and enhanced negative samples.

[0015] In one embodiment, the step of performing self-supervised contrastive learning on the pre-trained language model based on sample vectors to obtain a self-supervised language model includes:

[0016] Self-supervised comparative learning is performed using the pre-trained language model output of the target sample and the pre-trained language model output of the target sample after random discarding as positive samples, and the pre-trained language model output of the target sample and the pre-trained language model output of the non-target sample in the news headline sample as negative samples; the target sample is any sample in the news headline sample.

[0017] In one embodiment, generating enhanced positive and enhanced negative samples based on the news headline samples includes:

[0018] Identify key fields in the news headline sample, and replace the key fields to generate the enhanced positive sample and enhanced negative sample; the key fields include one or more of the noun field, verb field, number field and time field in the news headline sample.

[0019] In one embodiment, replacing the key field to generate the enhanced positive and enhanced negative samples includes:

[0020] The target object name in the noun field of the news headline sample is replaced with a non-target object name to generate the enhanced positive sample or the enhanced negative sample;

[0021] The verbs in the verb field of the news headline sample are replaced with synonyms or antonyms to generate the enhanced positive sample or the enhanced negative sample.

[0022] In one embodiment, the step of performing supervised contrastive learning on the self-supervised language model based on the enhanced positive samples and enhanced negative samples to obtain the title representation model includes:

[0023] Supervised contrastive learning is performed by using the self-supervised language model output of the target sample and the self-supervised language model output of the enhanced positive sample as positive samples, the self-supervised language model output of the target sample and the self-supervised language model output of the enhanced negative sample as negative samples, and the self-supervised language model output of the target sample and the self-supervised language model output of the non-target sample in the news headline sample as negative samples.

[0024] In one embodiment, the title vector library includes the news title samples through the output of the title representation model and / or the news titles of the news to be clustered through the output of the title representation model.

[0025] Secondly, this disclosure also provides a streaming news clustering device. The device includes:

[0026] The title extraction module is used to receive news articles to be clustered and extract their titles.

[0027] The title representation module is used to represent the news titles of the news to be clustered as title vectors through a title representation model; the title representation model is obtained by supervised contrastive learning training based on several news title samples;

[0028] The candidate title module is used to retrieve candidate vectors that are related to the title vector from a preset title vector library based on the title vector;

[0029] The title clustering module is used to perform clustering based on the title vector and candidate vectors to obtain clustering results.

[0030] In one embodiment, the apparatus further includes a model acquisition module, the model acquisition module comprising:

[0031] A pre-trained language model unit is used to build a pre-trained language model based on news headline samples, wherein the pre-trained language model is used to represent the news headline samples as sample vectors.

[0032] A self-supervised contrastive learning unit is used to perform self-supervised contrastive learning on the pre-trained language model based on sample vectors to obtain a self-supervised language model.

[0033] An enhanced sample unit is used to generate enhanced positive samples and enhanced negative samples based on the news headline samples;

[0034] A supervised contrastive learning unit is used to perform supervised contrastive learning on the self-supervised language model based on the enhanced positive samples and enhanced negative samples to obtain the title representation model.

[0035] In one embodiment, the self-supervised contrastive learning unit is further configured to perform self-supervised contrastive learning by using the pre-trained language model output of the target sample and the pre-trained language model output of the target sample after random discarding as positive samples, and using the pre-trained language model output of the target sample and the pre-trained language model output of the non-target sample in the news headline sample as negative samples; the target sample is any sample in the news headline sample.

[0036] In one embodiment, the enhanced sample unit is further configured to identify key fields in the news headline sample, and replace the key fields to generate the enhanced positive sample and the enhanced negative sample; the key fields include one or more of the noun field, verb field, number field and time field in the news headline sample.

[0037] In one embodiment, the enhanced sample unit includes:

[0038] The target object name subunit is used to replace the target object name in the noun field of the news headline sample with a non-target object name to generate the enhanced positive sample or the enhanced negative sample.

[0039] The verb subunit is used to replace the verbs in the verb field of the news headline sample with synonyms or antonyms to generate the enhanced positive sample or the enhanced negative sample.

[0040] In one embodiment, the supervised contrastive learning unit is further configured to perform supervised contrastive learning by using the self-supervised language model output of the target sample and the self-supervised language model output of the enhanced positive sample as positive samples, the self-supervised language model output of the target sample and the self-supervised language model output of the enhanced negative sample as negative samples, and the self-supervised language model output of the target sample and the self-supervised language model output of the non-target sample in the news headline sample as negative samples.

[0041] In one embodiment, the title vector library includes the news title samples through the output of the title representation model and / or the news titles of the news to be clustered through the output of the title representation model.

[0042] Thirdly, this disclosure also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described streaming news clustering method.

[0043] Fourthly, this disclosure also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described streaming news clustering method.

[0044] Fifthly, this disclosure also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described streaming news clustering method.

[0045] The aforementioned streaming news clustering methods, apparatus, computer equipment, storage media, and computer program products have at least the following beneficial effects:

[0046] This disclosure achieves the purpose of classifying and labeling news to be clustered by extracting and vectorizing the titles of incoming news to be clustered, and then performing vector clustering to obtain clustering results. At the same time, when vectorizing the news titles, the headline representation model trained by supervised contrastive learning is used for vectorization output, which greatly improves the accuracy of the semantic representation of news titles, and thus improves the accuracy of streaming news clustering. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments or conventional technologies of this disclosure, the accompanying drawings used in the description of the embodiments or conventional technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a diagram illustrating the application environment of a streaming news clustering method in one embodiment.

[0049] Figure 2 This is a flowchart illustrating a streaming news clustering method in one embodiment;

[0050] Figure 3 This is a flowchart illustrating the steps for obtaining the title representation model in one embodiment;

[0051] Figure 4 This is a flowchart illustrating the steps for obtaining the title representation model in another embodiment;

[0052] Figure 5 This is a structural block diagram of a streaming news clustering device in one embodiment;

[0053] Figure 6 This is a structural block diagram of a streaming news clustering device in another embodiment;

[0054] Figure 7 This is a structural block diagram of a streaming news clustering device in another embodiment;

[0055] Figure 8 This is a block diagram of the internal structure of a computer device in one embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure.

[0058] The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0059] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising / including” or “having,” etc., specify the presence of the stated features, wholes, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. Meanwhile, in this specification, the term “and / or” includes any and all combinations of the associated listed items.

[0060] The streaming news clustering method provided in this disclosure can be applied to, for example, Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. Server 104 can cluster streaming news generated by the network, respond to user news query operations through terminal 102, and display the clustered news through terminal 102 based on the news query operation. Specifically, server 104 is configured with a title representation model 106, which can vectorize news titles. Server 104 also includes a vector retrieval engine module 108 and a clustering module 110. Server 104 uses the vector retrieval engine module 108 to retrieve similar news titles from the title vector library 112, and then uses the clustering module 110 to perform clustering logic on the retrieved similar news titles to obtain the clustering results. The title vector library 112 can store a large number of clustered news titles and their corresponding vectors. The title vector library 112 can be integrated on server 104 or placed on the cloud or other network servers. The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0061] In some embodiments of this disclosure, such as Figure 2 As shown, a streaming news clustering method is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0062] Step 210: Receive the news to be clustered and extract the news titles of the news to be clustered.

[0063] News awaiting clustering can refer to news items that enter the server without undergoing clustering to generate news tags. News items awaiting clustering can flow into the server one by one or in batches for clustering.

[0064] For example, the server receives news articles to be clustered from the network and extracts their titles. Optionally, Python web scraping techniques can be used to extract the news titles.

[0065] Step 220: Represent the news titles of the news to be clustered as title vectors using a title representation model; the title representation model is obtained by supervised contrastive learning training based on several news title samples.

[0066] For example, the server inputs the extracted news titles into a pre-built title representation model, and obtains the title vectors output by the title representation model. The title representation model can be a pre-built language model used to represent the news titles of the news to be clustered as title vectors, specifically obtained through supervised contrastive learning training using several news title samples.

[0067] Step 230: Based on the title vector, retrieve candidate vectors that are related to the title vector from a preset title vector library.

[0068] For example, the server retrieves candidate vectors similar to the title vector from a title vector library. Candidate vectors that are associated with the title vector are considered; this association can be used to represent semantic similarity, specifically determined by the cosine distance between the title vectors. The title vector library can store a large number of news titles and their corresponding title vectors.

[0069] Optionally, after obtaining the title vector, the server can calculate the cosine distance between the title vector and the title vector. Then, it can retrieve candidate vectors from the title vector database whose cosine distance to the title vector falls within a certain range. For example, the server can use the open-source Vearch (a distributed vector search system) to calculate vector similarity, allowing it to search for vectors using vectors. It can return the top-k candidate vectors most similar to the title vector; the top-k is generally understood as the K results with the highest vector similarity among the recalled vectors.

[0070] Step 240: Cluster the data based on the title vector and candidate vectors to obtain the clustering results.

[0071] For example, the server performs clustering logic based on the title vector and candidate vectors. A classic one-pass clustering algorithm can be used to cluster similar news titles into a single class, yielding the clustering results. The clustering results must include at least the news category label to which the news to be clustered belongs.

[0072] In the above-mentioned streaming news clustering method, the clustering results are obtained by extracting and vectorizing the titles of the incoming news to be clustered, and then performing vector clustering. This achieves the purpose of classifying and labeling the news to be clustered. At the same time, when vectorizing the news titles, the headline representation model trained by supervised contrastive learning is used to output the vectorized headlines, which greatly improves the accuracy of the semantic representation of the news headlines and thus improves the accuracy of streaming news clustering.

[0073] In some embodiments of this disclosure, such as Figure 3 As shown, the steps for obtaining the title representation model include:

[0074] Step 310: Establish a pre-trained language model based on the news headline samples. The pre-trained language model is used to represent the news headline samples as sample vectors.

[0075] For example, the server pre-collects a large number of news headline samples and builds a pre-trained language model based on the collected news headline samples. The news headline samples may include, but are not limited to, one or more of the following: historical news headlines obtained by the server, titles of publicly available news collections, and news headlines selected based on business needs. The pre-trained language model can be a neural network training algorithm based on natural language processing technology. Optionally, it may specifically adopt the BERT model (Pre-training of Deep Bidirectional Transformers for Language Understanding), the GPT model (Improving Language Understanding by Generative Pre-Training (2018)), the ELMo model (Deep contextualized word representations (ACL 2018)), etc.

[0076] Step 320: Perform self-supervised contrastive learning on the pre-trained language model based on the sample vectors to obtain a self-supervised language model.

[0077] For example, after obtaining a pre-trained language model, self-supervised contrastive learning is performed on it. Self-supervised contrastive learning can train on a large number of unlabeled sample vectors, using the sample vectors themselves as supervision information to learn the feature representations of the sample vectors. After being trained by self-supervised contrastive learning, the accuracy of the pre-trained language model is improved, resulting in a self-supervised language model.

[0078] Step 330: Generate enhanced positive samples and enhanced negative samples based on the news headline samples.

[0079] For example, the server can generate samples that are semantically similar to or semantically opposite to existing news headline samples, serving as enhanced positive and enhanced negative samples. Optionally, enhanced positive and enhanced negative samples can be generated and entered manually.

[0080] Step 340: Based on the enhanced positive samples and enhanced negative samples, perform supervised comparative learning on the self-supervised language model to obtain the title representation model.

[0081] For example, after obtaining the self-supervised language model, supervised contrastive learning is performed on it. Self-supervised contrastive learning is essentially unsupervised contrastive learning, while supervised contrastive learning can narrow the cosine similarity distance of title samples based on generated enhanced positive samples and widen the cosine similarity distance based on generated enhanced negative samples. After being trained by supervised contrastive learning, the accuracy of the self-supervised language model is improved, resulting in the final title representation model.

[0082] In some embodiments of this disclosure, the server builds a pre-trained language model using sample vectors, and performs self-supervised contrastive learning training and supervised contrastive learning training on the pre-trained language model to finally obtain a title representation model, which further improves the accuracy of news title semantic representation and thus improves the accuracy of streaming news clustering.

[0083] In some embodiments of this disclosure, step 320 further includes:

[0084] Self-supervised comparative learning is performed using the pre-trained language model output of the target sample and the pre-trained language model output of the target sample after random discarding as positive samples, and the pre-trained language model output of the target sample and the pre-trained language model output of the non-target sample in the news headline sample as negative samples; the target sample is any sample in the news headline sample.

[0085] For example, when the server performs self-supervised contrastive learning on the pre-trained language model, it selects any sample from the news headline samples as the target sample x. The output of the pre-trained language model for the target sample is denoted by ptm(x), and the output of the pre-trained language model after a random dropout operation on the target sample is denoted by dropout(ptm(x)). Dropout is a random dropout operation typically used in neural networks, where the activation value of a neuron is stopped with a certain probability during forward propagation. In the self-supervised contrastive learning training process, ptm(x) and dropout(ptm(x)) are considered positive samples, and ptm(x) and the outputs of the pre-trained language model for other samples (i.e., non-target samples) in a batch of news headline samples are considered negative samples. At this time, the input of the pre-trained language model is a tuple (x, x).

[0086] This embodiment achieves self-supervised comparative learning training of the pre-trained language model by using the target sample itself and the samples after random discarding as positive samples, and using the target sample itself and other samples in the same batch as negative samples.

[0087] In some embodiments of this disclosure, step 330 further includes:

[0088] Identify key fields in the news headline sample, and replace the key fields to generate the enhanced positive sample and enhanced negative sample; the key fields include one or more of the noun field, verb field, number field and time field in the news headline sample.

[0089] For example, the server can identify news headline samples and extract key fields from them. Optionally, it can utilize the NER (Named Entity Recognition) interface for identification. Key fields can include one or more of noun, verb, numeric, and time fields; for example, it can identify names of people, places, organizations, dates and times, proper nouns, etc. The server can process the identified key fields by manual replacement or by using a near / antonym replacement tool to obtain enhanced positive and enhanced negative samples. Enhanced positive samples can refer to positive samples generated by the server that are not from the original news headline samples, and enhanced negative samples can refer to negative samples generated by the server that are not from the original news headline samples.

[0090] This embodiment identifies key fields in news headline samples and generates enhanced positive and enhanced negative samples by replacing key fields. These enhanced positive and enhanced negative samples are then used as positive and negative samples in supervised contrastive learning to further optimize the self-supervised language model.

[0091] In some embodiments of this disclosure, such as Figure 4 As shown, the step of replacing the key field to generate the enhanced positive and enhanced negative samples further includes:

[0092] Step 332: Replace the target object name in the noun field of the news headline sample with a non-target object name to generate the enhanced positive sample or the enhanced negative sample.

[0093] The target object can be an entity with specific meaning or strong referentiality, such as a company, a person's name, or a place name. Non-target object names can include aliases of the target object or names of other entities similar to the target object.

[0094] For example, the server can identify "Company A" as the target in a news headline sample "Company A has gone public". It can replace "Company A" in the news headline sample with "Company B" or / and "Company C" to generate an enhanced negative sample. Company B and Company C are not companies of Company A, but can be A's partner companies or competitors.

[0095] The server can also replace "Company A" in the news headline sample with "Company AA" to generate enhanced positive samples, where Company AA is an alternative name for Company A, and is essentially the same company as Company A.

[0096] Step 334: Replace the verbs in the verb field of the news headline sample with synonyms or antonyms to generate the enhanced positive sample or the enhanced negative sample.

[0097] For example, the server can identify the verb field "listed" in a news headline sample "Company A went public" and replace the verb "listed" in the verb field with the synonym "entered the market" to generate an enhanced positive sample.

[0098] The server can also replace the verb "listed" in the verb field with the antonym "withdrawn" to generate enhanced negative samples.

[0099] This embodiment generates enhanced positive and enhanced negative samples by replacing the target object name and verbs in key fields, which can further improve the accuracy of title vectorization, greatly reduce the problem of title similarity not meeting actual needs caused by easily confused titles, improve the accuracy of news title semantic representation, and thus improve the accuracy of streaming news clustering.

[0100] In some embodiments of this disclosure, step 340 includes:

[0101] Supervised contrastive learning is performed by using the self-supervised language model output of the target sample and the self-supervised language model output of the enhanced positive sample as positive samples, the self-supervised language model output of the target sample and the self-supervised language model output of the enhanced negative sample as negative samples, and the self-supervised language model output of the target sample and the self-supervised language model output of the non-target sample in the news headline sample as negative samples.

[0102] For example, when the server performs supervised contrastive learning on the pre-trained language model, it selects any sample from the news headline samples as the target sample x, x_positive is an enhanced positive sample generated based on the target sample, x_negative is an enhanced negative sample generated based on the target sample, and the output of the self-supervised language model is represented by ptm'().

[0103] The output of the self-supervised language model for the target sample is denoted by ptm'(x), the output of the self-supervised language model for enhancing positive samples is denoted by ptm'(x_positive), and the output of the self-supervised language model for enhancing negative samples is denoted by ptm'(x_negative). In the supervised contrastive learning training process, ptm'(x) and ptm'(x_positive) are considered positive samples, ptm'(x) and ptm'(x_negative) are considered negative samples, and the outputs of the self-supervised language model for ptm'(x) and other samples (i.e., non-target samples) in a batch of news headlines are considered negative samples. At this point, the input to the self-supervised language model is a triple (x, x_positive, _negative).

[0104] This embodiment achieves supervised comparative learning training of the self-supervised language model by using the target sample and its own enhanced positive sample as positive samples, the target sample and its own enhanced negative sample as negative samples, and the target sample itself and other samples in the same batch as negative samples.

[0105] In some embodiments of this disclosure, the title vector library includes the news title samples as output by the title representation model and / or the news titles of the news to be clustered as output by the title representation model.

[0106] For example, after training and obtaining the title representation model, the server can add news title samples to the title vector library through the output of the title representation model. The server can also add the news titles of the news to be clustered to the title vector library through the output of the title representation model after the news to be clustered has been clustered.

[0107] This embodiment adds news headline samples to the headline representation model and / or news headlines of the news to be clustered to the headline representation model to the headline vector library, which can update and expand the headline vector library in a timely manner.

[0108] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0109] Based on the same inventive concept, this disclosure also provides a streaming news clustering apparatus for implementing the streaming news clustering method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more streaming news clustering apparatus embodiments provided below can be found in the limitations of the streaming news clustering method described above, and will not be repeated here.

[0110] In some embodiments of this disclosure, such as Figure 5 As shown, a streaming news clustering device is provided.

[0111] The device 500 includes:

[0112] The title extraction module 510 is used to receive news to be clustered and extract the news titles of the news to be clustered.

[0113] The title representation module 520 is used to represent the news titles of the news to be clustered as title vectors through a title representation model; the title representation model is obtained by supervised contrastive learning training based on several news title samples;

[0114] The candidate title module 530 is used to retrieve candidate vectors that are related to the title vector from a preset title vector library based on the title vector;

[0115] The title clustering module 540 is used to perform clustering based on the title vector and candidate vectors to obtain clustering results.

[0116] In some embodiments of this disclosure, such as Figure 6 As shown, the device 500 further includes a model acquisition module 550, which includes:

[0117] The pre-trained language model unit 552 is used to build a pre-trained language model based on news headline samples, wherein the pre-trained language model is used to represent the news headline samples as sample vectors.

[0118] Self-supervised contrastive learning unit 554 is used to perform self-supervised contrastive learning on the pre-trained language model based on sample vectors to obtain a self-supervised language model.

[0119] Enhancement sample unit 556 is used to generate enhanced positive samples and enhanced negative samples based on the news headline samples;

[0120] The supervised contrastive learning unit 558 is used to perform supervised contrastive learning on the self-supervised language model based on the enhanced positive samples and enhanced negative samples to obtain the title representation model.

[0121] In some embodiments of this disclosure, the self-supervised contrastive learning unit is further configured to perform self-supervised contrastive learning by using the pre-trained language model output of the target sample and the pre-trained language model output of the target sample after random discarding as positive samples, and using the pre-trained language model output of the target sample and the pre-trained language model output of the non-target sample in the news headline sample as negative samples; the target sample is any sample in the news headline sample.

[0122] In some embodiments of this disclosure, the enhanced sample unit is further configured to identify key fields in the news headline sample, and replace the key fields to generate the enhanced positive sample and the enhanced negative sample; the key fields include one or more of the noun field, verb field, number field and time field in the news headline sample.

[0123] In some embodiments of this disclosure, such as Figure 7 As shown, the enhanced sample unit 556 includes:

[0124] The target object name subunit 5562 is used to replace the target object name in the noun field of the news title sample with a non-target object name to generate the enhanced positive sample or the enhanced negative sample.

[0125] The verb subunit 5564 is used to replace the verbs in the verb field of the news headline sample with synonyms or antonyms to generate the enhanced positive sample or the enhanced negative sample.

[0126] In some embodiments of this disclosure, the supervised contrastive learning unit is further configured to perform supervised contrastive learning by using the self-supervised language model output of the target sample and the self-supervised language model output of the enhanced positive sample as positive samples, the self-supervised language model output of the target sample and the self-supervised language model output of the enhanced negative sample as negative samples, and the self-supervised language model output of the target sample and the self-supervised language model output of the non-target sample in the news headline sample as negative samples.

[0127] In some embodiments of this disclosure, the title vector library includes the news title samples as output by the title representation model and / or the news titles of the news to be clustered as output by the title representation model.

[0128] Each module in the aforementioned streaming news clustering device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module. It should be noted that the module division in this embodiment is illustrative and only represents a logical functional division; in actual implementation, other division methods may be used.

[0129] Based on the aforementioned embodiments of the streaming news clustering method, in another embodiment provided in this disclosure, a computer device is provided. This computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a streaming news clustering method.

[0130] Those skilled in the art will understand that the structure shown in Figure 000 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0131] Based on the aforementioned embodiments of the streaming news clustering method, in another embodiment provided in this disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0132] Based on the embodiments described above for streaming news clustering methods, in another embodiment provided in this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0134] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0135] In the description of this specification, references to terms such as "some embodiments," "other embodiments," and "ideal embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative descriptions of the above terms do not necessarily refer to the same embodiments or examples.

[0136] It is understood that the various embodiments of the methods described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. Relevant details can be found in the descriptions of other method embodiments.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features of the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the scope of protection of this disclosure. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A streaming news clustering method, characterized in that, The method includes: Receive news articles to be clustered and extract their titles; The news titles of the news to be clustered are represented as title vectors using a title representation model; the title representation model is obtained through supervised contrastive learning training based on several news title samples. Based on the title vector, retrieve candidate vectors that are related to the title vector from a preset title vector library; Clustering is performed based on the title vector and candidate vectors to obtain the clustering results; The steps for obtaining the title representation model include: establishing a pre-trained language model based on news title samples, wherein the pre-trained language model is used to represent news title samples as sample vectors; performing self-supervised contrastive learning on the pre-trained language model based on the sample vectors to obtain a self-supervised language model; generating enhanced positive samples and enhanced negative samples based on the news title samples, wherein the output of the self-supervised language model of the target sample and the output of the self-supervised language model of the enhanced positive sample are mutually positive samples, the output of the self-supervised language model of the target sample and the output of the self-supervised language model of the enhanced negative sample are mutually negative samples, and the output of the self-supervised language model of the target sample and the output of the self-supervised language model of the non-target sample in the news title samples are mutually negative samples, and performing supervised contrastive learning; wherein the target sample is any sample in the news title samples; The training process of the title representation model includes: Identify key fields in the news headline samples, and replace the key fields to generate the enhanced positive samples and enhanced negative samples; The key fields include one or more of the noun field, verb field, number field, and time field in the news title sample; Self-supervised comparative learning is performed by using the pre-trained language model output of the target sample and the pre-trained language model output of the target sample after random discarding as positive samples, and using the pre-trained language model output of the target sample and the pre-trained language model output of the non-target samples in the news headline samples as negative samples.

2. The method according to claim 1, characterized in that, The step of generating enhanced positive and enhanced negative samples based on the news headline samples includes: Identify key fields in the news headline sample, and replace the key fields to generate the enhanced positive sample and enhanced negative sample; the key fields include one or more of the noun field, verb field, number field and time field in the news headline sample.

3. The method according to claim 2, characterized in that, The step of replacing the key fields to generate the enhanced positive and enhanced negative samples includes: The target object name in the noun field of the news headline sample is replaced with a non-target object name to generate the enhanced positive sample or the enhanced negative sample; The verbs in the verb field of the news headline sample are replaced with synonyms or antonyms to generate the enhanced positive sample or the enhanced negative sample.

4. The method according to claim 1, characterized in that, The title vector library includes the news title samples output by the title representation model and / or the news titles of the news to be clustered output by the title representation model.

5. A streaming news clustering device, characterized in that, The device includes: The title extraction module is used to receive news articles to be clustered and extract their titles. A title representation module is used to represent the news titles of the news to be clustered as title vectors using a title representation model. The title representation model is obtained through supervised contrastive learning training based on several news title samples. The steps for obtaining the title representation model include: establishing a pre-trained language model based on the news title samples, the pre-trained language model being used to represent the news title samples as sample vectors; performing self-supervised contrastive learning on the pre-trained language model based on the sample vectors to obtain a self-supervised language model; generating enhanced positive samples and enhanced negative samples based on the news title samples, with the output of the self-supervised language model of the target sample and the output of the self-supervised language model of the enhanced positive sample being considered positive samples, and the output of the self-supervised language model of the target sample and the output of the self-supervised language model of the enhanced negative sample being considered negative samples. The language model output and the self-supervised language model output of non-target samples in the news headline samples are used as negative samples for supervised contrastive learning. The training process of the headline representation model includes: identifying key fields in the news headline samples, replacing the key fields to generate enhanced positive and enhanced negative samples; the key fields include one or more of noun fields, verb fields, number fields, and time fields in the news headline samples; the pre-trained language model output of the target sample and the pre-trained language model output of the target sample after random discarding are used as positive samples, and the pre-trained language model output of the target sample and the pre-trained language model output of non-target samples in the news headline samples are used as negative samples for supervised contrastive learning; the target sample is any sample in the news headline samples. The candidate title module is used to retrieve candidate vectors that are related to the title vector from a preset title vector library based on the title vector; The title clustering module is used to perform clustering based on the title vector and candidate vectors to obtain clustering results.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images