News content core-oriented labeling method, equipment and medium

Abstract

The invention discloses a news content core-oriented labeling method and device and a medium, and relates to the technical field of electric digital data processing. The method comprises the following steps: dividing a preset news text data set into a labeled text example pool and an unlabeled text data set; performing preliminary labeling on the unlabeled text data set to obtain a weakly labeled text data set; screening a target confidence sample from the weakly labeled text data set, and generating a target sample set with a target pseudo label based on the target confidence sample through a preset screening strategy; according to the semantic neighbor sample, constructing a dialogue prompt template so as to output a refined pseudo tag corresponding to the weakly labeled text data set through the dialogue prompt template based on a preset macBERT model; and inputting the target sample set and the to-be-refined text as a training set into the macBERT model for training to obtain a trained text classification model, and outputting a core oriented label of the to-be-labeled news through the text classification model.

Description

Technical Field

[0001] The present application relates to the technical field of electronic digital data processing, and in particular to a news content core-oriented marking method, device and medium. Background Art

[0002] In the current data-driven era of artificial intelligence, data annotation plays a vital role in various fields as a key means of building high-quality datasets. Accurately expressing the core message of news reporting is crucial. Precisely extracting and structurally annotating the core message of news texts can help users quickly grasp the core message of a report from a vast amount of news, reducing the difficulty of obtaining relevant information and providing strong support for online content governance.

[0003] However, traditional annotation methods mainly rely on manual operations and adopt a multi-stage output process. When dealing with complex scenarios such as extensive news content, strong context dependence, implicit directional expression and cross-dimensionality, it is easy to lead to the accumulation and poor transmission of errors in each stage. For example, inaccurate identification of key trigger words will affect the development of subsequent annotation tasks, and the various stages fail to share a unified model for collaborative optimization, making it difficult to ensure the consistency and accuracy of the overall annotation. Summary of the Invention

[0004] To solve the above problems, this application proposes a news content core-oriented annotation method, including: Based on the text examples with manual annotation core guidance, the preset news text dataset is divided into an annotated text example pool and an unannotated text dataset; The news text dataset is converted into an embedding vector, and the unlabeled text dataset is preliminarily labeled based on the semantic similarity between the first embedding vector corresponding to the labeled text example pool and the second embedding vector corresponding to the unlabeled text dataset using a preset large model to obtain a weakly labeled text dataset; Based on the cross entropy loss and Gaussian mixture model, target confidence samples are screened from the weakly annotated text dataset, and a target sample set with target pseudo labels is generated based on the target confidence samples through a preset screening strategy; Determining text to be refined that is not included in the target sample set in the weakly annotated text dataset, retrieving semantic neighbor samples of the text to be refined from the annotated text example pool, and constructing a dialogue prompt template based on the semantic neighbor samples, so as to output refined pseudo-labels corresponding to the weakly annotated text dataset through the dialogue prompt template based on a preset macBERT model; The target sample set and the text to be refined are input as training sets into the macBERT model for training to obtain a trained text classification model, and the core-oriented annotations of the news to be annotated are output through the text classification model.

[0005] In one implementation of the present application, the unlabeled text dataset is preliminarily labeled based on the semantic similarity between the first embedding vector corresponding to the labeled text example pool and the second embedding vector corresponding to the unlabeled text dataset to obtain a weakly labeled text dataset, specifically including: Calculating semantic similarity between a first embedding vector corresponding to the pool of labeled text examples and a second embedding vector corresponding to the unlabeled text dataset; Filtering a preset number of similar text examples that are semantically similar to the unlabeled text dataset from the labeled text example pool in descending order of the semantic similarity; Associating the manually annotated core orientation corresponding to the text example with the similar text example to construct a contextual prompt; The unlabeled text dataset is preliminarily labeled based on the context prompts through a preset large model to obtain a weakly labeled text dataset.

[0006] In one implementation of the present application, based on the cross entropy loss and the Gaussian mixture model, target confidence samples are screened from the weakly annotated text dataset, specifically including: Based on a preset cross entropy loss function, a loss value corresponding to each sample in the weakly annotated text dataset is calculated, and samples whose corresponding loss value is less than a preset threshold are screened from the weakly annotated text dataset as candidate target confidence samples; The loss values of the candidate target confidence samples are distributed modeled by a Gaussian mixture model, so that samples corresponding to low-mean Gaussian components with lower means in each Gaussian distribution are used as target confidence samples.

[0007] In one implementation of the present application, after taking samples corresponding to low-mean Gaussian components with lower Gaussian distribution means in each Gaussian distribution as target confidence samples, the method further includes: Two random samples are selected from the target confidence samples, and the embedding vectors and pseudo labels corresponding to the random samples are linearly mixed to obtain the target confidence samples with the perturbation samples added thereto; Add random perturbations to the target confidence samples with the perturbed samples added thereto, and for each sample, determine a first version sample and a second version sample corresponding to the sample before and after the random perturbation is added, and minimize the KL divergence between the output probability distributions corresponding to the first version sample and the second version sample; The macBERT model is optimized in stages according to the target confidence samples and the target confidence samples with the perturbation samples added, to obtain an optimized macBERT model.

[0008] In one implementation of the present application, a target sample set with a target pseudo label is generated based on the target confidence sample through a preset screening strategy, specifically including: Dividing the target confidence samples into several target confidence sample subsets according to the categories of their corresponding pseudo labels; For the target confidence sample subset, filter out a clean data set whose corresponding cross entropy loss value is within a preset range; Perform k-medoids clustering on the clean data set, and select the center point as the representative sample from each cluster obtained after clustering; According to the representative samples corresponding to each category, a target sample set is constructed, and the target sample set is mixed. Based on the mixed samples and mixed pseudo labels obtained after mixing, the pseudo labels of the target confidence samples are optimized to obtain a target sample set with target pseudo labels.

[0009] In one implementation of the present application, the target sample set is mixed, and based on the mixed samples and mixed pseudo labels obtained after mixing, the pseudo labels of the target confidence samples are optimized to obtain a target sample set with target pseudo labels, specifically including: The target sample set is embedded and mixed using the following formula to obtain mixed target samples and their corresponding mixed pseudo labels:

[0010]

[0011] in, represents mixed target samples, and represents two random representative samples, and represents the pseudo label of the representative sample, represents mixed pseudo labels, represents the weight coefficient; According to the macBERT model, output the initial pseudo label corresponding to the mixed target sample; The macBERT model is optimized according to the difference between the initial pseudo-label and the mixed pseudo-label until the confidence of the initial pseudo-label is greater than a preset confidence, and then the target pseudo-label corresponding to the target sample set is output.

[0012] In one implementation of the present application, a dialogue prompt template is constructed based on the semantic neighbor samples, and based on a preset macBERT model, a refined pseudo-label corresponding to the weakly annotated text dataset is output through the dialogue prompt template, specifically including: According to the text to be refined, assign the pseudo label to the corresponding semantic neighbor sample, and generate a pseudo-annotated example pair according to the text example in the semantic neighbor sample and the pseudo label; Constructing a dialogue prompt template according to the pseudo-annotated example pairs, and outputting refined pseudo labels corresponding to the weakly annotated text dataset through the dialogue prompt template based on a preset macBERT model; The refined pseudo-label is mapped to a numerical label, and when the numerical label does not conform to a preset label, the refined pseudo-label is determined to be an abnormal label.

[0013] In one implementation of the present application, outputting the refined pseudo-label corresponding to the weakly annotated text dataset through the dialogue prompt template specifically includes: For the target confidence samples in the weakly annotated text dataset, the target pseudo labels corresponding to the target confidence samples are retained.

[0014] The present application provides a news content core-oriented annotation device, the device comprising: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a news content core-oriented marking method as described in any one of the above items.

[0015] An embodiment of the present application provides a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to: A news content core-oriented marking method as described in any of the above items.

[0016] The news content core-oriented annotation method proposed in this application can bring the following beneficial effects: By leveraging the collaborative and iterative optimization mechanism of a large model, leveraging its contextual understanding capabilities and multi-task joint modeling, we can accurately identify and extract this complex semantic information, ensuring that the core message of the news is comprehensively and accurately annotated. By synergizing the large model with dynamic distillation, we can generate a high-quality, core-oriented annotated dataset. After multiple rounds of annotation and optimization, this provides reliable training data for subsequent training and application, thereby improving the classification performance of the text classification model. By combining dynamic distillation with supervised fine-tuning techniques, we continuously iterate and refine the annotation results, effectively improving the accuracy and consistency of the annotations. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a manual labeling method provided in an embodiment of the present application; Figure 2 A flowchart of a news content core-oriented annotation method provided in an embodiment of the present application; Figure 3 A schematic diagram of a large model prompt word provided in an embodiment of the present application; Figure 4 A schematic diagram of a prompt word design for the initial marking stage of core-oriented news content provided by an embodiment of the present application; Figure 5 A flowchart of a news content core-oriented preliminary annotation phase provided in an embodiment of the present application; Figure 6 A flowchart of a news content core-oriented refining and annotation phase provided in an embodiment of the present application; Figure 7 A schematic diagram of a prompt word design for the core-oriented refined annotation stage of news content provided by an embodiment of the present application; Figure 8 A structural diagram of a news content core-oriented annotation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] The traditional manual annotation method has the following process: Figure 1 As shown, a labeling plan is formed based on the client's labeling requirements. Labelers then clean, evaluate, and analyze the data to be labeled, completing the labeling. After data review, the final data labeling results are obtained. This can easily lead to the accumulation and poor transmission of errors at each stage when dealing with complex scenarios such as news content with extensive content, strong context dependence, implicit guidance, and intersecting dimensions, making it difficult to ensure the consistency and accuracy of the overall labeling.

[0020] Currently, research on the annotation of core-oriented news content focuses primarily on leveraging advanced technologies such as deep learning and large language models to improve annotation efficiency and accuracy. While traditional data annotation platforms are relatively mature in the field of text annotation, the core-oriented content in news is often implicit and context-dependent. Traditional manual approaches suffer from high cost, low efficiency, and insufficient consistency. In recent years, semi-supervised and self-supervised learning methods have reduced reliance on manual annotation by combining small amounts of labeled data with large amounts of unlabeled data for training. However, these methods remain insufficient in capturing the complex semantic relationships and many-to-many dependencies in news text. Meanwhile, zero-shot and few-shot prompting methods driven by large language models demonstrate great potential in automated annotation due to their powerful contextual understanding and generation capabilities. However, single techniques are prone to error accumulation in multi-stage tasks and lack a unified model for collaborative optimization, making it difficult to achieve both comprehensiveness and accuracy in annotation.

[0021] From a practical application perspective, traditional news annotation methods are prone to error accumulation in key links when dealing with diverse, implicit, and overlapping expressions in news reports, resulting in insufficient generalization of the overall annotation. Based on this, the embodiments of the present application provide a core-oriented annotation method for news content. By integrating the contextual understanding of a large language model with few-shot learning technology, automatic annotation is achieved, thereby effectively improving annotation efficiency and accuracy, overcoming the limitations of the traditional multi-stage process where each link cannot be collaboratively optimized.

[0022] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0023] like Figure 2 As shown, the embodiment of the present application provides a news content core-oriented annotation method, including: S201: Based on text examples with manually annotated core orientations, a preset news text dataset is divided into an annotated text example pool and an unannotated text dataset.

[0024] From the URLs of news reports, we use web news data crawling tools to collect news data, store the data in local files, and clean the collected news text. After completing the news text cleaning, we need to filter out news that exceeds the set word count threshold and perform summary processing to reduce the text length and retain the key information of the news text. Specifically, we deploy a local large model and use the prompt word project to guide the large model to extract the core content of the news and delete redundant details. The prompt word design is as follows: Figure 3 The system provides detailed summary rules for the assistant to follow, while the user inputs the full text of the news article. The output, or assistant, generates the summary text. Taking the extraction of core planning-related guidelines from news articles as an example, the prompt word design needs to retain information related to the core guidelines of the news, such as keywords like "development" and "innovation."

[0025] The above process extracts the main information of the news through a large model, deletes irrelevant content according to the prompt word requirements, and obtains a news text summary. Next, based on the pre-defined news content orientation, multiple different dimensions are determined, such as "international cooperation", "science and technology", "innovation-driven", etc., and then the news text is annotated with manual example data, verification data, and test data. Manual classification is selected from the news dataset to construct text examples with manually annotated core orientations as reference samples for subsequent automatic annotation. Furthermore, based on whether the data carries text annotations, the preset news text dataset is divided into an annotated text example pool and an unannotated text dataset.

[0026] S202: Convert the news text dataset into an embedding vector. Using a preset large model, preliminarily annotate the unlabeled text dataset based on the semantic similarity between the first embedding vector corresponding to the labeled text example pool and the second embedding vector corresponding to the unlabeled text dataset to obtain a weakly annotated text dataset.

[0027] Tokenize and a pre-trained language model (Bidirectional Encoder Representations from Transformers, BERT) are loaded to convert the news text dataset into embedding vectors, providing high-quality input data for subsequent similarity calculations and classification tasks. The embedding vectors include a first embedding vector corresponding to the pool of labeled text examples and a second embedding vector corresponding to the unlabeled text dataset. After obtaining the embedding vectors, the pre-set large model Qwen model is used to perform preliminary annotation on the unlabeled text dataset based on the semantic similarity between the first embedding vector corresponding to the pool of labeled text examples and the second embedding vector corresponding to the unlabeled text dataset, resulting in a weakly labeled text dataset.

[0028] Specifically, based on the Chinese BERT model, the unlabeled text dataset is mapped into a 768-dimensional embedding vector, and the deep semantic information of the text is represented through a high-dimensional semantic space. Its specific form is:

[0029] in, is the encoder function, and d represents the embedding dimension. This process uses BERT’s Transformer encoder to capture text context features and provide a structured vector representation for subsequent similarity matching. Similarly, this method can also be used to obtain the first embedding vector of the annotated text example pool.

[0030] The second embedding vector of the unlabeled text dataset is compared with the first embedding vector of the labeled text example pool. Cosine similarity is used to reflect the semantic similarity between the two. Higher cosine similarity indicates higher semantic similarity. The labeled text examples build contextual references through labels and semantic relevance, providing high-quality prior knowledge for label reasoning.

[0031] The semantic similarity can be calculated using the following formula:

[0032] Among them, t represents the unlabeled text dataset, represents the i-th text example, represents the second embedding vector, Represents the first embedding vector corresponding to the i-th text example.

[0033] After calculating the semantic similarity between the unlabeled text dataset and each labeled text example, a preset number K of similar text examples with semantic similarity to the unlabeled text dataset are screened from the labeled text example pool in descending order of semantic similarity. The preset number here is generally 5-10. After screening multiple text examples with the highest matching degree, the manually annotated labels corresponding to these text examples are associated with similar text examples. Contextual prompts are constructed based on multi-round dialogue templates, and combined with predefined news content guidance, a contextual learning framework is constructed. The multi-round dialogue template is as follows: Figure 4 As shown in the figure, by leveraging the reasoning capability of the pre-set large model Qwen, based on contextual cues, preliminary pseudo-labels are generated according to the semantic relevance between the manually annotated labels and the text to be annotated, thus obtaining a weakly annotated text dataset with preliminary pseudo-labels.

[0034] like Figure 5The figure shows a flow chart of the news content core-oriented preliminary annotation stage. Through BERT, the output unlabeled text dataset and the labeled text example pool are converted into embedded vector representations. By designing prompt words and calculating semantic similarity, similarity matching between the unlabeled text dataset and the labeled text example pool is achieved. Based on the matched similar text examples and contextual prompts, preliminary pseudo-labels are generated for the unlabeled text dataset, thereby obtaining a weakly labeled text dataset.

[0035] S203: Based on the cross entropy loss and Gaussian mixture model, target confidence samples are screened from the weakly annotated text dataset, and a target sample set with target pseudo labels is generated based on the target confidence samples through a preset screening strategy.

[0036] The above process achieves preliminary annotation of the unlabeled text dataset. In order to improve the accuracy of the annotation results, it is necessary to filter the noise and improve the quality of the preliminary annotation results through dynamic distillation technology, so as to generate high-quality news content core-oriented annotation data. Dynamic distillation is mainly divided into two stages: robust training and example pool construction. Robust training reduces the negative impact of noisy annotations on the model by jointly optimizing sample screening and model training. Example pool construction aims to filter high-quality example pools and provide representative feedback index files for subsequent iterative optimization processes. In the robust training stage, target confidence samples are screened from the weakly annotated text dataset based on the cross-entropy loss model. Target confidence samples refer to samples with high confidence, which can effectively filter out noisy annotations; samples are divided into reliable / unreliable categories through a Gaussian mixture model, and low-loss samples are retained. In the example pool construction stage, a preset screening strategy is used to generate a target sample set with target pseudo-labels based on target confidence samples to optimize label quality.

[0037] In one embodiment, it is assumed that the weakly labeled text data output by the large model is , Represents the label space of all possible outputs of the large model, and uses the cross entropy loss function to evaluate the loss value of the sample. The cross entropy loss function is:

[0038] The cross-entropy loss function described above can be used to calculate the loss value for each sample in the weakly annotated text dataset. Then, samples with loss values less than a preset threshold are selected from the weakly annotated text dataset as candidate target confidence samples. By filtering samples based on the loss value, noisy annotations can be removed, ensuring the reliability of the training data.

[0039] The filtered labeled data above needs to be analyzed for reliability using a Gaussian mixture model. The Gaussian mixture model is used to model the distribution of the loss values of the candidate target confidence samples and cluster the data into reliable samples and unreliable samples. The Gaussian mixture model is expressed as:

[0040] in, is the mixing coefficient, and Represent the mean and covariance of the k-th Gaussian distribution, L represents the loss value, It is a mathematical symbol that obeys the Gaussian distribution.

[0041] After fitting the parameters through the expectation maximization algorithm, the samples corresponding to the low-mean Gaussian components with lower means in each Gaussian distribution are used as target confidence samples.

[0042] Furthermore, an embedding-level sample mixing strategy is adopted to generate new training samples by perturbing and mixing the embedding representations of high-confidence samples to improve data diversity. Specifically, two random samples are selected from the target confidence samples. and , the embedding vector corresponding to the random sample is calculated by weighted average and and its pseudo labels and , get the target confidence sample with the perturbation sample added With pseudo labels ,in, is uniformly distributed The weight coefficient of the sampling in the dataset can enhance data diversity and alleviate the problem of sparse annotation by embedding and mixing the sample data.

[0043] Furthermore, by introducing a technique combined with a consistency regularization mechanism, we apply perturbations to the training samples to generate multiple versions and calculate the output consistency between different versions to ensure that the model maintains stable output under different perturbations. This process can be constrained by KL divergence. Specifically, we add random perturbations to the target confidence samples with perturbed samples. For each sample, we determine the first version of the sample before and after adding random perturbations. and the second version sample , and minimize the KL divergence between the output probability distributions corresponding to the first version sample and the second version sample. The loss function is specifically expressed as:

[0044] in, represents the KL divergence, represents the output probability distribution of the first version of the sample, This regularization constraint can improve the robustness of the model in the face of unlabeled data and reduce the impact of data uncertainty.

[0045] Furthermore, the text classification model used to generate core-oriented annotations in the embodiment of the present application is obtained through training based on the macBERT model. In the robust training phase, the macBERT model needs to be optimized in stages based on the target confidence samples and the target confidence samples with perturbation samples added to obtain the optimized macBERT model. The training process is divided into two stages: the initial stage uses the full amount of data to coarsely learn semantic features; the progressive stage gradually transitions to using only high-confidence samples to avoid noise overfitting. The loss function is a comprehensive cross-entropy loss. , mixed loss and consistency loss , the weight coefficient is dynamically adjusted with the training round. The loss function is specifically expressed as:

[0046] in, , , Updated according to exponential decay strategy.

[0047] The above process completes the robust training of the weak text annotation dataset and screens out high-confidence target confidence samples. Next, the quality of its pseudo-labels needs to be optimized.

[0048] Specifically, for the labeled news core-oriented annotation data, the top R% samples with the smallest cross-entropy loss are screened by category, and then the pre-screened samples are clustered in the embedding space, and the class centers are selected as representative examples. Examples of each category are merged to ensure diversity and coverage. Finally, new samples are generated through embedding mixing, and the label quality is optimized in combination with multiple rounds of confidence verification.

[0049] In one embodiment, the screening strategy includes two steps: low-loss selection initial screening and k-medoids clustering fine screening, and is processed on a category basis to ensure that each category is covered and diversity is guaranteed.

[0050] First, the target confidence samples are divided into several target confidence sample subsets according to the categories of their corresponding pseudo labels. For the target confidence sample subset, filter out clean datasets whose corresponding cross entropy loss values fall within a preset range. During this screening process, select samples with the smallest cross entropy loss values and proportionally select the top R% of samples with the smallest cross entropy loss. The minimum loss criterion ensures high dataset accuracy by selecting samples with the smallest cross entropy loss. Setting R to a small value ensures high accuracy for the selected samples.

[0051] Then, the clustering algorithm k-medoids is used to process the embedded representation of the pre-trained model, namely the macBERT model, so as to obtain Filter out the most representative samples and build a more representative example data set The algorithm divides the data points into k clusters and selects the center point of each cluster as a representative sample. Finally, these representative samples are collected as When the k-medoids algorithm converges, the center samples of the k clusters are representative samples, which are used as high-quality examples for subsequent training and fine-tuning.

[0052] According to the representative samples corresponding to each category, the target sample set is constructed, which is specifically expressed as:

[0053] Target sample The collection contains representative samples from all categories, ensuring the diversity and quality of the dataset.

[0054] Finally, based on the cleaned samples, a mixing strategy is embedded to enhance data diversity. That is, the target sample set is mixed, and based on the mixed samples and mixed pseudo-labels obtained after mixing, the pseudo-labels of the target confident samples are optimized to obtain the target sample set with the target pseudo-labels.

[0055] Specifically, the target sample set is embedded and mixed to obtain mixed target samples and their corresponding mixed pseudo labels:

[0056]

[0057] in, represents mixed target samples, and represents two random representative samples, and represents the pseudo label of the sample, represents mixed pseudo labels, Represents the weight coefficient.

[0058] According to the macBERT model, the initial pseudo-label corresponding to the mixed target sample is output. Then, based on the difference between the initial pseudo-label and the mixed pseudo-label, the macBERT model is optimized until the confidence of the initial pseudo-label is greater than the preset confidence, and the target pseudo-label corresponding to the target sample set is output.

[0059] S204: Determine the text to be refined that is not included in the target sample set in the weakly annotated text dataset, retrieve semantic neighbor samples of the text to be refined from the annotated text example pool, and construct a dialogue prompt template based on the semantic neighbor samples, so as to output the refined pseudo-label corresponding to the weakly annotated text dataset through the dialogue prompt template based on the preset macBERT model.

[0060] After generating the pseudo-labels corresponding to the weakly annotated text dataset, it is necessary to refine the annotations to improve the accuracy and semantic consistency of the annotated data. First, determine the text to be refined that is not included in the target sample set in the weakly annotated text dataset, retrieve the semantic neighbor samples of the text to be refined from the annotated text example pool, and build a dialogue prompt template based on the semantic neighbor samples. Based on the preset macBERT model, the dialogue prompt template outputs the refined pseudo-labels corresponding to the weakly annotated text dataset. Refining the annotations based on the preliminary annotation results guided by the news core aims to improve the accuracy and semantic consistency of the annotation data. The specific process is as follows: Figure 6 shown.

[0061] In one embodiment, based on the target pseudo-labels generated by the dynamic distillation process, an example library for the refinement process is constructed. For the text to be refined t, its semantic neighbor samples are retrieved from the annotated text example pool to form a reference example set. .

[0062] Leveraging precomputed embedding vectors , retrieve the Top-K similar sample indexes from the training data ,satisfy:

[0063] in, is the cosine similarity function, For samples in the training set The embedding vector of .

[0064] According to the pseudo label corresponding to the text to be refined, for each example Assign pseudo labels generated by preliminary annotations , and construct pseudo-labeled example pairs based on text examples and their corresponding pseudo labels .

[0065] The prompt word design in the core-oriented refined annotation stage of news content is as follows: Figure 7As shown in the figure, structured context injection enhances the model's semantic understanding capabilities. Dynamic distillation feedback is combined with the text to be annotated to construct a dialogue prompt template. This is based on pseudo-annotated examples, guiding the macBERT model to generate more accurate and refined pseudo-labels in semantically relevant scenarios. Each example and its pseudo-label are converted into a "user-assistant" dialogue pair, forming a learning paradigm for annotation rules. The prompts clearly define the classification dimensions and output constraints, limiting selection to preset dimensions, and strengthening the model's understanding and adherence to the rules.

[0066] A dual calibration strategy is designed for the refined results to balance annotation correction and stability. Therefore, after generating refined pseudo-labels, the labels need to be mapped and exceptions handled. Specifically, the refined pseudo-labels of the Chinese dimension output by the model are mapped to numerical labels. If the numerical labels do not conform to the preset labels, the refined pseudo-labels are determined to be abnormal labels and marked. .

[0067] It should be noted that when refining and optimizing pseudo labels, it is necessary to protect high-confidence target confidence samples. That is to say, if a sample belongs to the target confidence sample, for this type of high-quality sample set, its initial pseudo label is retained to avoid miscorrection of reliable labels.

[0068] S205: Input the target sample set and the text to be refined as a training set into the macBERT model for training to obtain a trained text classification model, and output the core-oriented annotations of the news to be annotated through the text classification model.

[0069] The above steps achieve efficient and accurate annotation of the core orientation of news texts through the synergy of semantic embedding, dynamic distillation, and iterative optimization. Based on the high-quality news core orientation annotation data generated by iterative annotation, namely the target sample set and the text to be refined, the Chinese pre-trained model macBERT is used to perform supervised training on the dataset. Combined with data augmentation, model optimization, adversarial training, and learning rate scheduling, a trained text classification model is obtained. This text classification model can then determine the core orientation of each news article.

[0070] First, we cleaned and enhanced the text data to improve the model's adaptability to different types of input. In addition to standard cleaning operations like removing stop words and punctuation, we also introduced simplified and traditional Chinese character conversion and synonym replacement techniques to enrich the diversity of the training data. Using the OpenCC tool, we randomly converted text between simplified and traditional Chinese to enhance the model's adaptability to multiple languages.

[0071] Generate a synonym dictionary based on the Word2Vec model. For each word, calculate the similarity between its word vector and other words, and filter out the most similar synonyms based on the set similarity threshold.

[0072] Secondly, additional classification layers are added, including fully connected layers, ReLU activation functions, and LayerNorm, to enhance the model's expressiveness and classification accuracy to cope with the complexity of the task and the imbalance of data categories. Focal Loss is used as the loss function. Focal Loss adjusts the weights of difficult and easy samples, allowing the model to pay more attention to difficult-to-classify samples during training. Its implementation is as follows:

[0073] in, is the probability predicted by the model, is the category weight, is the adjustment parameter.

[0074] Then, the robustness and generalization ability of the model in text classification tasks are enhanced. The gradient of the embedding layer is calculated, and perturbations are added according to the gradient direction to generate adversarial samples. The model calculates the loss on the original samples and adversarial samples separately, and updates the parameters through double backpropagation. In each training step, the loss of the current batch is first calculated and backpropagated to obtain the gradient information of the embedding layer. Subsequently, small perturbations are added according to the gradient direction to generate adversarial samples. The model calculates the loss on the original samples and adversarial samples separately, and updates the parameters through double backpropagation. In this way, the model can learn feature representations that are insensitive to input perturbations, thereby effectively alleviating the overfitting problem and maintaining high accuracy and stability in the face of noisy data. In each training step, adversarial samples are generated by adding gradient-based perturbations to the embedding layer, so that the model can maintain good classification performance even in the face of slight perturbations.

[0075] Finally, the training process uses the AdamW optimizer, combined with a cosine annealing learning rate scheduler to dynamically adjust the learning rate for smooth model convergence. An early stopping mechanism is introduced to terminate training early if the validation set accuracy does not improve over multiple epochs to avoid overfitting.

[0076] Through the combination of the above technologies, we can finally obtain a text classification model that can accurately identify the core orientation of news, realizing the transformation from high-quality labeled data to an efficient classifier.

[0077] The above are embodiments of the method proposed in this application. Based on the same idea, some embodiments of this application also provide devices and non-volatile computer storage media corresponding to the above methods.

[0078] Figure 8This is a structural diagram of a news content core-oriented annotation device provided in an embodiment of the present application. Figure 8 Shown, including: at least one processor; and, at least one processor communicatively connected to a memory; wherein, The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute a news content core-oriented marking method as described in any one of the above items.

[0079] An embodiment of the present application provides a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute a news content core-oriented annotation method as described in any one of the above items.

[0080] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device and medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For relevant portions, refer to the descriptions of the method embodiments.

[0081] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0082] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0084] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0086] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0087] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0088] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0089] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0090] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A news content core-oriented annotation method, characterized in that: The method comprises: Based on the text examples with manual annotation core guidance, the preset news text dataset is divided into an annotated text example pool and an unannotated text dataset; The news text dataset is converted into an embedding vector, and the unlabeled text dataset is preliminarily labeled based on the semantic similarity between the first embedding vector corresponding to the labeled text example pool and the second embedding vector corresponding to the unlabeled text dataset using a preset large model to obtain a weakly labeled text dataset; Based on the cross entropy loss and Gaussian mixture model, target confidence samples are screened from the weakly annotated text dataset, and a target sample set with target pseudo labels is generated based on the target confidence samples through a preset screening strategy; Determining text to be refined that is not included in the target sample set in the weakly annotated text dataset, retrieving semantic neighbor samples of the text to be refined from the annotated text example pool, and constructing a dialogue prompt template based on the semantic neighbor samples, so as to output refined pseudo-labels corresponding to the weakly annotated text dataset through the dialogue prompt template based on a preset macBERT model; The target sample set and the text to be refined are input as training sets into the macBERT model for training to obtain a trained text classification model, and the core-oriented annotations of the news to be annotated are output through the text classification model.

2. A news content core-oriented annotation method according to claim 1, characterized in that: Preliminarily annotating the unlabeled text dataset based on the semantic similarity between the first embedding vector corresponding to the labeled text example pool and the second embedding vector corresponding to the unlabeled text dataset to obtain a weakly annotated text dataset, specifically comprising: Calculating semantic similarity between a first embedding vector corresponding to the pool of labeled text examples and a second embedding vector corresponding to the unlabeled text dataset; Filtering a preset number of similar text examples that are semantically similar to the unlabeled text dataset from the labeled text example pool in descending order of the semantic similarity; Associating the manually annotated core orientation corresponding to the text example with the similar text example to construct a contextual prompt; The unlabeled text dataset is preliminarily labeled based on the context prompts through a preset large model to obtain a weakly labeled text dataset.

3. A news content core-oriented annotation method according to claim 1, characterized in that: Based on the cross entropy loss and Gaussian mixture model, target confidence samples are screened from the weakly annotated text dataset, specifically including: Based on a preset cross entropy loss function, a loss value corresponding to each sample in the weakly annotated text dataset is calculated, and samples whose corresponding loss value is less than a preset threshold are screened from the weakly annotated text dataset as candidate target confidence samples; The loss values of the candidate target confidence samples are distributed modeled by a Gaussian mixture model, so that samples corresponding to low-mean Gaussian components with lower means in each Gaussian distribution are used as target confidence samples.

4. A news content core-oriented annotation method according to claim 3, characterized in that: After taking samples corresponding to low-mean Gaussian components with lower Gaussian distribution means in each Gaussian distribution as target confidence samples, the method further includes: Two random samples are selected from the target confidence samples, and the embedding vectors and pseudo labels corresponding to the random samples are linearly mixed to obtain the target confidence samples with the perturbation samples added thereto; Add random perturbations to the target confidence samples with the perturbed samples added thereto, and for each sample, determine a first version sample and a second version sample corresponding to the sample before and after the random perturbation is added, and minimize the KL divergence between the output probability distributions corresponding to the first version sample and the second version sample; The macBERT model is optimized in stages according to the target confidence samples and the target confidence samples with the perturbation samples added, to obtain an optimized macBERT model.

5. A news content core-oriented annotation method according to claim 1, characterized in that: Through a preset screening strategy, based on the target confidence samples, a target sample set with target pseudo labels is generated, specifically including: Dividing the target confidence samples into several target confidence sample subsets according to the categories of their corresponding pseudo labels; For the target confidence sample subset, filter out a clean data set whose corresponding cross entropy loss value is within a preset range; Perform k-medoids clustering on the clean data set, and select the center point as the representative sample from each cluster obtained after clustering; According to the representative samples corresponding to each category, a target sample set is constructed, and the target sample set is mixed. Based on the mixed samples and mixed pseudo labels obtained after mixing, the pseudo labels of the target confidence samples are optimized to obtain a target sample set with target pseudo labels.

6. A news content core-oriented annotation method according to claim 5, characterized in that: The target sample set is mixed, and based on the mixed samples and the mixed pseudo labels obtained after mixing, the pseudo labels of the target confidence samples are optimized to obtain a target sample set with target pseudo labels, specifically including: The target sample set is embedded and mixed using the following formula to obtain mixed target samples and their corresponding mixed pseudo labels: in, represents mixed target samples, and represents two random representative samples, and represents the pseudo label of the representative sample, represents mixed pseudo labels, represents the weight coefficient; According to the macBERT model, output the initial pseudo label corresponding to the mixed target sample; The macBERT model is optimized according to the difference between the initial pseudo-label and the mixed pseudo-label until the confidence of the initial pseudo-label is greater than a preset confidence, and then the target pseudo-label corresponding to the target sample set is output.

7. A news content core-oriented annotation method according to claim 5, characterized in that: Constructing a dialogue prompt template according to the semantic neighbor samples, and outputting refined pseudo labels corresponding to the weakly annotated text dataset through the dialogue prompt template based on a preset macBERT model, specifically including: According to the text to be refined, assign the pseudo label to the corresponding semantic neighbor sample, and generate a pseudo-annotated example pair according to the text example in the semantic neighbor sample and the pseudo label; Constructing a dialogue prompt template according to the pseudo-annotated example pairs, and outputting refined pseudo labels corresponding to the weakly annotated text dataset through the dialogue prompt template based on a preset macBERT model; The refined pseudo-label is mapped to a numerical label, and when the numerical label does not conform to a preset label, the refined pseudo-label is determined to be an abnormal label.

8. A news content core-oriented annotation method according to claim 7, characterized in that: Outputting refined pseudo labels corresponding to the weakly annotated text dataset using the dialogue prompt template specifically includes: For the target confidence samples in the weakly annotated text dataset, the target pseudo labels corresponding to the target confidence samples are retained.

9. A news content core-oriented annotation device, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a news content core-oriented marking method as described in any one of claims 1-8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to: A news content core-oriented annotation method as described in any one of claims 1 to 8.

Images