A method for intelligent content filtering based on a specified theme scenario

Abstract

The application relates to a method for intelligent content filtering based on a specified theme scene, which realizes accurate filtering of irrelevant content of ASR output text in the specified theme scene. The method comprises the following steps: optimizing a zero-shot classification model to obtain an optimized zero-shot classification model; performing theme classification on first ASR output text by using the optimized zero-shot classification model to obtain first irrelevant content text; constructing a dynamic word library, calculating the relevance of the first ASR output text and the dynamic word library, and screening second irrelevant content text according to the relevance; comparing and repeating the first irrelevant content text and the second irrelevant content text to obtain third irrelevant content text; and removing the third irrelevant content text from the first ASR output text to obtain filtered first ASR output text.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic digital data processing, in particular to a method for intelligent content filtering based on a specified theme scenario. BACKGROUND

[0002] With the rapid development of speech recognition and speech interaction technology, speech analysis has been widely applied in the fields of information processing, human-computer interaction, intelligent customer service, speech recognition, etc. However, in actual application, speech data often contains a large amount of invalid information, such as background noise, irrelevant speech, repeated content, invalid instructions, etc. These information not only affects the recognition accuracy, but also increases the system processing burden.

[0003] To improve the efficiency of speech information processing, various technical means are often used in the prior art to filter and identify invalid information. For example, speech recognition models based on natural language processing (NLP) and deep learning can perform semantic analysis and classification on speech content, and identify and filter invalid content. In addition, voice activity detection (VAD) technology can be used to identify valid speech segments in speech segments, and filter background noise and silent segments. At the same time, multi-modal fusion technology (such as speech, text, image, etc.) is also used to improve the accuracy and robustness of information processing.

[0004] However, the prior art still has many deficiencies in complex scenarios. For example, in complex environments (such as noisy environments, multiple speakers, far-field speech), speech recognition and recognition accuracy still face challenges; some models have limited context understanding ability, making it difficult to effectively identify invalid information in the context; in addition, the cost of model training and deployment is high, and it has strong dependence on computing resources and data.

[0005] Through literature retrieval of prior art, it is found that the patent with application number 202410727979.6 and the patent name of "a telecommunication customer service business recommendation method and system based on intelligent speech analysis" collects the speech data of the customer from the telecommunication customer service system, converts the speech data of the customer into text information, and obtains the intonation information of the speech data; the intonation information of the obtained speech data and the historical business crisis information are introduced into the business crisis value analysis strategy to analyze the business crisis value; the converted text information is substituted into the keyword extraction strategy to extract the keywords; the keywords and the historical keywords are introduced into the business importance analysis strategy to analyze the business importance; the business crisis value and the business importance obtained by analysis are introduced into the business recommendation coefficient analysis strategy to obtain the business recommendation coefficient, and the business department is recommended according to the obtained business recommendation coefficient. However, this patent lacks the classification of the theme of the recording text, and uses a pre-set traditional static word library, which has deficiencies in recognition accuracy, efficiency and robustness. SUMMARY

[0006] Based on this, it is necessary to provide a method of intelligent content filtering based on a specified theme scenario to achieve accurate filtering of irrelevant content in ASR output text in the specified theme scenario.

[0007] To achieve the above technical problems, the present application provides the following technical solutions:

[0008] The present application provides a method of intelligent content filtering based on a specified theme scenario, which comprises:

[0009] S1: optimizing a zero-shot classification model to obtain an optimized zero-shot classification model, wherein the zero-shot classification model classifies themes according to a constructed knowledge graph prompt template, and the knowledge graph prompt template comprises theme labels and keywords matched with the theme labels;

[0010] S2: using the optimized zero-shot classification model to classify the themes of a first ASR output text to obtain a first irrelevant content text;

[0011] S3: constructing a dynamic word library, calculating the relevance of the first ASR output text and the dynamic word library, and screening a second irrelevant content text according to the relevance, wherein the dynamic word library is updated synchronously with the keywords in the knowledge graph prompt template, and is used to fine-tune the optimized zero-shot classification model at a preset time;

[0012] S4: comparing and repeating the contents of the first irrelevant content text and the second irrelevant content text to obtain a third irrelevant content text;

[0013] S5: removing the third irrelevant content text from the first ASR output text to obtain a filtered first ASR output text.

[0014] Preferably, the optimization of the zero-shot classification model comprises:

[0015] The second ASR output text is cut according to sentences and the speaker's identity is marked;

[0016] The constructed knowledge graph prompt template is used to divide the themes of each sentence in the second ASR output text to obtain the preliminary theme labels corresponding to each sentence;

[0017] According to the continuity of the time stamp of the sentence and the preliminary theme label, the second ASR output text is divided into several segments;

[0018] The sentences with similar semantics in the same segment are classified as positive samples, and the sentences in different segments or with dissimilar semantics are classified as negative samples;

[0019] According to the classified positive samples and negative samples, a contrast loss function is used to fine-tune the zero-shot classification model, and an optimized zero-shot classification model is obtained.

[0020] Preferably, the contrast loss function calculation expression is as follows:

[0021]

[0022] wherein, represents the contrast loss, represents the embedding representation of the current sentence, represents the embedding representation of the positive sample sentence similar to the semantic, represents the embedding representation of the negative sample sentence, represents the cosine similarity function, represents the temperature hyperparameter, used to adjust the distribution sharpness, represents the sentence encoding function, represents the natural exponential function, represents all negative sample sentences, represents the i-th negative sample sentence.

[0023] Preferably, step S2 comprises:

[0024] The first ASR output text is classified by using the optimized zero-shot classification model to obtain irrelevant content topic labels, irrelevant content texts and confidences, wherein the confidence is the relevance probability of the irrelevant content text and the irrelevant content topic label.

[0025] The irrelevant content texts with a confidence higher than a preset threshold are screened to obtain first irrelevant content texts.

[0026] Preferably, step S3 comprises:

[0027] A dynamic word library is constructed, and a TF-IDF algorithm is used to calculate the relevance of the first ASR output text and the dynamic word library.

[0028] The texts in the first ASR output text with a relevance lower than a preset threshold are screened to obtain second irrelevant content texts.

[0029] Preferably, constructing the dynamic word library comprises:

[0030] The first ASR output text is subjected to keyword extraction to obtain candidate keywords.

[0031] It is judged whether the candidate keywords are in the dynamic word library. If yes, the candidate keywords are not subjected to three-dimensional correlation degree evaluation, and if not, the candidate keywords are subjected to three-dimensional correlation degree evaluation.​​

[0032] The candidate keyword is subjected to three-dimensional correlation degree evaluation to obtain an evaluation score, if the evaluation score is greater than a first preset score, the candidate keyword is added to the dynamic word library, if the evaluation score is less than the first preset score and greater than a second preset score, the candidate keyword is subjected to manual review, if the evaluation score is less than the second preset score, the candidate keyword is not added to the dynamic word library.

[0033] Preferably, the candidate keyword is obtained, comprising:

[0034] The N-gram model and the TF-IDF algorithm are adopted to extract high-frequency words from the first ASR output text to obtain high-frequency keywords;

[0035] The KeyBERT model is adopted to extract keywords from the first ASR output text to obtain zero-sample keywords;

[0036] The high-frequency keywords and the zero-sample keywords are merged to obtain the candidate keyword.

[0037] Preferably, the merging comprises:

[0038] The stop words and the general words in the high-frequency keywords and the zero-sample keywords are removed to obtain first candidate keywords;

[0039] The keywords in the first candidate keywords with a repetition frequency ranking less than a preset ranking in the high-frequency keywords or the zero-sample keywords are reserved to obtain second candidate keywords;

[0040] The repeated words in the second candidate keywords are removed and the keywords with similar semantics are merged to obtain the candidate keyword.

[0041] Preferably, the three-dimensional correlation degree evaluation comprises:

[0042] The cosine similarity of the candidate keyword and a preset core keyword library under a specified theme scenario is calculated;

[0043] The co-occurrence probability of the candidate keyword in the dialogue and an action verb under the specified theme scenario is calculated;

[0044] The business word corresponding to the candidate keyword is matched with a preset business dictionary to obtain a preset weight value;

[0045] The evaluation score is calculated according to the cosine similarity, the co-occurrence probability and the preset weight value.

[0046] Preferably, if the keyword in the dynamic word library is not used for theme classification within a preset number of days, the keyword is removed from the dynamic word library.

[0047] Compared with the prior art, the application has the beneficial effects that:

[0048] The application provides a method for intelligent content filtering based on a specified theme scenario, optimizes a zero-shot classification model by taking a constructed knowledge graph prompt template as a prompt word of the zero-shot classification model, and effectively improves the classification accuracy of the optimized zero-shot classification model. A sustainable dynamic word library is constructed, which is used in two-stage content parallel filtering on one hand. Only content that meets the filtering requirements in both the zero-shot classification stage and the dynamic word library matching stage will be filtered, thereby minimizing false filtering. On the other hand, the dynamic word library is updated synchronously with the keywords in the knowledge graph prompt template, and the zero-shot classification model is optimized by regular training, thereby improving the adaptability of the model in a specific field. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 A flowchart of the method for intelligent content filtering based on a specified theme scenario in an embodiment;

[0050] Figure 2 A zero-shot classification model optimization diagram of the method for intelligent content filtering based on a specified theme scenario in an embodiment;

[0051] Figure 3 A three-dimensional correlation degree evaluation flowchart of the method for intelligent content filtering based on a specified theme scenario in an embodiment;

[0052] Figure 4 A system architecture diagram of the method for intelligent content filtering based on a specified theme scenario in an embodiment. DETAILED DESCRIPTION

[0053] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0054] Embodiment 1

[0055] As shown in Figure 1 , the present embodiment proposes a method for intelligent content filtering based on a specified theme scenario, which comprises:

[0056] S1: optimizing a zero-shot classification model to obtain an optimized zero-shot classification model, wherein the zero-shot classification model performs theme classification according to a constructed knowledge graph prompt template, and the knowledge graph prompt template comprises theme labels and keywords matched with the theme labels;

[0057] In this step, the specific embodiments are as follows:

[0058] Taking real estate-themed projects as an example, the goal of zero-shot classification model optimization is to improve the zero-shot model's ability to understand fine-grained semantics in the real estate field and avoid misjudging terms such as "unit type" and "usable floor area ratio" by general models. Figure 2 As shown, the zero-shot classification model is optimized by using knowledge graph prompt templates and fine-tuning them through dialogue scenario comparison.

[0059] Knowledge Graph Hint Template: This template extracts core entities and relationships from a project database within a specified thematic scenario, forming a structured graph. Taking a real estate project database as an example, entities include: apartment layouts, pricing strategies, competitor comparisons, and customer objections. Relationships related to apartment layouts include: number of bedrooms, number of bathrooms, and usable floor area ratio. Relationships related to pricing strategies include: discount rates, payment methods, and competitor benchmarking. Relationships triggered by customer objections include: transportation amenities, school district resources, and delivery standards. After the knowledge graph is constructed, the graph relationships need to be converted into zero-shot classification hints to enhance domain semantic awareness. This is the knowledge graph hint template, which includes multiple topic tags such as apartment layout description, price negotiation, competitor comparison, customer needs analysis, and objection response. In text converted from ASR (Automatic Sentence Recognition), mentioning keywords like "three bedrooms and two living rooms" or "usable floor area ratio" can classify it as an apartment layout description; discussing keywords like "discount" or "loan ratio" can classify it as price negotiation; and mentioning keywords like "XX apartment comparison" can classify it as competitor comparison.

[0060] Dialogue Structure Fine-tuning: 500,000 unlabeled dialogues between real estate consultants and clients were collected as fine-tuning data and input into a zero-shot classification model. Unsupervised contrastive learning was used to train the model to distinguish the semantic weights of consultant (proactive sales) and client (needs / concerns) statements. A Transformer encoder was used to capture cross-statement dependencies (such as clients repeatedly asking about apartment layouts, which were marked as high-weight segments). A contrastive learning loss function was used during training to bring semantically similar statement representations closer together and push away irrelevant statement representations. This loss function compares the similarity of positive and negative samples, enabling the model to learn to distinguish statements of different semantic types and improve the representation quality of the dialogue structure.

[0061] The specific steps are as follows:

[0062] Input preprocessing: The ASR-processed data is segmented by statement and the speaker's identity (consultant / client) is marked.

[0063] Knowledge graph topic classification: Using the constructed knowledge graph prompt template, zero-sample topic classification is performed on each statement to obtain preliminary topic tags (such as "apartment type description", "price negotiation", etc.).

[0064] Dialogue segment division: divide the dialogue into several semantically coherent segments according to the time stamp and topic continuity of the sentences;

[0065] Positive and negative sample construction: positive samples are pairs of sentences with similar semantics within the same segment, and negative samples are pairs of sentences from different segments or with unrelated semantics;

[0066] Contrastive learning fine-tuning: fine-tune the pre-trained language model using the above loss function to optimize the sentence representation;

[0067] Output optimization topic: the fine-tuned model classifies the sentences again to get more accurate topic labels, which provides the basis for subsequent keyword extraction.

[0068] After the optimization of the zero-shot classification model, the topic classification accuracy on the test set is improved from 74% of the general model to 89%, and the term misjudgment rate is reduced by 62%.

[0069] S2: Use the optimized zero-shot classification model to classify the first ASR output text by topic to obtain the first irrelevant content text;

[0070] The specific implementation of this step is as follows:

[0071] Use the optimized zero-shot classification model to classify the first ASR output text by topic, obtain the zero-shot classification result, and the probability distribution (confidence) corresponding to each class. The optimized zero-shot classification model is a multi-classification model, which outputs the relevance probability (confidence) of each class. Take a real estate theme project as an example. For example, the relevant content categories include: house price, house type, location, orientation, floor, garage, etc. The irrelevant content categories include: food, entertainment, retail, manufacturing, medical care, etc. Each type will output a confidence and a preset threshold. The first ASR output text is filtered in one stage, and the confidence of irrelevant content is compared with the threshold. If the probability is higher than the set threshold, it is filtered out to obtain the first irrelevant content text.

[0072] S3: Construct a dynamic word library, calculate the relevance of the first ASR output text and the dynamic word library, and filter the second irrelevant content text according to the relevance. The dynamic word library is updated synchronously with the keywords in the knowledge graph prompt template, and is used to fine-tune the optimized zero-shot classification model at a preset time;

[0073] The specific implementation of this step is as follows:

[0074] Taking a real estate theme project as an example, in the real estate marketing scene, the conversation content between the sales consultant and the customer is complex and diverse, including explicit real estate terms and a large amount of marginal content related to real estate (such as surrounding supporting facilities, school district policies, and transportation planning). To ensure that no potential business opportunities are lost, a two-stage independent filtering mechanism is used to increase the second stage of filtering. Only conversations that are consistently determined to be "irrelevant content" by both stages will be filtered, otherwise they will be retained, maximizing the integrity of business information. As shown in Figure 4 The second path is a dynamically updated dynamic word library that cooperates with the first zero-shot classification path to improve system performance. The core role of the dynamic word library in the second path is to provide continuous optimization of domain knowledge support for the zero-shot classification engine of the first path. Specifically:

[0075] Feedback optimization mechanism: new keywords (such as the new policy term "public housing new policy" and the new project name "Star Horizon TOD") mined from the second path in the vast amount of conversations will be added to the dynamic word library in real time. To ensure that no potential business opportunities are lost, a two-stage independent filtering mechanism is used, and the dynamic word library is used for the second filtering after zero-shot classification filtering.

[0076] Enhanced classification capability: the updated dynamic word library is synchronized with the keywords in the knowledge graph prompt template, enabling the zero-shot classification model to recognize these emerging terms. Since the reasoning process relies on the prompt words of the knowledge graph, which will affect the reasoning efficiency as the capacity of the knowledge graph increases, the updated dynamic word library will also be used as a training set to fine-tune the bert zero-shot classification model periodically, building a new zero-shot classification model for classification, forming a business closed loop, and improving the accuracy and performance of the model.

[0077] Closed-loop optimization system: the two paths form a continuous optimization closed loop, the second path updates the word library, and the enhanced word library improves the filtering accuracy of the first path. This design ensures that the system can adapt to language changes in the real estate marketing scene and overcome the limitations of traditional static word libraries.

[0078] By constructing a dynamic word library, the first ASR output text is filtered in two stages. The first ASR output text and the dynamic word library are calculated for relevance using the TF-IDF algorithm. Specifically, the first ASR output text is preprocessed and tokenized, and the preprocessed and tokenized text is compared with the dynamic word library to count the matching words related to real estate. The relevant word density is obtained by dividing the total number of irrelevant content text words. When the relevant word density is below a preset threshold, it is determined that the irrelevant content text needs to be filtered, and the second irrelevant content text is obtained.

[0079] S4: Compare the first irrelevant content text and the second irrelevant content text for duplicate content to obtain the third irrelevant content text;

[0080] The specific implementation method for this step is as follows:

[0081] Only conversations that are consistently deemed irrelevant by both stages will be filtered; otherwise, they will be retained to ensure the integrity of business information to the greatest extent possible.

[0082] S5: Remove the third irrelevant content text from the first ASR output text to obtain the filtered first ASR output text.

[0083] Example 2

[0084] This embodiment further explains the method for intelligent content filtering based on a specified theme scenario proposed in Embodiment 1.

[0085] Optimization of the zero-shot classification model includes:

[0086] The second ASR output text is segmented by sentence and the speaker's identity is marked;

[0087] The constructed knowledge graph prompt template is used to divide each sentence in the second ASR output text into topics, and the preliminary topic labels corresponding to each sentence are obtained.

[0088] Based on the timestamps of the statements and the continuity of the initial topic tags, the second ASR output text is divided into several segments;

[0089] Statements with similar meanings within the same segment are classified as positive samples, while statements from different segments or with different meanings are classified as negative samples.

[0090] Based on the classified positive and negative samples, the zero-shot classification model is fine-tuned using a contrastive loss function to obtain an optimized zero-shot classification model.

[0091] The expression for calculating the contrastive loss function is as follows:

[0092]

[0093] in, Indicates comparative loss, This indicates the embedding representation of the current statement. Indicates and Embedded representation of semantically similar positive sample statements The embedding representation of negative sample statements. Represents the cosine similarity function. This represents a temperature hyperparameter used to adjust the sharpness of the distribution. represents a natural exponential function, represents a natural exponential function, represents all negative sample sentences, represents the first negative sample sentence.

[0094] Embodiment 3

[0095] This embodiment further supplements the method for intelligent content filtering based on a specified theme scenario proposed in Embodiment 1.

[0096] Step S2 comprises:

[0097] The optimized zero-sample classification model is used for subject classification on the first ASR output text, and irrelevant content subject labels, irrelevant content text and confidence are obtained through classification, wherein the confidence is the relevance probability of the irrelevant content text and the irrelevant content subject label.

[0098] Irrelevant content text with a confidence higher than a preset threshold is screened out to obtain first irrelevant content text.

[0099] Step S3 comprises:

[0100] A dynamic word library is constructed, and a TF-IDF algorithm is used to calculate the relevance of the first ASR output text and the dynamic word library.

[0101] Text in the first ASR output text with a relevance lower than a preset threshold is screened out to obtain second irrelevant content text.

[0102] The dynamic word library is constructed, comprising:

[0103] Key words are extracted from the first ASR output text to obtain candidate key words.

[0104] It is judged whether the candidate key words are in the dynamic word library, if yes, the three-dimensional correlation degree of the candidate key words is not evaluated, and if not, the three-dimensional correlation degree of the candidate key words is evaluated.

[0105] The three-dimensional correlation degree of the candidate key words is evaluated to obtain an evaluation score, if the evaluation score is greater than a first preset score, the candidate key words are added to the dynamic word library, if the evaluation score is less than the first preset score and greater than a second preset score, the candidate key words are manually reviewed, and if the evaluation score is less than the second preset score, the candidate key words are not added to the dynamic word library.

[0106] The candidate key words are obtained, comprising:

[0107] An N-gram model and a TF-IDF algorithm are used to extract high-frequency words from the first ASR output text to obtain high-frequency key words.

[0108] perform key word extraction on the first ASR output text using a KeyBERT model to obtain zero-sample key words;

[0109] merge the high-frequency key words and the zero-sample key words to obtain candidate key words.

[0110] The implementation of this step is as follows:

[0111] Taking a real estate theme project as an example, in order to break through the limitation of the static word library and dynamically capture the emerging terms in the marketing scene (such as “purchase limit relaxation” and “old-for-new policy”), a double-path collaborative extraction of new key words is adopted. Specifically, an N-gram model and a TF-IDF algorithm are used to extract high-frequency words from the first ASR output text to obtain high-frequency key words; a KeyBERT model is used to extract key words from the first ASR output text to obtain zero-sample key words; and the high-frequency key words and the zero-sample key words are merged to obtain candidate key words. The implementation steps are as follows:

[0112] input the first ASR output text (divided by sentences) classified by the theme and the theme labels (such as “price negotiation” and “school district demand”) output by the zero-sample classification model;

[0113] use the theme labels as Seed Keywords of KeyBERT to limit the semantic direction of key word extraction, call the KeyBERT model to extract key words, and merge the key words with the same semantics in the advisor and customer dialogue (for example, the customer asks “house rate”, and the advisor explains “unit area ratio”, which are merged as “house rate = unit area ratio”) to obtain zero-sample key words; in addition to the KeyBERT path guided by zero samples to extract zero-sample key words, a traditional N-gram model and TF-IDF algorithm path are also used to extract high-frequency key words such as “three-bedroom” and “down payment ratio”; merge the zero-sample key words and the high-frequency key words, remove stop words (such as “this” and “then”) and general words (such as “good” and “understand”); keep at least the top 10 candidate words in one path; remove duplicate words and merge semantically similar words; finally obtain candidate key words. For example, the traditional path outputs: “three-bedroom”, “down payment ratio”, “school district”; the zero-sample path outputs: “purchase limit relaxation”, “old-for-new policy”, “house rate”; and the merged result is: “three-bedroom”, “down payment ratio”, “school district”, “purchase limit relaxation”, “old-for-new policy”, “house rate”. After obtaining the candidate key words, it is determined whether the candidate key words are in the dynamic word library. If not, three-dimensional correlation degree evaluation is performed, and then it is determined whether to add the candidate key words to the dynamic word library.

[0114] The merging includes:

[0115] Remove stop words and general words in high-frequency keywords and zero-sample keywords to obtain first candidate keywords;

[0116] Retain keywords in the first candidate keywords that have a repetition rank less than a preset rank in the high-frequency keywords or the zero-sample keywords to obtain second candidate keywords;

[0117] Remove repeated words in the second candidate keywords and merge keywords with similar semantics to obtain candidate keywords.

[0118] The three-dimensional correlation degree evaluation includes:

[0119] Calculate the cosine similarity of the candidate keywords with a preset core keyword library under a specified theme scenario;

[0120] Calculate the co-occurrence probability of the candidate keywords with action verbs under the specified theme scenario in the dialogue;

[0121] Match the business words corresponding to the candidate keywords with a preset business dictionary to obtain a preset weight value;

[0122] According to the cosine similarity, the co-occurrence probability, and the preset weight value, calculate the evaluation score.

[0123] The specific implementation of this step is as follows:

[0124] Taking a real estate theme project as an example, in order to quantify the real correlation between new words and real estate marketing and avoid the misclassification of irrelevant words (such as “hot weather” which may be high-frequency but not key), a three-dimensional correlation degree evaluation model is used to evaluate the candidate keywords, and the evaluation process is as shown in Figure 3 The specific three dimensions are as follows:

[0125] Semantic similarity (40%): Calculate the cosine similarity of the candidate keywords with a real estate core keyword library (such as “property”, “house type”, “price”); use a pre-trained Chinese embedding model (such as text2vec-base-chinese); threshold setting: greater than or equal to 0.65 is effective correlation.

[0126] Context dependence (30%): analyze the co-occurrence probability of the candidate keywords with marketing action verbs (“purchase”, “compare”, “recommend”, etc.) in the dialogue; based on sliding window co-occurrence statistics (window size = 5 words); threshold setting: greater than or equal to 0.6 is effective correlation.

[0127] Business weight (30%): match the enterprise preset business dictionary (project name, regional policy, company characteristic service, etc.); manually preset weight value (such as “Xinghui City” = 0.95, “new public housing policy” = 0.85); threshold setting: greater than or equal to 0.7 is effective correlation.

[0128] As shown in Table 1, when the three-dimensional comprehensive score is greater than or equal to 0.7, the candidate keyword is added to the dynamic word library; when the three-dimensional comprehensive score is greater than or equal to 0.5 and less than 0.7, the candidate keyword is manually reviewed to determine whether to be added to the dynamic word library; and when the three-dimensional comprehensive score is less than 0.5, the candidate keyword is discarded.

[0129]

[0130] Table 1

[0131] If the keyword in the dynamic word library is not used for subject classification within a preset number of days, the keyword will be removed from the dynamic word library.

Claims

1. A method for intelligent content filtering based on specified theme scenarios, characterized in that, Comprise: S1: optimize the zero sample classification model, obtain the optimized zero sample classification model, the zero sample classification model is classified according to the constructed knowledge graph prompt template, the knowledge graph prompt template includes theme label and theme label matching keyword, the optimization includes the second ASR output text is cut according to sentence, and the speaker identity is marked; The constructed knowledge graph prompt template is used for theme division to each sentence in the second ASR output text, and the corresponding preliminary theme label of each sentence is obtained; According to the continuity of the time stamp and the preliminary theme label of the sentence, the second ASR output text is divided into several segments;The sentences with similar semantics in the same segment are classified as positive samples, and the sentences with different segments or dissimilar semantics are classified as negative samples;According to the classified positive samples and negative samples, the contrast loss function is used to fine-tune the zero sample classification model, and the optimized zero sample classification model is obtained;The contrast loss function calculation expression is as follows: wherein, represents a contrastive loss, represents an embedding representation of a current sentence, represents an embedding representation of a positive sample sentence that is semantically similar to represents an embedding representation of a negative sample sentence, represents a cosine similarity function, represents a temperature hyperparameter used to adjust the degree of distribution sharpness, represents a sentence encoding function, represents a natural exponential function, represents all negative sample sentences, represents the th negative sample sentence;​ S2: the first ASR output text is classified by the optimized zero sample classification model, and the first irrelevant content text is obtained; S3: a dynamic word library is constructed, the relevance of the first ASR output text and the dynamic word library is calculated, and the second irrelevant content text is obtained according to the relevance, the dynamic word library is updated synchronously with the keyword in the knowledge graph prompt template, and is used for fine-tuning the optimized zero sample classification model according to the preset time; S4: the first irrelevant content text and the second irrelevant content text are compared and repeated, and the third irrelevant content text is obtained; S5: the third irrelevant content text is removed from the first ASR output text, and the filtered first ASR output text is obtained. 2.The method for filtering intelligent content under a specified theme scene based on the theme scene according to claim 1, characterized in that, Step S2 includes: The first ASR output text is classified by the optimized zero sample classification model, and the irrelevant content theme label, the irrelevant content text and the confidence are obtained, the confidence is the relevance probability of the irrelevant content text and the irrelevant content theme label; The irrelevant content text with confidence higher than the preset threshold is screened out, and the first irrelevant content text is obtained. 3.The method for filtering intelligent content under a specified theme scene based on the theme scene according to claim 1, characterized in that, Step S3 includes: A dynamic word library is constructed, and the relevance of the first ASR output text and the dynamic word library is calculated by using TF-IDF algorithm; The text in the first ASR output text with relevance lower than the preset threshold is screened out, and the second irrelevant content text is obtained.

4. The method for intelligent content filtering under specified theme based scenario as claimed in claim 1 wherein, The dynamic word library is constructed, including: The first ASR output text is extracted to obtain candidate keywords; Determine whether the candidate keywords are in the dynamic word library, if yes, do not perform three-dimensional correlation degree evaluation on the candidate keywords, if not, perform three-dimensional correlation degree evaluation on the candidate keywords; The candidate keywords are evaluated by three-dimensional correlation degree, and the evaluation score is obtained, if the evaluation score is greater than the first preset score, the candidate keywords are added to the dynamic word library, if the evaluation score is less than the first preset score and greater than the second preset score, the candidate keywords are manually reviewed, if the evaluation score is less than the second preset score, the candidate keywords are not added to the dynamic word library.

5. The method for intelligent content filtering under specified theme-based scenarios according to claim 4, characterized in that, The candidate keywords are obtained, including: The high-frequency keywords are extracted from the first ASR output text by using an N-gram model and a TF-IDF algorithm. The zero-sample keywords are extracted from the first ASR output text by using a KeyBERT model. The high-frequency keywords and the zero-sample keywords are merged to obtain candidate keywords.

6. The method for intelligent content filtering under specified theme-based scenarios according to claim 5, characterized in that, The merging includes: removing stop words and general words in the high-frequency keywords and the zero-sample keywords to obtain first candidate keywords; retaining keywords in the first candidate keywords that have a frequency ranking less than a preset ranking in the high-frequency keywords or the zero-sample keywords to obtain second candidate keywords; removing repeated keywords in the second candidate keywords and merging keywords with similar semantics to obtain the candidate keywords.

7. The method for intelligent content filtering under specified theme based scenario as claimed in claim 4 wherein, The three-dimensional correlation degree evaluation includes: calculating a cosine similarity of the candidate keywords and a core keyword library in a preset designated theme scenario; calculating a co-occurrence probability of the candidate keywords and action verbs in the designated theme scenario in a dialogue; matching a business word corresponding to the candidate keywords with a preset business dictionary to obtain a preset weight value; calculating an evaluation score according to the cosine similarity, the co-occurrence probability, and the preset weight value.

8. The method for intelligent content filtering under specified theme based scenarios as claimed in claim 1 wherein, If a keyword in the dynamic keyword library is not used for theme classification within a preset number of days, the keyword is removed from the dynamic keyword library.

Images