A knowledge point extraction method and system based on large model

Abstract

The present invention discloses a knowledge point extraction method and system based on a large model, which comprises the following steps: obtaining a knowledge text to be extracted, and obtaining important words of the text through large model preprocessing; vectorizing the important words of the text obtained through preprocessing to obtain text word vectors, and obtaining a text vector data set through aggregation; clustering analysis of the text vector data set to divide the text vector data set into different clusters; analyzing the text word vectors, selecting keywords representing the theme of the cluster from each cluster as the knowledge theme of the cluster; analyzing the degree of association between each important word of the text and the knowledge theme in each cluster, and extracting the important words of the text that meet preset conditions as the knowledge points of each cluster; and continuously aggregating and updating the knowledge points extracted from each cluster to obtain a final knowledge point set. The present invention can extract useful knowledge information from text data, and help better understand the content and theme of the text data by continuously updating and improving the knowledge point set.

Description

Technical Field

[0001] The present invention relates to the technical field of text extraction, and in particular to a knowledge point extraction method and system based on a large model. Background Art

[0002] Large-scale model-based knowledge point extraction methods leverage the major breakthroughs achieved in the field of natural language processing in recent years, including technologies such as deep learning, natural language understanding, and representation learning. These methods aim to automatically extract meaningful knowledge points from massive amounts of text data to help people better understand and utilize text information. With the emergence of large pre-trained language models (such as GPT, BERT, etc.), researchers and engineers have begun to explore how to use these models to achieve more accurate and efficient knowledge point extraction. These large models can learn rich language knowledge and semantic representations by pre-training on large-scale corpora, enabling them to better understand and analyze text data.

[0003] However, traditional knowledge point extraction methods usually need to rely on manually designed features to represent text. These features may not be able to fully capture the complex semantic information of the text, resulting in the extracted knowledge points being incomplete and inaccurate. At the same time, traditional methods often perform poorly when processing long texts and complex contexts, and it is difficult to fully mine the rich semantic information in the text. In addition, traditional methods have limitations in processing semantic associations and contextual understanding, and cannot fully consider the complex semantic relationships between words, resulting in the extracted knowledge points being difficult to meet the high requirements for text understanding. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a knowledge point extraction method and system based on a large model, comprising:

[0005] Obtain the knowledge text to be extracted and pre-process it using a large model to obtain important words in the text;

[0006] Perform vectorization on the important words in the preprocessed text to obtain text word vectors, and then aggregate the text word vectors to obtain a text vector dataset;

[0007] Perform cluster analysis on the text vector dataset and divide the text vector dataset into different clusters based on the analysis results;

[0008] Analyze the text word vectors in each cluster and select the keywords representing the cluster theme from each cluster as the knowledge theme of the cluster;

[0009] Analyze the correlation between each important word in each cluster and the knowledge topic, and extract the important words in the text that meet the preset correlation conditions as the knowledge points of each cluster;

[0010] The knowledge points extracted from each cluster are continuously collected and updated to obtain the final knowledge point set.

[0011] Furthermore, the preprocessing of the knowledge text to be extracted by the large model includes:

[0012] The large model is used to clean the knowledge text to be extracted, remove special characters, punctuation marks, and HTML tags in the knowledge text to be extracted, and perform denoising on the knowledge text to be extracted;

[0013] Use the large model to perform word segmentation on the knowledge text to be extracted after denoising, dividing the text into words;

[0014] The large model removes stop words from the words after word segmentation, and performs part-of-speech analysis and syntactic analysis on the remaining words to determine the important words in the text related to the knowledge points;

[0015] The determined important words in the text are standardized.

[0016] Furthermore, the part-of-speech analysis and syntactic analysis of the remaining words to determine the important words in the text related to the knowledge point include:

[0017] Obtaining the parts of speech of the remaining words, determining the part-of-speech types of the remaining words, and determining the first scores of the remaining words according to a pre-set part-of-speech-score mapping relationship;

[0018] Obtaining the structure of the sentence in which the remaining words are located, determining the structural type of the sentence, and determining the second score of the remaining words based on a pre-set sentence structure-score mapping relationship;

[0019] Add the first score and the second score to obtain the total score of the remaining words;

[0020] The remaining words whose total scores exceed the preset score threshold are filtered out and determined to be important words in the text related to the knowledge point.

[0021] Furthermore, the important words in the text obtained by preprocessing are vectorized to obtain text word vectors, and the text word vectors are aggregated to obtain a text vector dataset, including:

[0022] The important words in the text obtained by preprocessing are mapped to a continuous vector space through word embedding technology, and the important words in the text are converted into low-dimensional dense text word vectors in the continuous vector space;

[0023] The converted text word vectors are aggregated to obtain a text vector dataset.

[0024] Furthermore, the text vector dataset is subjected to cluster analysis, and the text vector dataset is divided into different clusters according to the analysis results, including:

[0025] Obtain the number of text word vectors in the text vector data, and determine the number of clusters to be divided according to the number of text word vectors;

[0026] Preset several distance intervals and determine the distance value from each text word vector to the cluster center based on the K-Means clustering algorithm;

[0027] Determine the distance interval between each text word vector and the cluster center, and assign text word vectors with distance values ​​in different distance intervals to different clusters, thus completing the division of the text vector dataset into different clusters.

[0028] Furthermore, the text word vectors in each cluster are analyzed, and keywords representing the cluster theme are selected from each cluster as the knowledge theme of the cluster, including:

[0029] Determine the important words corresponding to each text word vector in each cluster, analyze the meaning of each important word in each cluster, and label each important word with corresponding information according to its meaning;

[0030] Extract the keywords of each information tag of each important word in each text, select the keywords that appear the most times to represent the cluster theme, and use them as the knowledge theme of the cluster.

[0031] Furthermore, the analysis of the correlation between each important word in each cluster and the knowledge theme, and extracting the important words in the text that meet the preset correlation conditions as the knowledge points of each cluster, includes:

[0032] Get the frequency of occurrence of each important word in the text in the corresponding cluster, and use the frequency as the coefficient of each important word in the text;

[0033] Represent the knowledge topic with a vector to obtain the knowledge topic vector, and calculate the relevance between each text word vector and the knowledge topic vector;

[0034] Calculate the degree of association between each important word in the text and the knowledge topic based on the coefficient of each important word in the text and the correlation between each text word vector and the knowledge topic vector;

[0035] The important words in the text whose correlation degree exceeds the preset correlation threshold are extracted as the knowledge points of each cluster.

[0036] Furthermore, the calculation formula for the degree of association between each important word in the text and the knowledge topic is:

[0037] L=G / 1-α,

[0038] Among them, L is the degree of association between each important word in the text and the knowledge topic, G is the correlation between each text word vector and the knowledge topic vector, and α is the coefficient of each important word in the text.

[0039] The present invention also provides a knowledge point extraction system based on a large model, comprising:

[0040] The acquisition module is used to obtain the knowledge text to be extracted and pre-process the knowledge text to be extracted through the large model to obtain important words in the text;

[0041] The processing module is used to vectorize the important words in the text obtained by preprocessing to obtain text word vectors, and to aggregate the text word vectors to obtain a text vector dataset;

[0042] A partitioning module is used to perform cluster analysis on the text vector dataset and divide the text vector dataset into different clusters based on the analysis results;

[0043] The selection module is used to analyze the text word vectors in each cluster and select the keywords representing the cluster theme from each cluster as the knowledge theme of the cluster;

[0044] The extraction module is used to analyze the degree of correlation between each important word in each cluster and the knowledge topic, and extract the important words in the text that meet the preset correlation conditions as the knowledge points of each cluster;

[0045] The integration module is used to continuously aggregate and update the knowledge points extracted from each cluster to obtain the final knowledge point set.

[0046] Compared with the prior art, the method and system for extracting knowledge points based on a large model in the embodiment of the present invention have the following beneficial effects:

[0047] By using a large model for text preprocessing, the present invention can effectively identify important words in the text and extract keywords based on the specific text content and context, thereby better reflecting the theme and content of the text;

[0048] The vectorization processing of word vectors of the present invention can be achieved through word embedding models. These models can help convert words in the text into vector representations with semantic information. The vector representation can better capture the semantic relationship between words, thereby better expressing the semantic information of the text;

[0049] The present invention performs cluster analysis on text vector datasets to help divide text data into different clusters, thereby better understanding the structure and theme of text data;

[0050] The present invention selects keywords representing cluster themes through keyword extraction technology, which can help better understand the theme and content of each cluster and provide a basis for subsequent knowledge point extraction;

[0051] The present invention analyzes the degree of association between important words in the text and knowledge topics, which can help determine which words are more important, and better reflect the association between important words in the text and knowledge topics, helping to extract the most representative knowledge points;

[0052] The present invention can help obtain a more complete and accurate knowledge point set by aggregating and updating the extracted knowledge points, which helps to better understand the content and theme of text data. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of the process structure of the knowledge point extraction method based on the large model in an embodiment of the present invention;

[0054] Figure 2 Schematic diagram of the composition of a knowledge point extraction system based on a large model in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0056] In the description of this application, it should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are the orientation or position relationship shown in the accompanying drawings, which are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the platform or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on this application.

[0057] The terms "second" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly specifying the number of the technical features indicated. Therefore, a feature specified with "second" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0058] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0059] like Figure 1 As shown, in an embodiment of the present application, a knowledge point extraction method based on a large model is provided, including: S100: obtaining the knowledge text to be extracted, and preprocessing the knowledge text to be extracted through the large model to obtain important text words; S200: vectorizing the important text words obtained by preprocessing to obtain text word vectors, and aggregating the text word vectors to obtain a text vector data set; S300: clustering analysis is performed on the text vector data set, and the text vector data set is divided into different clusters according to the analysis results; S400: analyzing the text word vectors in each cluster, and selecting keywords representing the theme of the cluster from each cluster as the knowledge theme of the cluster; S500: analyzing the degree of correlation between each important text word and the knowledge theme in each cluster, and extracting the important text words whose correlation degree reaches a preset condition as the knowledge point of each cluster; S600: continuously aggregating and updating the knowledge points extracted from each cluster to obtain a final knowledge point set.

[0060] Furthermore, the present invention can effectively identify important words in the text by using a large model for text preprocessing, and can extract keywords based on the specific text content and context, thereby better reflecting the theme and content of the text; the vectorization processing of the word vector of the present invention can be achieved through a word embedding model, which can help convert the words in the text into vector representations with semantic information, and the vector representation can better capture the semantic relationship between words, thereby better expressing the semantic information of the text; the present invention can help divide the text data into different clusters by clustering analysis of the text vector data set, thereby better understanding the structure and theme of the text data; the present invention selects keywords representing the cluster theme through keyword extraction technology, which can help better understand the theme and content of each cluster, and provide a basis for subsequent knowledge point extraction; the present invention analyzes the degree of correlation between important words in the text and knowledge themes, which can help determine which words are more important, and better reflects the correlation between important words in the text and knowledge themes, and helps extract the most representative knowledge points; the present invention can help obtain a more complete and accurate knowledge point set for the extracted knowledge points by aggregating and updating, which helps to better understand the content and theme of the text data.

[0061] In an embodiment of the present application, a knowledge point extraction method based on a large model is provided, wherein the knowledge text to be extracted is preprocessed by the large model, including: cleaning the knowledge text to be extracted by the large model, removing special characters, punctuation marks and HTML tags in the knowledge text to be extracted, and denoising the knowledge text to be extracted; performing word segmentation on the denoised knowledge text to be extracted by the large model, dividing the text into words; removing stop words in the words after word segmentation by the large model, and performing part-of-speech analysis and syntactic analysis on the remaining words to determine important words in the text related to the knowledge points; and standardizing the determined important words in the text.

[0062] Specifically, a large-scale model is used to clean the knowledge text to be extracted, removing special characters, punctuation, and HTML tags, and simultaneously performing denoising to remove noise information and irrelevant content to retain the key information in the text; the large-scale model is used to perform word segmentation on the cleaned and denoised text, dividing the text into words or phrases for subsequent semantic analysis and understanding; the large-scale model is used to filter stop words on the words after word segmentation, removing common stop words and retaining words with actual meaning and information content; the large-scale model is used to perform part-of-speech analysis and syntactic analysis on the remaining words to determine the role and grammatical relationship of the words in the sentence, thereby understanding the semantic structure of the text; through part-of-speech analysis and syntactic analysis, the text-related important words related to the knowledge points are determined, that is, the most representative and informative words in the text; the determined text-important words are standardized, including word form normalization and synonym replacement, to ensure the accuracy and consistency of the knowledge points. Through the above steps, using a large-scale model to perform comprehensive semantic analysis and understanding of the text can better extract the knowledge points in the text, providing strong support for subsequent information retrieval, knowledge management, and intelligent decision-making.

[0063] In an embodiment of the present application, a knowledge point extraction method based on a large model is provided, wherein part-of-speech analysis and syntactic analysis are performed on the remaining words to determine important text words related to the knowledge points, including: obtaining the part-of-speech of the remaining words, and judging the part-of-speech type of the remaining words, and determining the first score of the remaining words according to a preset part-of-speech-score mapping relationship; obtaining the structure of the sentence in which the remaining words are located, and judging the structure type of the sentence, and determining the second score of the remaining words according to a preset preset sentence structure-score mapping relationship; adding the first score and the second score to obtain the total score of the remaining words; screening out the remaining words whose total scores exceed the preset score threshold and determining them as important text words related to the knowledge points.

[0064] Specifically, a large-scale model is used to perform part-of-speech tagging on the remaining words to determine the part-of-speech type of each word; a first score is determined for each remaining word based on a pre-set part-of-speech-score mapping relationship, for example, different scores are assigned to parts of speech such as nouns, verbs, and adjectives to reflect their importance in the text; a large-scale model is used to perform structural analysis on the sentence in which the remaining words are located to determine the sentence structure type, and a second score is determined based on a pre-set sentence structure-score mapping relationship, for example, different scores are assigned to structures such as simple sentences, complex sentences, and parallel sentences to reflect the complexity and information content of the sentence; the first score is added to the second score to obtain the total score of the remaining words; and the remaining words whose total score exceeds the preset score threshold are screened out and determined as important words in the text related to the knowledge point. Through the above steps, using a large-scale model to perform part-of-speech and syntactic analysis on the text, combined with the preset score mapping relationship, it is possible to effectively determine important words in the text related to the knowledge point. This score calculation method can help screen out the most representative and informative words in the text, providing an effective means for knowledge point extraction.

[0065] In an embodiment of the present application, a knowledge point extraction method based on a large model is provided, in which the important words of the text obtained by preprocessing are vectorized to obtain text word vectors, and the text word vectors are aggregated to obtain a text vector dataset, including: mapping the important words of the text obtained by preprocessing to a continuous vector space through word embedding technology, and converting the important words of the text into low-dimensional dense text word vectors in the continuous vector space; and aggregating the text word vectors obtained after the conversion to obtain a text vector dataset.

[0066] Specifically, word embedding techniques (such as Word2Vec, Globes, FastText, etc.) are used to map the important words in the preprocessed text to a continuous vector space, converting each important word into a low-dimensional dense text word vector; the converted text word vectors are aggregated to obtain a text vector dataset, in which each important word in the text corresponds to a low-dimensional dense word vector; the word vectors corresponding to all important words in the text are combined into a dataset, in which each important word in the text corresponds to a low-dimensional dense vector representation. By mapping important words in the text to a continuous vector space, the semantic information and similarity relationship between words can be better captured, providing a richer and more effective feature representation for subsequent text similarity calculations, clustering analysis, classification tasks, etc. This text representation method based on word embedding has been widely used in the field of natural language processing and can improve the performance and effectiveness of text processing tasks.

[0067] In an embodiment of the present application, a knowledge point extraction method based on a large model is provided, which performs cluster analysis on a text vector dataset and divides the text vector dataset into different clusters based on the analysis results, including: obtaining the number of text word vectors in the text vector data, and determining the number of clusters to be divided based on the number of text word vectors; pre-setting a number of distance intervals, and determining the distance value of each text word vector to the cluster center based on the K-Means clustering algorithm; judging the distance interval of the distance value of each text word vector to the cluster center, and assigning text word vectors with distance values ​​in different distance intervals to different clusters, thereby completing the division of the text vector dataset into different clusters.

[0068] Specifically, the number of text word vectors in the text vector data is counted to determine the number of clusters to be divided, and several cluster centers are randomly determined. Several distance intervals are pre-set to divide the distance values ​​between the text word vectors and the cluster centers. Based on the K-Means clustering algorithm, the distance value from each text word vector to the cluster center is calculated, and the text word vectors are divided into the cluster corresponding to the closest cluster center. The distance interval of the distance value from each text word vector to the cluster center is determined, and text word vectors with distance values ​​in different distance intervals are assigned to different clusters. Based on the distance interval, the text vector dataset is divided into different clusters, each containing a group of similar text word vectors. The K-means clustering algorithm can be used to divide the text word vectors in the text vector dataset into different clusters, each containing text word vectors with similar characteristics. This can help discover potential patterns and structures in text data, providing strong support for further text classification, clustering analysis, and information retrieval.

[0069] In an embodiment of the present application, a knowledge point extraction method based on a large model is provided, in which the text word vectors in each cluster are analyzed, and keywords representing the theme of the cluster are selected from each cluster as the knowledge theme of the cluster, including: determining the important text words corresponding to each text word vector in each cluster, and analyzing the meaning of each important text word in each cluster, and marking each important text word with a corresponding information label according to the meaning; extracting the keywords of each information label of each important text word, and selecting the keywords with the most occurrences as the keywords representing the theme of the cluster, and using them as the knowledge theme of the cluster.

[0070] Specifically, for each cluster, determine the important text words corresponding to each text word vector, perform semantic analysis and meaning understanding on the important text words in each cluster to determine their importance and contribution in the text content; based on the analysis results, give each important text word a corresponding information label to reflect its specific meaning, theme or category in the text content; extract the keywords of each information label of each important text word, and select the keywords with the most occurrences to represent the theme of the cluster; use the selected keywords as the knowledge theme of the cluster, representing the main content and theme characteristics of the cluster. Through the above steps, after clustering the text vector data, by analyzing the important text words in each cluster, giving each important text word an information label, and extracting keywords to determine the knowledge theme of each cluster, it helps to discover the hidden themes and key information in the text data, and provides strong support for the extraction of knowledge themes and the understanding of text content.

[0071] In an embodiment of the present application, a knowledge point extraction method based on a large model is provided, which analyzes the degree of association between each important text word and the knowledge topic in each cluster, and extracts the important text words whose association degree reaches a preset condition as the knowledge point of each cluster, including: obtaining the frequency of occurrence of each important text word in the corresponding cluster, and using the frequency of occurrence as the coefficient of each important text word; representing the knowledge topic with a vector to obtain a knowledge topic vector, and calculating the correlation between each text word vector and the knowledge topic vector respectively; calculating the degree of association between each important text word and the knowledge topic based on the coefficient of each important text word and the correlation between each text word vector and the knowledge topic vector; extracting the important text words whose association degree exceeds a preset association threshold as the knowledge point of each cluster.

[0072] Specifically, for each cluster, the frequency of occurrence of each important text word in the cluster is counted as the coefficient of the important text word; the knowledge topic is represented by a vector, which can be a vector representation obtained by embedding the keywords of the knowledge topic, and the correlation between each text word vector and the knowledge topic vector is calculated using a correlation calculation method (such as cosine similarity); based on the coefficient of each important text word and the correlation between each text word vector and the knowledge topic vector, the degree of association between each important text word and the knowledge topic is calculated; the important text words whose correlation exceeds the preset correlation threshold are extracted as the knowledge points of each cluster. These important text words are highly correlated with the knowledge topic and represent the core topic and key information of the cluster. Through the above steps, the frequency of occurrence of important text words, the knowledge topic vector and the correlation of text word vectors can be used to calculate the degree of association between each important text word and the knowledge topic, thereby extracting important text words that are highly correlated with the knowledge topic as the knowledge points of each cluster, helping to discover key information that is highly correlated with the topic in text data, and providing strong support for the extraction of knowledge points and topic analysis.

[0073] In an embodiment of the present application, a knowledge point extraction method based on a large model is provided, wherein the calculation formula for the degree of association between each important word in the text and the knowledge topic is:

[0074] L=G / 1-α,

[0075] Among them, L is the degree of association between each important word in the text and the knowledge topic, G is the correlation between each text word vector and the knowledge topic vector, and α is the coefficient of each important word in the text.

[0076] like Figure 2 As shown, in an embodiment of the present application, a knowledge point extraction system based on a large model is provided, including: an acquisition module for acquiring the knowledge text to be extracted, and preprocessing the knowledge text to be extracted through the large model to obtain important text words; a processing module for vectorizing the important text words obtained by preprocessing to obtain text word vectors, and aggregating the text word vectors to obtain a text vector data set; a division module for clustering the text vector data set, and dividing the text vector data set into different clusters according to the analysis results; a selection module for analyzing the text word vectors in each cluster, and selecting keywords representing the theme of the cluster from each cluster as the knowledge theme of the cluster; an extraction module for analyzing the degree of correlation between each important text word and the knowledge theme in each cluster, and extracting the important text words whose correlation degree reaches a preset condition as the knowledge point of each cluster; an integration module for continuously aggregating and updating the knowledge points extracted from each cluster to obtain a final knowledge point set.

[0077] In summary, the embodiment of the present invention provides a knowledge point extraction method and system based on a large model, which includes: obtaining the knowledge text to be extracted, and obtaining important words of the text through large model preprocessing; vectorizing the important words of the text obtained through preprocessing to obtain text word vectors, and obtaining a text vector data set through aggregation; clustering analysis of the text vector data set, dividing the text vector data set into different clusters; analyzing the text word vectors, selecting keywords representing the theme of the cluster from each cluster as the knowledge theme of the cluster; analyzing the degree of correlation between each important word of the text and the knowledge theme in each cluster, and extracting the important words of the text that meet the preset conditions as the knowledge points of each cluster; continuously aggregating and updating the knowledge points extracted from each cluster to obtain a final knowledge point set. The present invention can extract useful knowledge information from text data, and help better understand the content and theme of text data by continuously updating and improving the knowledge point set.

[0078] Finally, it should be noted that it is apparent that those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, to the extent such modifications and variations fall within the scope of the claims and their equivalents, the present invention is intended to encompass such modifications and variations.

[0079] The above description is only an example of an embodiment of the present invention, but it does not limit the scope of the present invention. Any structural changes made according to the present invention, as long as they do not lose the essence of the present invention, should be considered to fall within the scope of protection of the present invention and be subject to restrictions. Those skilled in the art will clearly understand that for the convenience and simplicity of description, the specific working process and related instructions of the platform described above can refer to the corresponding process in the aforementioned platform embodiment, and will not be repeated here.

[0080] The term "comprise," "comprising," or any other similar term is intended to cover a non-exclusive inclusion such that a process, platform, article, or apparatus / platform that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, platform, article, or apparatus / platform.

[0081] Thus far, the technical solutions of the present invention have been described in conjunction with the further embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

[0082] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A knowledge point extraction method based on a large model, characterized in that: include: Obtain the knowledge text to be extracted and pre-process it using a large model to obtain important words in the text; Perform vectorization on the important words in the preprocessed text to obtain text word vectors, and then aggregate the text word vectors to obtain a text vector dataset; Perform cluster analysis on the text vector dataset and divide the text vector dataset into different clusters based on the analysis results; Analyze the text word vectors in each cluster and select the keywords representing the cluster theme from each cluster as the knowledge theme of the cluster; Analyze the correlation between each important word in each cluster and the knowledge topic, and extract the important words in the text that meet the preset correlation conditions as the knowledge points of each cluster; The knowledge points extracted from each cluster are continuously collected and updated to obtain the final knowledge point set; The preprocessing of the knowledge text to be extracted by the large model includes: The large model is used to clean the knowledge text to be extracted, remove special characters, punctuation marks, and HTML tags in the knowledge text to be extracted, and perform denoising on the knowledge text to be extracted; Use the large model to perform word segmentation on the knowledge text to be extracted after denoising, dividing the text into words; The large model removes stop words from the words after word segmentation, and performs part-of-speech analysis and syntactic analysis on the remaining words to determine the important words in the text related to the knowledge points; Standardize the identified important words in the text; The remaining words are subjected to part-of-speech analysis and syntactic analysis to determine important words in the text related to the knowledge point, including: Obtaining the parts of speech of the remaining words, determining the part-of-speech types of the remaining words, and determining the first scores of the remaining words according to a pre-set part-of-speech-score mapping relationship; Obtaining the structure of the sentence in which the remaining words are located, determining the structural type of the sentence, and determining the second score of the remaining words based on a pre-set sentence structure-score mapping relationship; Add the first score and the second score to obtain the total score of the remaining words; The remaining words whose total scores exceed the preset score threshold are filtered out and determined to be important words in the text related to the knowledge point; The analysis of the correlation between each important word in each cluster and the knowledge theme, and the extraction of important words in the text that meet the preset correlation conditions as the knowledge points of each cluster, includes: Get the frequency of occurrence of each important word in the text in the corresponding cluster, and use the frequency as the coefficient of each important word in the text; Represent the knowledge topic with a vector to obtain the knowledge topic vector, and calculate the relevance between each text word vector and the knowledge topic vector; Calculate the degree of association between each important word in the text and the knowledge topic based on the coefficient of each important word in the text and the correlation between each text word vector and the knowledge topic vector; The important words in the text whose correlation degree exceeds the preset correlation threshold are extracted as the knowledge points of each cluster.

2. A knowledge point extraction method based on a large model according to claim 1, characterized in that: The important words in the text obtained by preprocessing are vectorized to obtain text word vectors, and the text word vectors are aggregated to obtain a text vector dataset, including: The important words in the text obtained by preprocessing are mapped to a continuous vector space through word embedding technology, and the important words in the text are converted into low-dimensional dense text word vectors in the continuous vector space; The converted text word vectors are aggregated to obtain a text vector dataset.

3. The method for extracting knowledge points based on a large model according to claim 2, characterized in that: The text vector dataset is clustered and divided into different clusters according to the analysis results, including: Obtain the number of text word vectors in the text vector data, and determine the number of clusters to be divided according to the number of text word vectors; Preset several distance intervals and determine the distance value from each text word vector to the cluster center based on the K-Means clustering algorithm; Determine the distance interval between each text word vector and the cluster center, and assign text word vectors with distance values ​​in different distance intervals to different clusters, thus completing the division of the text vector dataset into different clusters.

4. The method for extracting knowledge points based on a large model according to claim 3, characterized in that: The text word vectors in each cluster are analyzed, and keywords representing the cluster theme are selected from each cluster as the knowledge theme of the cluster, including: Determine the important words corresponding to each text word vector in each cluster, analyze the meaning of each important word in each cluster, and label each important word with corresponding information according to its meaning; Extract the keywords of each information tag of each important word in each text, select the keywords that appear the most times to represent the cluster theme, and use them as the knowledge theme of the cluster.

5. The method for extracting knowledge points based on a large model according to claim 1, characterized in that: The calculation formula for the degree of association between each important word in the text and the knowledge topic is: L=G / 1-α, Among them, L is the degree of association between each important word in the text and the knowledge topic, G is the correlation between each text word vector and the knowledge topic vector, and α is the coefficient of each important word in the text.

6. A knowledge point extraction system based on a large model, characterized in that: include: The acquisition module is used to obtain the knowledge text to be extracted and pre-process the knowledge text to be extracted through the large model to obtain important words in the text; The processing module is used to vectorize the important words in the text obtained by preprocessing to obtain text word vectors, and to aggregate the text word vectors to obtain a text vector dataset; A partitioning module is used to perform cluster analysis on the text vector dataset and divide the text vector dataset into different clusters based on the analysis results; The selection module is used to analyze the text word vectors in each cluster and select the keywords representing the cluster theme from each cluster as the knowledge theme of the cluster; The extraction module is used to analyze the degree of correlation between each important word in each cluster and the knowledge topic, and extract the important words in the text that meet the preset correlation conditions as the knowledge points of each cluster; The integration module is used to continuously aggregate and update the knowledge points extracted from each cluster to obtain the final knowledge point set; The preprocessing of the knowledge text to be extracted by the large model includes: The large model is used to clean the knowledge text to be extracted, remove special characters, punctuation marks, and HTML tags in the knowledge text to be extracted, and perform denoising on the knowledge text to be extracted; Use the large model to perform word segmentation on the knowledge text to be extracted after denoising, dividing the text into words; The large model removes stop words from the words after word segmentation, and performs part-of-speech analysis and syntactic analysis on the remaining words to determine the important words in the text related to the knowledge points; Standardize the identified important words in the text; The remaining words are subjected to part-of-speech analysis and syntactic analysis to determine important words in the text related to the knowledge point, including: Obtaining the parts of speech of the remaining words, determining the part-of-speech types of the remaining words, and determining the first scores of the remaining words according to a pre-set part-of-speech-score mapping relationship; Obtaining the structure of the sentence in which the remaining words are located, determining the structural type of the sentence, and determining the second score of the remaining words based on a pre-set sentence structure-score mapping relationship; Add the first score and the second score to obtain the total score of the remaining words; The remaining words whose total scores exceed the preset score threshold are filtered out and determined to be important words in the text related to the knowledge point; The analysis of the correlation between each important word in each cluster and the knowledge theme, and the extraction of important words in the text that meet the preset correlation conditions as the knowledge points of each cluster, includes: Get the frequency of occurrence of each important word in the text in the corresponding cluster, and use the frequency as the coefficient of each important word in the text; Represent the knowledge topic with a vector to obtain the knowledge topic vector, and calculate the relevance between each text word vector and the knowledge topic vector; Calculate the degree of association between each important word in the text and the knowledge topic based on the coefficient of each important word in the text and the correlation between each text word vector and the knowledge topic vector; The important words in the text whose correlation degree exceeds the preset correlation threshold are extracted as the knowledge points of each cluster.

Images