Meeting information processing method and system based on big data

Abstract

The invention provides a big data-based conference information processing method and system. The method comprises the following steps of: obtaining a to-be-processed conference document and a conference document corpus to which the to-be-processed conference document belongs; calculating the position of the lexical item and the regional weighted word frequency; calculating intra-class inverse document frequency of the lexical item in the class cluster to which the lexical item belongs based on a division result of the document theme class cluster; obtaining an initial weight of the lexical item based on a product of the position and region weighted word frequency and the intra-class inverse document frequency; screening out core lexical items of which the weight is greater than a first preset threshold value; constructing a weighted co-occurrence network graph of the core lexical items; and taking the initial weight of each core lexical item as the initial weight in the weighted co-occurrence network graph, iteratively updating the weight of each core lexical item node through a preset graph sorting iterative algorithm according to the connection relationship of the nodes in the graph and the weight of the edge, and when the weight of each node converges or reaches a preset number of iterations, carrying out iterative updating on the weight of each core lexical item node. And taking the obtained weight as the weight of the core lexical item.

Description

Technical Field

[0001] This application belongs to the field of data processing, and in particular relates to a method and system for processing conference information based on big data. Background Technology

[0002] With the rapid development of information technology, the number of electronic documents generated by academic conferences has increased dramatically. These conference documents, such as papers, reports, and presentations, contain a wealth of professional knowledge and cutting-edge developments. Quickly and accurately extracting core information, such as keywords and themes, from these massive amounts of conference documents is crucial for scholars to conduct literature reviews, track academic hotspots, and construct knowledge graphs. Keyword extraction is one of the core technologies for achieving this goal, and traditional keyword extraction methods mainly rely on statistical models. Among them, the term frequency-inverse document frequency (TF-IDF) algorithm is the most widely used classic statistical method. If a word has a high frequency (TF) in a document but a low frequency (IDF) in other documents across the entire document corpus, then the word is considered to be of high importance to that document. However, it ignores the positional and structural information of words within the document, treating words in the title, abstract, and body text equally. Words that usually appear in the title and abstract are more likely to summarize the main idea of ​​the document. Moreover, the standard IDF calculation method is based on the entire corpus, which may lead to some technical terms that are very important in a specific subject area but appear frequently in the entire corpus being incorrectly assigned lower weights. TF-IDF is a bag-of-words model that calculates the weight of each word in isolation, completely ignoring the semantic associations and co-occurrence relationships between words.

[0003] Graph-based algorithms, such as the TextRank algorithm, have improved upon the above problems. These methods abstract a document into a word network graph and use an algorithm similar to PageRank to calculate the importance of nodes (words) in the graph, thus obtaining the co-occurrence relationships between words. However, when constructing the network graph, they often treat all candidate words equally, failing to fully utilize the statistical features of the words themselves as the initial basis for judging importance, or the edge weights are set too simply, failing to accurately measure the strength of association between words. Although some studies have attempted to combine TF-IDF with graph models, for example, using TF-IDF values ​​as the initial weights in graph ranking algorithms, these methods mostly follow the standard TF-IDF calculation framework, failing to address its inherent shortcomings in word position, document structure, and topic sensitivity. Existing technologies still have shortcomings in integrating the statistical features and semantic structure information of words, making it difficult to comprehensively and accurately assess the importance of core terms in conference documents. Summary of the Invention

[0004] The purpose of this invention is to provide a meeting information processing method and system based on big data, which solves the problem in existing meeting information processing where the importance of core terms is difficult to determine, leading to inaccurate extraction of core information. The method includes the following steps: S1: Obtain the meeting document to be processed and its associated meeting document corpus; for any term in the meeting document to be processed, calculate the position and region-weighted word frequency of the term based on the preset document region weight coefficient and the inverse position decay function of the term within the document region. The document area includes a title, a summary, and the main text, with the weight coefficients of the three decreasing sequentially. S2: Using a document vectorization model, convert each document in the conference document corpus into a document vector, and perform a hierarchical clustering algorithm on the document vectors to obtain multiple document topic clusters; based on the partitioning results of the document topic clusters, calculate the in-class inverse document frequency of the term within its respective cluster. ; S3: Weighted word frequency based on the aforementioned location and region With the inverse document frequency within the class The product of the terms yields the initial weight of the term; based on the initial weight, core terms with weights greater than a first preset threshold are selected. S4: Construct a weighted co-occurrence network graph of the core terms, where nodes are core terms. When the distance between any two core terms in a document is less than a second preset threshold, establish a connection edge between the corresponding nodes. The weight of the edge is the point mutual information value of the two core terms. S5: The initial weights of each core term are used as the initial weights in the weighted co-occurrence network graph. The weights of each core term node are iteratively updated according to the connection relationship of the nodes and the weights of the edges in the graph using a preset graph sorting iteration algorithm. When the weights of each node converge or reach the preset number of iterations, the obtained weights are used as the weights of the core terms.

[0005] Optionally, the step of calculating the position of the term and its region-weighted frequency based on a preset document region weight coefficient and a word's reverse position decay function within the document region is performed. ,include: For each occurrence of the term in the document, a single weighted value is calculated. The single weighted value is obtained by multiplying the weight coefficient of the region where the term is located by the positional decay value in the region. The position and region-weighted word frequency of the terms This is the sum of all single-weighted values ​​of the term in the document; The weighting coefficients for the document regions are as follows: title weighting coefficient is 1.0, abstract weighting coefficient is 0.8, and body text weighting coefficient is 0.5; the position decay value is obtained through a reverse position decay function. The calculation is performed where p is the position number of the term within the region, counted from back to front or from front to back, starting from 1.

[0006] Optionally, based on the document topic clustering results, the intra-category inverse document frequency of the term is calculated within its respective cluster. ,include: The Doc2Vec model was used to train the conference document corpus, and each document in the corpus was converted into a document vector. An agglomerative hierarchical clustering algorithm based on the Ward minimum variance criterion is applied to all document vectors to divide the documents into a predetermined number of document topic clusters; The inverse document frequency of the term within its respective cluster The calculation formula is: ,in The total number of documents within the cluster C to which term t belongs. Let be the number of documents containing term t within cluster C.

[0007] Optionally, the step of filtering out core terms with weights greater than a first preset threshold based on the initial weights includes: Calculate the arithmetic mean of the initial weights of all terms. ; The first preset threshold is set to the arithmetic mean. The preset multiple; Terms with an initial weight greater than the first preset threshold are selected as core terms.

[0008] Optionally, constructing the weighted co-occurrence network graph of the core terms includes: The second preset threshold is set to N. When the distance between any two core terms in the document is less than N, a connection edge is established between the corresponding nodes. The edge weights are calculated using the point mutual information (PMI) value, and the calculation formula is as follows: ; In this context, a sliding window is constructed using a continuous sequence of sentences of length N from the document. This represents the total number of sliding windows in the document. Core term and The number of sliding windows that appear together and Each contains core terms and The number of sliding windows.

[0009] Optionally, the step of iteratively updating the weights of each core term node using a preset graph sorting iterative algorithm based on the connection relationships of nodes and the weights of edges in the graph includes: A personalized graph sorting iterative algorithm is adopted, and a damping coefficient d is set; The formula for updating the weights of each core term node is as follows: ,in Let i be the weight of node i. Let i be the set of nodes adjacent to node i. Let be the weight of the edge connecting node i and node j. Let be the normalized initial weight of node i. The normalized initial weight is obtained by dividing the initial weight of the core term by the sum of the initial weights of all core terms. The iteration termination condition is set as follows: the change in the weight value of all nodes in two consecutive iterations is less than the preset value, or the number of iterations reaches the preset number.

[0010] This invention also provides a conference information processing system based on big data, comprising the following modules: The position- and region-weighted word frequency calculation module is used to acquire the meeting document to be processed and its associated meeting document corpus; for any word in the meeting document to be processed, based on a preset document region weight coefficient and a word's inverse position decay function within the document region, the position- and region-weighted word frequency of the word is calculated. The document area includes a title, a summary, and the main text, with the weight coefficients of the three decreasing sequentially. The intra-class inverse document frequency (INV) calculation module is used to convert each document in the conference document corpus into a document vector using a document vectorization model, and to perform a hierarchical clustering algorithm on the document vectors to obtain multiple document topic clusters; based on the partitioning results of the document topic clusters, the module calculates the intra-class INV of the term within its respective cluster. ; The core term filtering module is used to weight term frequencies based on the location and region. With the inverse document frequency within the class The product of the terms yields the initial weight of the term; based on the initial weight, core terms with weights greater than a first preset threshold are selected. The network graph construction module is used to construct a weighted co-occurrence network graph of the core terms. The nodes in the graph are core terms. When the sentence distance between any two core terms in the document is less than a second preset threshold, a connection edge is established between the corresponding nodes. The weight of the edge is the point mutual information value of the two core terms. The core term weight calculation module is used to take the initial weight of each core term as the initial weight in the weighted co-occurrence network graph. Through a preset graph sorting iteration algorithm, the weight of each core term node is iteratively updated according to the connection relationship of the nodes in the graph and the weight of the edges. When the weight of each node converges or reaches the preset number of iterations, the obtained weight is used as the weight of the core term.

[0011] Optionally, the step of calculating the position of the term and its region-weighted frequency based on a preset document region weight coefficient and a word's reverse position decay function within the document region is performed. ,include: For each occurrence of the term in the document, a single weighted value is calculated. The single weighted value is obtained by multiplying the weight coefficient of the region where the term is located by the positional decay value in the region. The position and region-weighted word frequency of the terms This is the sum of all single-weighted values ​​of the term in the document; The weighting coefficients for the document regions are as follows: title weighting coefficient is 1.0, abstract weighting coefficient is 0.8, and body text weighting coefficient is 0.5; the position decay value is obtained through a reverse position decay function. The calculation is performed where p is the position number of the term within the region, counted from back to front or from front to back, starting from 1.

[0012] Optionally, based on the document topic clustering results, the intra-category inverse document frequency of the term is calculated within its respective cluster. ,include: The Doc2Vec model was used to train the conference document corpus, and each document in the corpus was converted into a document vector. An agglomerative hierarchical clustering algorithm based on the Ward minimum variance criterion is applied to all document vectors to divide the documents into a predetermined number of document topic clusters; The inverse document frequency of the term within its respective cluster The calculation formula is: ,in The total number of documents within the cluster C to which term t belongs. Let be the number of documents containing term t within cluster C.

[0013] Optionally, the step of filtering out core terms with weights greater than a first preset threshold based on the initial weights includes: Calculate the arithmetic mean of the initial weights of all terms. ; The first preset threshold is set to the arithmetic mean. The preset multiple; Terms with an initial weight greater than the first preset threshold are selected as core terms.

[0014] Optionally, constructing the weighted co-occurrence network graph of the core terms includes: The second preset threshold is set to N. When the distance between any two core terms in the document is less than N, a connection edge is established between the corresponding nodes. The edge weights are calculated using the point mutual information (PMI) value, and the calculation formula is as follows: ; In this context, a sliding window is constructed using a continuous sequence of sentences of length N from the document. This represents the total number of sliding windows in the document. Core term and The number of sliding windows that appear together and Each contains core terms and The number of sliding windows.

[0015] Optionally, the step of iteratively updating the weights of each core term node using a preset graph sorting iterative algorithm based on the connection relationships of nodes and the weights of edges in the graph includes: A personalized graph sorting iterative algorithm is adopted, and a damping coefficient d is set; The formula for updating the weights of each core term node is as follows: ,in Let i be the weight of node i. Let i be the set of nodes adjacent to node i. Let be the weight of the edge connecting node i and node j. Let be the normalized initial weight of node i. The normalized initial weight is obtained by dividing the initial weight of the core term by the sum of the initial weights of all core terms. The iteration termination condition is set as follows: the change in the weight value of all nodes in two consecutive iterations is less than the preset value, or the number of iterations reaches the preset number.

[0016] This invention calculates word frequencies by introducing document region weights and term position decay functions, highlighting the value of important words in key regions such as titles and abstracts. Simultaneously, by performing topic clustering on documents and calculating inverse document frequencies within each cluster, it avoids the dilution of weights for core terms within specific topics by general domain terminology, thus improving the topic relevance of weight calculations. A weighted co-occurrence network is constructed using point mutual information, and iterative ranking is initiated with a high-quality initial weight graph, thereby integrating the statistical features, structural information, and semantic relationships between words. This invention can more accurately assess the importance of core terms in conference documents, improving the accuracy and effectiveness of keyword extraction. Attached Figure Description

[0017] Figure 1 This is a flowchart of the first specific embodiment; Figure 2 This is a schematic diagram of the position decay function; Figure 3 This is a schematic diagram of the document hierarchical clustering results; Figure 4 A diagram illustrating the filtering of core terms. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0019] The first specific embodiment provides a meeting information processing method based on big data, such as... Figure 1 As shown, it includes the following steps: S1: Obtain the meeting document to be processed and its associated meeting document corpus; for any term in the meeting document to be processed, calculate the position and region-weighted word frequency of the term based on the preset document region weight coefficient and the inverse position decay function of the term within the document region. The document area includes a title, a summary, and the main text, with the weight coefficients of the three decreasing sequentially. Conference documents in PDF or XML format were batch-acquired from academic databases such as IEEE Xplore and ACM Digital Library using web crawlers or API interfaces to form a corpus. Each document was then parsed to extract the text content of three structured regions: title, abstract, and body. Next, natural language processing tools, such as Jieba for Chinese, were used to segment the text in each region, and a predefined stop word list was loaded to remove meaningless terms such as modal particles and conjunctions. Simultaneously, stemming or lemmatization was performed on English terms to complete data preprocessing. Weighting coefficients were assigned to the title, abstract, and body regions, for example, 1.0, 0.8, and 0.5 respectively. The reverse position decay function was defined as the reciprocal of the position number of each occurrence of a term within that region. Figure 2 A schematic diagram of a positional decay function; weighted term frequency of a term. This is the sum of the products of the decay function values ​​and the corresponding weight coefficients for all occurrences of the term in the title, abstract, and body text regions; for example, if the term model appears in the 5th position of the title and the 20th position of the body text, then its... It is 9 / 40.

[0020] S2: Using a document vectorization model, convert each document in the conference document corpus into a document vector, and perform a hierarchical clustering algorithm on the document vectors to obtain multiple document topic clusters; based on the partitioning results of the document topic clusters, calculate the in-class inverse document frequency of the term within its respective cluster. ; A pre-trained BERT model was used as the document vectorization tool. The full text of each pre-processed document was input into the model, and the output vector corresponding to its CLS flag was extracted as the semantic vector representation of the document. Then, an agglomerative hierarchical clustering algorithm was applied to all document vectors, using Ward's minimum variance method as the distance metric between clusters. By analyzing the dendrogram and setting a distance threshold, the optimal number of clusters was determined, thereby dividing the corpus into multiple topic-related document sets, such as... Figure 3 As shown; in one embodiment, for any term, its intra-class inverse document frequency The term is calculated logarithmically as follows: the total number of documents in its cluster divided by the number of documents in that cluster that contain the term.

[0021] S3: Weighted word frequency based on the aforementioned location and region With the inverse document frequency within the class The product of the terms yields the initial weight of the term; based on the initial weight, core terms with weights greater than a first preset threshold are selected. The calculated value of S1 Calculated with S2 Multiply to obtain the initial weight value of each term; sort all terms in the document from high to low according to the initial weight, and select the top 30 terms after sorting as core terms, with 30 being the first preset threshold.

[0022] S4: Construct a weighted co-occurrence network graph of the core terms, where nodes are core terms. When the distance between any two core terms in a document is less than a second preset threshold, establish a connection edge between the corresponding nodes. The weight of the edge is the point mutual information value of the two core terms. The core terms selected by S3 are used as nodes in the network graph; a sliding window of size 5 is set as the sentence distance, i.e., the second preset threshold is 5; the entire document is traversed, and if two core terms appear in the same sliding window at the same time, an edge is established between their corresponding nodes; the weight of the edge is calculated using Point Mutual Information (PMI), which is the logarithm of the probability that the two core terms appear together in all sliding windows, divided by the product of their probabilities of appearing independently in any sliding window, where the probability is estimated by frequency.

[0023] S5: The initial weights of each core term are used as the initial weights in the weighted co-occurrence network graph. The weights of each core term node are iteratively updated according to the connection relationship of the nodes and the weights of the edges in the graph using a preset graph sorting iteration algorithm. When the weights of each node converge or reach the preset number of iterations, the obtained weights are used as the weights of the core terms.

[0024] The weighted TextRank algorithm is used as the graph sorting iterative algorithm. The initial weights of each core term calculated by S3 are assigned to the corresponding nodes in the graph as their initial scores. The damping coefficient is set to 0.85, and then iterative calculation is performed. In each iteration, the updated score of a node consists of two parts: one part is a fixed random jump probability, and the other part is the result of weighted summation of the scores of all its neighboring nodes according to the PMI weights of the connecting edges. This process is continuously iterated until the change in the scores of all nodes is less than a very small convergence threshold, such as 0.0001, or the preset maximum number of iterations, such as 100, is reached. At this point, the scores of each node are the final weights of the core terms.

[0025] In an optional embodiment, the position of the term and its region-weighted frequency are calculated based on a preset document region weight coefficient and a word's reverse position decay function within the document region. ,include: For each occurrence of the term in the document, a single weighted value is calculated. The single weighted value is obtained by multiplying the weight coefficient of the region where the term is located by the positional decay value in the region. The position and region-weighted word frequency of the terms This is the sum of all single-weighted values ​​of the term in the document; The weighting coefficients for the document regions are as follows: title weighting coefficient is 1.0, abstract weighting coefficient is 0.8, and body text weighting coefficient is 0.5; the position decay value is obtained through a reverse position decay function. The calculation is performed where p is the position number of the term within the region, counted from back to front or from front to back, starting from 1.

[0026] For example, a term has a weighting factor of 1.0 if it appears in the title area, 0.8 if it appears in the abstract area, and 0.5 in the body text area. Furthermore, the later the term appears in the text, the smaller the attenuation value becomes; for example, the last word in the area, with position index p1, has an attenuation value of 0.5. Multiplying the region weight by the position attenuation value yields a single-weighted value. Then, summing the single-weighted values ​​for all occurrences of the term in the document gives the final position-region weighted term frequency. In one embodiment, p can also be used as the position number of the term in the region, counted from front to back. Whether to use back to front or front to back depends on the specific format of the document. For example, in some meeting conclusions or summary paragraphs, the last sentence is the most important, so back to front is used; if the preceding sentences are the most important, front to back is used.

[0027] In an optional embodiment, the inverse document frequency of the term within its respective cluster is calculated based on the document topic clustering results. ,include: The Doc2Vec model was used to train the conference document corpus, and each document in the corpus was converted into a document vector. An agglomerative hierarchical clustering algorithm based on the Ward minimum variance criterion is applied to all document vectors to divide the documents into a predetermined number of document topic clusters; The inverse document frequency of the term within its respective cluster The calculation formula is: ,in The total number of documents within the cluster C to which term t belongs. Let be the number of documents containing term t within cluster C.

[0028] The Doc2Vec model is used to process the document corpus, transforming each document into a vector. Then, a Ward minimum variance criterion clustering algorithm is applied to divide these document vectors into different topic clusters. When calculating the inverse document frequency within a cluster, taking a specific cluster as an example, assuming that the cluster contains 100 documents, the frequency is... If the value is 100, and a certain term t appears in 10 of those documents, then... The value is 10. This term's... The value is the logarithm of the quotient of 100 divided by 11, reflecting the rarity of a term within a specific topic cluster.

[0029] In an optional embodiment, the step of filtering out core terms with weights greater than a first preset threshold based on the initial weights includes: Calculate the arithmetic mean of the initial weights of all terms. ; The first preset threshold is set to the arithmetic mean. The preset multiple; Terms with an initial weight greater than the first preset threshold are selected as core terms.

[0030] To select the most important core terms, it is necessary to calculate the arithmetic mean of the initial weights of all candidate terms. Set the first preset threshold to a specific multiple of this average value, such as 1.5 times. Identify all terms with an initial weight greater than 1.5 times the arithmetic mean as core terms, while filtering out terms with lower weights, such as... Figure 4 As shown.

[0031] In an optional embodiment, constructing the weighted co-occurrence network graph of the core terms includes: The second preset threshold is set to N. When the distance between any two core terms in the document is less than N, a connection edge is established between the corresponding nodes. The edge weights are calculated using the point mutual information (PMI) value, and the calculation formula is as follows: ; In this context, a sliding window is constructed using a continuous sequence of sentences of length N from the document. This represents the total number of sliding windows in the document. Core term and The number of sliding windows that appear together and Each contains core terms and The number of sliding windows.

[0032] The selected core terms are used as nodes in the network graph, and connections are established based on their co-occurrence relationships in the documents. For example, if two core terms are separated by less than N=5 sentences in the original text, an edge is drawn between the two nodes representing them. The weight of this edge, i.e., the strength of the association between the two terms, is quantified by the Point Mutual Information (PMI) value. It uses a sliding window of length 5 sentences to count the co-occurrence frequency of terms, thereby calculating the increase in their probability of co-occurrence relative to their individual probabilities of occurrence.

[0033] In an optional embodiment, the step of iteratively updating the weights of each core term node using a preset graph sorting iterative algorithm based on the connection relationships of nodes and the weights of edges in the graph includes: A personalized graph sorting iterative algorithm is adopted, and the damping coefficient d is set to 0.85; The formula for updating the weights of each core term node is as follows: ,in Let i be the weight of node i. Let i be the set of nodes adjacent to node i. Let be the weight of the edge connecting node i and node j. Let be the normalized initial weight of node i. The normalized initial weight is obtained by dividing the initial weight of the core term by the sum of the initial weights of all core terms. The iteration termination condition is set as follows: the change in the weight value of all nodes in two consecutive iterations is less than the preset value, or the number of iterations reaches the preset number.

[0034] A personalized graph sorting iterative algorithm is used to update the final weights of each core term node in the network. In each iteration, the weight update of a node depends not only on its own initial weight but also on the weights of all its neighboring nodes. The damping coefficient d is set to 0.85, meaning that 85% of a node's weight comes from the contributions of its neighbors. The iteration process continues until the change in the weight values ​​of all nodes in two consecutive calculations is less than, for example, 0.0001, or the total number of iterations reaches, for example, 100. At this point, the process stops and the final weight results are output. After obtaining the weights of the core terms, other processing can be performed, including summary generation.

[0035] This invention also provides a conference information processing system based on big data, comprising the following modules: The position- and region-weighted word frequency calculation module is used to acquire the meeting document to be processed and its associated meeting document corpus; for any word in the meeting document to be processed, based on a preset document region weight coefficient and a word's inverse position decay function within the document region, the position- and region-weighted word frequency of the word is calculated. The document area includes a title, a summary, and the main text, with the weight coefficients of the three decreasing sequentially. The intra-class inverse document frequency (INV) calculation module is used to convert each document in the conference document corpus into a document vector using a document vectorization model, and to perform a hierarchical clustering algorithm on the document vectors to obtain multiple document topic clusters; based on the partitioning results of the document topic clusters, the module calculates the intra-class INV of the term within its respective cluster. ; The core term filtering module is used to weight term frequencies based on the location and region. With the inverse document frequency within the class The product of the terms yields the initial weight of the term; based on the initial weight, core terms with weights greater than a first preset threshold are selected. The network graph construction module is used to construct a weighted co-occurrence network graph of the core terms. The nodes in the graph are core terms. When the sentence distance between any two core terms in the document is less than a second preset threshold, a connection edge is established between the corresponding nodes. The weight of the edge is the point mutual information value of the two core terms. The core term weight calculation module is used to take the initial weight of each core term as the initial weight in the weighted co-occurrence network graph. Through a preset graph sorting iteration algorithm, the weight of each core term node is iteratively updated according to the connection relationship of the nodes in the graph and the weight of the edges. When the weight of each node converges or reaches the preset number of iterations, the obtained weight is used as the weight of the core term.

[0036] In an optional embodiment, the position of the term and its region-weighted frequency are calculated based on a preset document region weight coefficient and a word's reverse position decay function within the document region. ,include: For each occurrence of the term in the document, a single weighted value is calculated. The single weighted value is obtained by multiplying the weight coefficient of the region where the term is located by the positional decay value in the region. The position and region-weighted word frequency of the terms This is the sum of all single-weighted values ​​of the term in the document; The weighting coefficients for the document regions are as follows: title weighting coefficient is 1.0, abstract weighting coefficient is 0.8, and body text weighting coefficient is 0.5; the position decay value is obtained through a reverse position decay function. The calculation is performed where p is the position number of the term within the region, counted from back to front or from front to back, starting from 1.

[0037] In an optional embodiment, the inverse document frequency of the term within its respective cluster is calculated based on the document topic clustering results. ,include: The Doc2Vec model was used to train the conference document corpus, and each document in the corpus was converted into a document vector. An agglomerative hierarchical clustering algorithm based on the Ward minimum variance criterion is applied to all document vectors to divide the documents into a predetermined number of document topic clusters; The inverse document frequency of the term within its respective cluster The calculation formula is: ,in The total number of documents within the cluster C to which term t belongs. Let be the number of documents containing term t within cluster C.

[0038] In an optional embodiment, the step of filtering out core terms with weights greater than a first preset threshold based on the initial weights includes: Calculate the arithmetic mean of the initial weights of all terms. ; The first preset threshold is set to the arithmetic mean. The preset multiple; Terms with an initial weight greater than the first preset threshold are selected as core terms.

[0039] In an optional embodiment, constructing the weighted co-occurrence network graph of the core terms includes: The second preset threshold is set to N. When the distance between any two core terms in the document is less than N, a connection edge is established between the corresponding nodes. The edge weights are calculated using the point mutual information (PMI) value, and the calculation formula is as follows: ; In this context, a sliding window is constructed using a continuous sequence of sentences of length N from the document. This represents the total number of sliding windows in the document. Core term and The number of sliding windows that appear together and Each contains core terms and The number of sliding windows.

[0040] In an optional embodiment, the step of iteratively updating the weights of each core term node using a preset graph sorting iterative algorithm based on the connection relationships of nodes and the weights of edges in the graph includes: A personalized graph sorting iterative algorithm is adopted, and the damping coefficient d is set to 0.85; The formula for updating the weights of each core term node is as follows: ,in Let i be the weight of node i. Let i be the set of nodes adjacent to node i. Let be the weight of the edge connecting node i and node j. Let be the normalized initial weight of node i. The normalized initial weight is obtained by dividing the initial weight of the core term by the sum of the initial weights of all core terms. The iteration termination condition is set as follows: the change in the weight value of all nodes in two consecutive iterations is less than the preset value, or the number of iterations reaches the preset number.

[0041] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0042] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0043] The method and electronic device for providing product object information provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for processing meeting information based on big data, characterized in that, Includes the following steps: S1: Obtain the meeting document to be processed and its associated meeting document corpus; for any term in the meeting document to be processed, calculate the position and region-weighted word frequency of the term based on the preset document region weight coefficient and the inverse position decay function of the term within the document region. The document area includes a title, a summary, and the main text, with the weight coefficients of the three decreasing sequentially. S2: Using a document vectorization model, convert each document in the conference document corpus into a document vector, and perform a hierarchical clustering algorithm on the document vectors to obtain multiple document topic clusters; based on the partitioning results of the document topic clusters, calculate the in-class inverse document frequency of the term within its respective cluster. ; S3: Weighted word frequency based on the aforementioned location and region With the inverse document frequency within the class The product of these terms yields the initial weight of the term; Based on the initial weights, core terms with weights greater than the first preset threshold are selected. S4: Construct a weighted co-occurrence network graph of the core terms, where nodes are core terms. When the distance between any two core terms in a document is less than a second preset threshold, establish a connection edge between the corresponding nodes. The weight of the edge is the point mutual information value of the two core terms. S5: The initial weights of each core term are used as the initial weights in the weighted co-occurrence network graph. The weights of each core term node are iteratively updated according to the connection relationship of the nodes and the weights of the edges in the graph using a preset graph sorting iteration algorithm. When the weights of each node converge or reach the preset number of iterations, the obtained weights are used as the weights of the core terms.

2. The method according to claim 1, characterized in that, The method calculates the position of the term and its region-weighted frequency based on a preset document region weight coefficient and a word's reverse position decay function within the document region. ,include: For each occurrence of the term in the document, a single weighted value is calculated. The single weighted value is obtained by multiplying the weight coefficient of the region where the term is located by the positional decay value in the region. The position and region-weighted word frequency of the terms This is the sum of all single-weighted values ​​of the term in the document; The weighting coefficients for the document regions are as follows: title weighting coefficient is 1.0, abstract weighting coefficient is 0.8, and body text weighting coefficient is 0.5; the position decay value is obtained through a reverse position decay function. The calculation is performed where p is the position number of the term within the region, counted from back to front or from front to back, starting from 1.

3. The method according to claim 1, characterized in that, Based on the document topic clustering results, the intra-category inverse document frequency of the term is calculated within its respective cluster. ,include: The Doc2Vec model was used to train the conference document corpus, and each document in the corpus was converted into a document vector. An agglomerative hierarchical clustering algorithm based on the Ward minimum variance criterion is applied to all document vectors to divide the documents into a predetermined number of document topic clusters; The inverse document frequency of the term within its respective cluster The calculation formula is: ,in The total number of documents within the cluster C to which term t belongs. Let be the number of documents containing term t within cluster C.

4. The method according to claim 1, characterized in that, The process of filtering out core terms with weights greater than a first preset threshold based on the initial weights includes: Calculate the arithmetic mean of the initial weights of all terms. ; The first preset threshold is set to the arithmetic mean. The preset multiple; Terms with an initial weight greater than the first preset threshold are selected as core terms.

5. The method according to claim 1, characterized in that, The construction of the weighted co-occurrence network graph of the core terms includes: The second preset threshold is set to N. When the distance between any two core terms in the document is less than N, a connection edge is established between the corresponding nodes. The edge weights are calculated using the point mutual information (PMI) value, and the calculation formula is as follows: ; In this context, a sliding window is constructed using a continuous sequence of sentences of length N from the document. This represents the total number of sliding windows in the document. Core term and The number of sliding windows that appear together and Each contains core terms and The number of sliding windows.

6. The method according to claim 1, characterized in that, The step involves iteratively updating the weights of each core term node using a preset graph sorting iterative algorithm, based on the connection relationships between nodes and the weights of edges in the graph. This includes: A personalized graph sorting iterative algorithm is adopted, and a damping coefficient d is set; The formula for updating the weights of each core term node is as follows: ,in Let i be the weight of node i. Let i be the set of nodes adjacent to node i. Let be the weight of the edge connecting node i and node j. Let be the normalized initial weight of node i. The normalized initial weight is obtained by dividing the initial weight of the core term by the sum of the initial weights of all core terms. The iteration termination condition is set as follows: the change in the weight value of all nodes in two consecutive iterations is less than the preset value, or the number of iterations reaches the preset number.

7. A conference information processing system based on big data, characterized in that, Includes the following modules: The position- and region-weighted word frequency calculation module is used to acquire the meeting document to be processed and its associated meeting document corpus; for any word in the meeting document to be processed, based on a preset document region weight coefficient and a word's inverse position decay function within the document region, the position- and region-weighted word frequency of the word is calculated. The document area includes a title, a summary, and the main text, with the weight coefficients of the three decreasing sequentially. The intra-class inverse document frequency (INV) calculation module is used to convert each document in the conference document corpus into a document vector using a document vectorization model, and to perform a hierarchical clustering algorithm on the document vectors to obtain multiple document topic clusters; based on the partitioning results of the document topic clusters, the module calculates the intra-class INV of the term within its respective cluster. ; The core term filtering module is used to weight term frequencies based on location and region. With the inverse document frequency within the class The product of these terms yields the initial weight of the term; Based on the initial weights, core terms with weights greater than the first preset threshold are selected. The network graph construction module is used to construct a weighted co-occurrence network graph of the core terms. The nodes in the graph are core terms. When the sentence distance between any two core terms in the document is less than a second preset threshold, a connection edge is established between the corresponding nodes. The weight of the edge is the point mutual information value of the two core terms. The core term weight calculation module is used to take the initial weight of each core term as the initial weight in the weighted co-occurrence network graph. Through a preset graph sorting iteration algorithm, the weight of each core term node is iteratively updated according to the connection relationship of the nodes in the graph and the weight of the edges. When the weight of each node converges or reaches the preset number of iterations, the obtained weight is used as the weight of the core term.

8. The system according to claim 7, characterized in that, The method calculates the position of the term and its region-weighted frequency based on a preset document region weight coefficient and a word's reverse position decay function within the document region. ,include: For each occurrence of the term in the document, a single weighted value is calculated. The single weighted value is obtained by multiplying the weight coefficient of the region where the term is located by the positional decay value in the region. The position and region-weighted word frequency of the terms This is the sum of all single-weighted values ​​of the term in the document; The weighting coefficients for the document regions are as follows: title weighting coefficient is 1.0, abstract weighting coefficient is 0.8, and body text weighting coefficient is 0.5; the position decay value is obtained through a reverse position decay function. The calculation is performed where p is the position number of the term within the region, counted from back to front or from front to back, starting from 1.

9. The system according to claim 7, characterized in that, Based on the document topic clustering results, the intra-category inverse document frequency of the term is calculated within its respective cluster. ,include: The Doc2Vec model was used to train the conference document corpus, and each document in the corpus was converted into a document vector. An agglomerative hierarchical clustering algorithm based on the Ward minimum variance criterion is applied to all document vectors to divide the documents into a predetermined number of document topic clusters; The inverse document frequency of the term within its respective cluster The calculation formula is: ,in The total number of documents within the cluster C to which term t belongs. Let be the number of documents containing term t within cluster C.

10. The system according to claim 7, characterized in that, The process of filtering out core terms with weights greater than a first preset threshold based on the initial weights includes: Calculate the arithmetic mean of the initial weights of all terms. ; The first preset threshold is set to the arithmetic mean. The preset multiple; Terms with an initial weight greater than the first preset threshold are selected as core terms.

Images