A mind map generation method, device and system based on natural language processing

Abstract

The present invention discloses a mind map generation method, device and system based on natural language processing, which collects voiceprint data in real time and transcribes it into text data through speech transcription technology, and performs vectorization processing and database storage on the text data; retrieves and inputs the first vector data from the database into a text weight calculation model based on a multi-head attention mechanism, calculates and obtains the first information weight value, repeats the above retrieval and calculation steps until all vector data in the database have their corresponding information weight values, sorts the vector data according to the information weight value to obtain a vector data table, and fills the data into the mind map template according to the vector data table to generate the corresponding mind map. The efficiency, accuracy, completeness and real-time performance of speech content recording are improved through speech collection and speech transcription technology; the speech content mind map generated according to the information weight value improves the timeliness and accuracy of the transmission of the core information of the speech.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a mind map generation method, device and system based on natural language processing. Background Art

[0002] Mind mapping helps people organize and summarize knowledge points. It is easy to extract the most core words from multiple texts as representative nodes, and it is also easy to see the relationship between each node and the knowledge classification from the mind map. In important meetings, drawing the speaker's words into a mind map can make the participants more clearly understand the knowledge structure of the content explained by the speaker in this meeting.

[0003] In the past, the content of meetings was usually memorized or recorded by humans, and then manually compiled into meeting minutes after the meeting. If it is to be compiled into a mind map, humans are required to accurately record the speaker's key points, sort out the logical order of the entire speech process, and finally draw a mind map manually. However, there is a high probability of errors and omissions when relying on humans to manually record the speaker's key points. Manually drawn mind maps naturally have many loopholes and their accuracy is also low. Summary of the invention

[0004] The present invention provides a method, device and system for generating a mind map based on natural language processing, so as to achieve the technical effect of fully automatic collection of voiceprint data and generation of a mind map, while improving the accuracy of the mind map.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a mind map generation method based on natural language processing, comprising the following steps:

[0006] In response to a signal generated by the mind map, voiceprint data is collected in real time, the voiceprint data is transcribed into text data through speech transcription technology, the text data is vectorized, and then the vectorized data is stored in a vector database;

[0007] Retrieving first vector data from a vector database, inputting the first vector data into a text weight calculation model based on a multi-head attention mechanism for weight calculation, outputting a corresponding first information weight value, and repeatedly retrieving second vector data from the vector database until each vector data in the vector database has a corresponding information weight value;

[0008] The vector data are sorted according to the information weight value to obtain a vector data table, and the vector data in the vector data table are sequentially filled into a preset mind map template to obtain and output a corresponding mind map.

[0009] After the mind map generation method provided by the present invention responds to the mind map generation signal, the system starts to collect voiceprint data in real time, and transcribes the collected voiceprint data into text data through voice transcription technology. Compared with manual recording of the speaker's speech content, the real-time transcription of voiceprint data into text data through real-time voice collection and voice transcription technology not only improves the accuracy and completeness of the speech content record, but also improves the real-time and efficiency of the record. After obtaining the text data, the system vectorizes it to facilitate subsequent model weight calculation, and also stores it in the vector database for subsequent system calls. At the same time, through real-time collection of voiceprint data and real-time transcription by voice transcription technology, when multiple people speak, there is no need to sort out the speeches one by one after the speeches are finished, which improves the convenience of summarizing the speech content and ensures the accuracy and completeness of the speech content record.

[0010] After generating vectorized data, the system will randomly retrieve the first vector data from the vector database, and input it into the text weight calculation model based on the multi-head attention mechanism for weight calculation, so as to determine the information weight value corresponding to each element in the first vector data, and perform weighted summation processing on each information weight value obtained by calculation, and calculate and determine the information weight value of the first vector data, that is, the first information weight value. After associating the first information weight value with the first vector data, the system can retrieve a random vector data other than the first vector data from the vector database, that is, the second vector data, and perform weight calculation on it until all vector data in the vector database are associated with a corresponding information weight value, and then determine that the basic weight calculation of the speech content is completed. By performing information weight calculation on all vector data in the vector database, the core points of the speaker's speech, that is, the key content of each sentence of the speaker, can be determined, which is convenient for the subsequent mind map generation and also improves the accuracy of the generated mind map.

[0011] After determining the information weight value of each vector data, all vector data in the vector database will be sorted according to the information weight value to obtain a vector data table, and then the data in the table will be filled into the user's pre-selected mind map template in sequence according to the order of the vector data table to obtain the generated mind map in sequence. The speech content mind map generated according to the information weight value, that is, the mind map is drawn based on the core points of the speaker, which can not only help the rest of the people who listen to the speech to summarize and organize the speaker's logic and knowledge, but also improve the timeliness and accuracy of the transmission of the core information of the speech.

[0012] As a preferred example, the inputting of the first vector data into a text weight calculation model based on a multi-head attention mechanism to perform weight calculation and outputting a corresponding first information weight value specifically includes:

[0013] The first vector data is used as a first input sequence to be linearly transformed and projected into a query matrix, a key matrix and a value matrix;

[0014] Split and distribute the query matrix, key matrix, and value matrix to multiple heads respectively, and calculate the attention score for each head;

[0015] Calculate the probability distribution of the attention score of each head respectively, and perform weighted summation of the score values ​​of each head according to the probability distribution result to obtain the weighted output;

[0016] The weighted outputs of all heads are integrated, and a linear transformation is performed on the integrated weighted outputs to obtain the first information weight value.

[0017] After obtaining the vectorized data, the system builds a text weight calculation model with a multi-head attention mechanism as the core to calculate the information weight value of the vector data in the vector database, so as to determine the information weight value of each vector data. The input sequence obtained after the vector data is serialized is linearly transformed and projected into three matrices: the query matrix, the key matrix, and the value matrix, and then multi-head allocation and attention calculation are performed based on the above three matrices.

[0018] The attention mechanism allows the model to focus the sequence data processing process on the part of the input sequence that is most relevant to the current task by calculating the importance of each element in the input sequence to the current element when processing sequence data. In the multi-head attention mechanism, the model does not calculate only a single attention weight, but calculates multiple attention weights in parallel, and each weight corresponds to a different "head". Therefore, through the weight calculation model based on the multi-head attention mechanism, it is possible to capture important information of sequence data at different positions and abstract levels.

[0019] As a preferred example, the first vector data is used as a first input sequence for linear transformation and projected to a query matrix, a key matrix and a value matrix, specifically:

[0020] The first vector data is serialized and embedded to obtain the first input sequence, and the query weight matrix, key weight matrix and value weight matrix are retrieved to perform a linear transformation on the first input sequence to obtain the query matrix, the key matrix and the value matrix.

[0021] The model performs a linear transformation on the first input sequence formed by the first vector data through a query weight matrix, a key weight matrix and a value weight matrix. The above mapping is a learnable parameter and is usually represented by different matrices. The input sequence is linearly transformed with the above three weight matrices, that is, the input sequence is projected to the query matrix, key matrix and value matrix through three different weight matrices respectively.

[0022] Since the self-attention mechanism will calculate the importance of each element in the first input sequence to the current element according to the given target item result, and assign different probability distribution values ​​to the items to form multiple sets of weights, namely, the mapping matrix, the items closer to the correct result will be assigned a higher calculation probability, which will provide higher calculation accuracy than the direct linear unit output.

[0023] As a preferred example, the query matrix, key matrix and value matrix are respectively divided and distributed to multiple heads, and the attention score is calculated for each head, specifically:

[0024] Performing a segmentation and dimensionality reduction operation on the query matrix, the key matrix, and the value matrix respectively, and distributing a plurality of sub-matrices obtained after the segmentation and dimensionality reduction to a plurality of heads of the model, so that each head contains at least one query sub-matrix, one key sub-matrix, and one value sub-matrix;

[0025] Each subset in the first input sequence is respectively assigned to each head of the model for processing, so that each head processes a corresponding subset, and the attention score of each head is calculated through the query submatrix and the key submatrix corresponding to each head.

[0026] The query matrix, key matrix and value matrix obtained by mapping are distributed to multiple heads through splitting operations, and the query matrix, key matrix and value matrix are reduced in dimension using a multi-head computing method. Each head processes a subset of the input sequence through the multi-head computing method, so that the model can capture information from different angles, that is, each head performs an information weight calculation method, and the information weight calculation of each head is achieved by calculating the attention scores of the query submatrix and key submatrix of each head.

[0027] Since the weight calculation model is based on a multi-head attention mechanism, the model does not calculate only a single attention weight, but calculates multiple attention weights in parallel. Each weight corresponds to a different "head", and each head calculates its corresponding attention score in parallel, providing a data basis for the subsequent calculation of the information weight value of each vector data.

[0028] As a preferred example, the sorting of the vector data according to the information weight value to obtain a vector data table specifically includes:

[0029] Sort the vector data in the vector database according to the information weight value to obtain an initial vector data table;

[0030] Calculate the weight value of each vector data in all the vector data in the vector database respectively by using the TF-IDF algorithm;

[0031] The information weight value of each of the vector data is weightedly calculated according to the specific weight value to obtain the weight value of the vector data, and then the vector data is sorted according to the weight value to output the vector data table.

[0032] After calculating the information weight values ​​corresponding to all vector data in the vector database, the system will sort the vector data in the vector database from high to low according to the calculated information weight values ​​to obtain an initial vector data table. Each vector data in the initial vector table has been sorted once according to the information weight value, and in order to further improve the clarity of the core information of the speech content in the output mind map, the system will also sort the vector data in the initial vector data table again according to the weight value of each vector data.

[0033] The TF-IDF algorithm is used to calculate the weight of each vector data in the vector database, that is, to calculate the weight of each vector data in the full text of the speech. The weight of the vector data is calculated by determining the number of times a certain vector data appears in the full text, and the weight of the vector data is calculated to achieve secondary weight calculation of the vector data. The weight of the vector data can be obtained by weighting the information weight of the vector data obtained by the weight value obtained by the calculation, and the vector data in the initial vector data table is sorted secondary according to the weight value obtained by the calculation, so as to further improve the clarity of the core information of the speech content in the mind map generated according to the vector data table, and the accuracy of determining the key points of the speech content.

[0034] As a preferred example, the step of sequentially filling the vector data in the vector data table into a preset mind map template specifically includes:

[0035] Performing a rationality test on the vector data table, outputting a corresponding test result and storing the vector data table as a first data table in a sample library;

[0036] If the detection result is that the rationality detection of the vector data table passes, then the corresponding mind map template is obtained by matching in the mind map template library according to the vector data table, and the vector data in the vector data table is filled into the mind map template according to the table order;

[0037] If the detection result is that the rationality check of the vector data table fails, the process returns to retrieving the first vector data from the vector database until the detection result is that the rationality check of the vector data table passes.

[0038] After secondary sorting of the vector data to obtain the vector data table, the system will also perform a rationality check on the vector data table to determine whether the output vector data table is reasonable. Before filling it into the mind map template to generate the corresponding mind map, the unreasonable vector data table will be excluded and returned to the step of calculating the information weight value of the vector data in the vector database until it is determined that the output vector data table meets the system preset rationality test.

[0039] After determining that the output vector data meets the rationality test of the system, the corresponding mind map template will be matched in the mind map template library according to the output vector data, and the vector data in the vector data table will be filled into the matched mind map template in order to obtain and output the mind map. The accuracy and practicality of the vector data table are further improved by performing secondary sorting and rationality testing on the vector data, and the accuracy of the mind map generated based on the vector data table after the test and the core points of the speech summarized by it are also effectively improved.

[0040] Correspondingly, the embodiment of the invention also provides a mind map generation device based on natural language processing, the mind map generation device includes a text data vectorization module, an information weight calculation module and a mind map generation module;

[0041] The text data vectorization module is used to respond to the mind map generation signal, collect voiceprint data in real time, transcribe the voiceprint data into text data through speech transcription technology, vectorize the text data, and then store the vectorized data in the vector database;

[0042] The information weight calculation module is used to retrieve first vector data from the vector database, input the first vector data into a preset text weight calculation model for weight calculation, output a corresponding first information weight value, and repeatedly retrieve second vector data from the vector database until each vector data in the vector database has a corresponding information weight value;

[0043] The mind map generation module is used to sort the vector data according to the information weight value to obtain a vector data table, and fill the vector data in the vector data table into a preset mind map template in sequence to obtain and output a corresponding mind map.

[0044] As a preferred example, the information weight calculation module inputs the first vector data into a preset text weight calculation model to perform weight calculation, and outputs a corresponding first information weight value, specifically including:

[0045] The first vector data is used as a first input sequence to be linearly transformed and projected into a query matrix, a key matrix and a value matrix;

[0046] Split and distribute the query matrix, key matrix, and value matrix to multiple heads respectively, and calculate the attention score for each head;

[0047] Calculate the probability distribution of the attention score of each head respectively, and perform weighted summation of the score values ​​of each head according to the probability distribution result to obtain the weighted output;

[0048] The weighted outputs of all heads are integrated, and a linear transformation is performed on the integrated weighted outputs to obtain the first information weight value.

[0049] Correspondingly, an embodiment of the present invention also provides a mind map generation system based on natural language processing, the mind map generation system includes a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor processes the computer program, it implements any of the above-mentioned mind map generation methods based on natural language processing.

[0050] Correspondingly, an embodiment of the present invention further provides a storage medium, on which a computer program is stored. The computer program is called and executed by a processor to implement any of the above-mentioned mind map generation methods based on natural language processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 : A flow chart of an embodiment of a method for generating a mind map based on natural language processing provided by the present invention;

[0052] Figure 2 : A structural schematic diagram of an embodiment of a mind map generation device based on natural language processing provided by the present invention. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Embodiment 1

[0055] Please refer to Figure 1 , which is a flow chart of an embodiment of a method for generating a mind map based on natural language processing provided by the present invention, including steps 101 to 103, each of which is specifically as follows:

[0056] Step 101: In response to a signal generated by a mind map, voiceprint data is collected in real time and the voiceprint data is transcribed into text data through speech transcription technology, the text data is vectorized, and then the vectorized data is stored in a vector database.

[0057] After the mind map generation method provided by the embodiment of the present invention responds to the mind map generation signal, the system starts to collect voiceprint data in real time, and transcribes the collected voiceprint data into text data through voice transcription technology. Compared with manual recording of the speaker's speech content, the real-time transcription of voiceprint data into text data through real-time voice collection and voice transcription technology not only improves the accuracy and completeness of the speech content record, but also improves the real-time and efficiency of the record. After obtaining the text data, the system vectorizes it to facilitate subsequent model weight calculation, and also stores it in the vector database for subsequent system calls. At the same time, through real-time collection of voiceprint data and real-time transcription by voice transcription technology, when multiple people speak, there is no need to sort out the speeches one by one after the speeches are finished, which improves the convenience of summarizing the speech content and ensures the accuracy and completeness of the speech content record.

[0058] In this embodiment, the system will transcribe the real-time collected speech content, namely, voiceprint data, into text data through the existing general voiceprint recognition or speech recognition and speech transcription technology, that is, transcribe the voiceprint data into text, and convert the text content into a vectorized form of a fixed length and store it in a vector database. That is, the system vectorizes the converted text content so that the text data is converted into several fixed-length n-dimensional vectors, and stores the vectorized data in the vector database for subsequent system calls. At the same time, by vectorizing the text data, the collected text data is also converted into a vectorized data form suitable for the text weight calculation model.

[0059] Step 102: Retrieve the first vector data from the vector database, and input the first vector data into a text weight calculation model based on a multi-head attention mechanism for weight calculation, output the corresponding first information weight value, and repeatedly retrieve the second vector data from the vector database until each vector data in the vector database has a corresponding information weight value.

[0060] After generating vectorized data, the system will randomly retrieve the first vector data from the vector database, and input it into the text weight calculation model based on the multi-head attention mechanism for weight calculation, so as to determine the information weight value corresponding to each element in the first vector data, and perform weighted summation processing on each information weight value obtained by calculation, and calculate and determine the information weight value of the first vector data, that is, the first information weight value. After associating the first information weight value with the first vector data, the system can retrieve a random vector data other than the first vector data from the vector database, that is, the second vector data, and perform weight calculation on it until all vector data in the vector database are associated with a corresponding information weight value, and then determine that the basic weight calculation of the speech content is completed. By performing information weight calculation on all vector data in the vector database, the core points of the speaker's speech, that is, the key content of each sentence of the speaker, can be determined, which is convenient for the subsequent mind map generation and also improves the accuracy of the generated mind map.

[0061] Specifically, in this embodiment, the first vector data is input into a text weight calculation model based on a multi-head attention mechanism to perform weight calculation, and the corresponding first information weight value is output, which specifically includes:

[0062] The first vector data is used as a first input sequence to be linearly transformed and projected into a query matrix, a key matrix and a value matrix;

[0063] Split and distribute the query matrix, key matrix, and value matrix to multiple heads respectively, and calculate the attention score for each head;

[0064] Calculate the probability distribution of the attention score of each head respectively, and perform weighted summation of the score values ​​of each head according to the probability distribution result to obtain the weighted output;

[0065] The weighted outputs of all heads are integrated, and a linear transformation is performed on the integrated weighted outputs to obtain the first information weight value.

[0066] After obtaining the vectorized data, the system builds a text weight calculation model with a multi-head attention mechanism as the core to calculate the information weight value of the vector data in the vector database, so as to determine the information weight value of each vector data. The input sequence obtained after the vector data is serialized is linearly transformed and projected into three matrices: the query matrix, the key matrix, and the value matrix, and then multi-head allocation and attention calculation are performed based on the above three matrices.

[0067] The attention mechanism allows the model to focus the sequence data processing process on the part of the input sequence that is most relevant to the current task by calculating the importance of each element in the input sequence to the current element when processing sequence data. In the multi-head attention mechanism, the model does not calculate only a single attention weight, but calculates multiple attention weights in parallel, and each weight corresponds to a different "head". Therefore, through the weight calculation model based on the multi-head attention mechanism, it is possible to capture important information of sequence data at different positions and abstract levels.

[0068] Furthermore, in this embodiment, the first vector data is used as a first input sequence for linear transformation and projected to a query matrix, a key matrix, and a value matrix, specifically:

[0069] The first vector data is serialized and embedded to obtain the first input sequence, and the query weight matrix, key weight matrix and value weight matrix are retrieved to perform a linear transformation on the first input sequence to obtain the query matrix, the key matrix and the value matrix.

[0070] The model performs a linear transformation on the first input sequence formed by the first vector data through a query weight matrix, a key weight matrix and a value weight matrix. The above mapping is a learnable parameter and is usually represented by different matrices. The input sequence is linearly transformed with the above three weight matrices, that is, the input sequence is projected to the query matrix, key matrix and value matrix through three different weight matrices respectively.

[0071] Since the self-attention mechanism will calculate the importance of each element in the first input sequence to the current element according to the given target item result, and assign different probability distribution values ​​to the items to form multiple sets of weights, namely, the mapping matrix, the items closer to the correct result will be assigned a higher calculation probability, which will provide higher calculation accuracy than the direct linear unit output.

[0072] In this embodiment, when the system performs weight calculation on the first vector data, it first performs serialized embedding representation on the first vector data, that is, embedding the first vector data as a first input sequence. In natural language processing, the input sequence may be an embedded representation of a word or a character. For a given first input sequence, the text weight calculation model based on the multi-head attention mechanism will first perform three different linear transformations on the first input sequence, namely the three weight matrices described in this embodiment, and map them to the query matrix (Q), key matrix (K) and value matrix (V) respectively. The mapping is a learnable parameter, which is usually represented by different matrices. The mapping linear transformation expression described in this embodiment is as follows:

[0073] Q=W Q ×XK=W K×XV=W V ×X

[0074] Where X is the first input sequence, W Q , W K and W V They are the query weight matrix, key weight matrix, and value weight matrix respectively.

[0075] Furthermore, in this embodiment, the query matrix, the key matrix, and the value matrix are divided and distributed to multiple heads, and the attention score is calculated for each head, specifically:

[0076] Performing a segmentation and dimensionality reduction operation on the query matrix, the key matrix, and the value matrix respectively, and distributing a plurality of sub-matrices obtained after the segmentation and dimensionality reduction to a plurality of heads of the model, so that each head contains at least one query sub-matrix, one key sub-matrix, and one value sub-matrix;

[0077] Each subset in the first input sequence is respectively assigned to each head of the model for processing, so that each head processes a corresponding subset, and the attention score of each head is calculated through the query submatrix and the key submatrix corresponding to each head.

[0078] The query matrix, key matrix and value matrix obtained by mapping are distributed to multiple heads through splitting operations, and the query matrix, key matrix and value matrix are reduced in dimension using a multi-head computing method. Each head processes a subset of the input sequence through the multi-head computing method, so that the model can capture information from different angles, that is, each head performs an information weight calculation method, and the information weight calculation of each head is achieved by calculating the attention scores of the query submatrix and key submatrix of each head.

[0079] Since the weight calculation model is based on a multi-head attention mechanism, the model does not calculate only a single attention weight, but calculates multiple attention weights in parallel. Each weight corresponds to a different "head", and each head calculates its corresponding attention score in parallel, providing a data basis for the subsequent calculation of the information weight value of each vector data.

[0080] In this embodiment, the obtained query matrix (Q), key matrix (K) and value matrix (V) are distributed to multiple heads through a split operation, that is, the above three vector matrices are reduced to one-n of the original dimensions for calculation, where n is the number of heads. By splitting, reducing the dimensions and distributing the three vector matrices, each head contains at least one query submatrix, one key submatrix and one value submatrix, so each head can process a subset of the input sequence, so that the model can capture information from different angles.

[0081] For the calculation of the attention score of each head, the attention score is calculated by the query submatrix and key submatrix of each head. The calculation method is usually dot product (or other similarity metrics). The specific calculation expression is as follows:

[0082] Attention Score=(Q T ×K) / (d k ) 1 / 2

[0083] Among them, d k is the dimension of the key vector, (d k ) 1 / 2 It is used to scale the dot product to prevent the gradient from disappearing or exploding. After calculating the attention score of each head, the system will also calculate the probability distribution of the attention score of each head, that is, apply the Softmax function to the attention score of each head to obtain the probability distribution, which is used to represent the importance of each input element to the current head. The calculation formula is as follows:

[0084] Attention Weights=Softmax(Attention Score)

[0085] After calculating the probability distribution results of each head, the system will perform weighted calculation on the score value of each head according to the probability distribution results, that is, the probability distribution obtained by Softmax is weighted and summed to obtain the weighted output. The calculation formula is as follows:

[0086] Head Output=Attention Weights×V

[0087] After obtaining the weighted output of each head through the above formulas, the system will integrate the weighted outputs of all heads and perform a linear transformation on them to obtain a first information weight value, which is the information weight value corresponding to the first vector data input. In the method of merging the outputs of all heads, this embodiment preferably concatenates the outputs of all heads, and then merges the information through a linear transformation of the concatenated results to obtain the final output, which is the first information weight value of this embodiment.

[0088] Step 103: sorting each of the vector data according to the information weight value to obtain a vector data table, and filling the vector data in the vector data table into a preset mind map template in sequence to obtain and output a corresponding mind map.

[0089] After determining the information weight value of each vector data, all vector data in the vector database will be sorted according to the information weight value to obtain a vector data table, and then the data in the table will be filled into the user's pre-selected mind map template in sequence according to the order of the vector data table to obtain the generated mind map in sequence. The speech content mind map generated according to the information weight value, that is, the mind map is drawn based on the core points of the speaker, which can not only help the rest of the people who listen to the speech to summarize and organize the speaker's logic and knowledge, but also improve the timeliness and accuracy of the transmission of the core information of the speech.

[0090] Specifically, in this embodiment, the vector data are sorted according to the information weight value to obtain a vector data table, which specifically includes:

[0091] Sort the vector data in the vector database according to the information weight value to obtain an initial vector data table;

[0092] Calculate the weight value of each vector data in all the vector data in the vector database respectively by using the TF-IDF algorithm;

[0093] The information weight value of each of the vector data is weightedly calculated according to the specific weight value to obtain the weight value of the vector data, and then the vector data is sorted according to the weight value to output the vector data table.

[0094] After calculating the information weight values ​​corresponding to all vector data in the vector database, the system will sort the vector data in the vector database from high to low according to the calculated information weight values ​​to obtain an initial vector data table. Each vector data in the initial vector table has been sorted once according to the information weight value, and in order to further improve the clarity of the core information of the speech content in the output mind map, the system will also sort the vector data in the initial vector data table again according to the weight value of each vector data.

[0095] The TF-IDF algorithm is used to calculate the weight of each vector data in the vector database, that is, to calculate the weight of each vector data in the full text of the speech. The weight of the vector data is calculated by determining the number of times a certain vector data appears in the full text, and the weight of the vector data is calculated to achieve secondary weight calculation of the vector data. The weight of the vector data can be obtained by weighting the information weight of the vector data obtained by the weight value obtained by the calculation, and the vector data in the initial vector data table is sorted secondary according to the weight value obtained by the calculation, so as to further improve the clarity of the core information of the speech content in the mind map generated according to the vector data table, and the accuracy of determining the key points of the speech content.

[0096] In this embodiment, when the system determines that all vector data in the vector database have been assigned their corresponding information weight values, it means that the system has collected all the speech content of the speaker in real time, and converted all the speech content into vector data and temporarily stored it in the vector database. Then, after the system determines that no subsequent speech content has been collected and all vector data have been configured with their corresponding information weight values, the TF-IDF algorithm will be used to calculate the proportion weight value of each vector data in the full text of the collected speech content, that is, the proportion weight value of each vector data in all the vector data in the vector database will be calculated, and the information weight value will be weighted according to the proportion weight value, and then the vector data will be re-sorted according to the calculated total weight value to obtain the corresponding vector data table.

[0097] Furthermore, in this embodiment, the vector data in the vector data table are sequentially filled into a preset mind map template, specifically including:

[0098] Performing a rationality test on the vector data table, outputting a corresponding test result and storing the vector data table as a first data table in a sample library;

[0099] If the detection result is that the rationality detection of the vector data table passes, then the corresponding mind map template is obtained by matching in the mind map template library according to the vector data table, and the vector data in the vector data table is filled into the mind map template according to the table order;

[0100] If the detection result is that the rationality check of the vector data table fails, the process returns to retrieving the first vector data from the vector database until the detection result is that the rationality check of the vector data table passes.

[0101] After secondary sorting of the vector data to obtain the vector data table, the system will also perform a rationality check on the vector data table to determine whether the output vector data table is reasonable. Before filling it into the mind map template to generate the corresponding mind map, the unreasonable vector data table will be excluded and returned to the step of calculating the information weight value of the vector data in the vector database until it is determined that the output vector data table meets the system preset rationality test.

[0102] After determining that the output vector data meets the rationality test of the system, the corresponding mind map template will be matched in the mind map template library according to the output vector data, and the vector data in the vector data table will be filled into the matched mind map template in order to obtain and output the mind map. The accuracy and practicality of the vector data table are further improved by performing secondary sorting and rationality testing on the vector data, and the accuracy of the mind map generated based on the vector data table after the test and the core points of the speech summarized by it are also effectively improved.

[0103] In this embodiment, in addition to performing rationality detection on the vector data table through the system's rationality detection algorithm, the system can also export and display it, and the user can perform manual detection on it. When the detection is unreasonable, the information weight value of the vector data can be recalculated by clicking the reset button preset by the system until the output is a result that satisfies the user.

[0104] Similarly, if the user is not satisfied with the output mind map, he or she can also manually select a satisfactory mind map template and input it into the system. The system will replace the matching mind map template with the mind map template selected by the user. Since the mind map template input by the user has a higher priority than the mind map template matched by the system, the system will give priority to the mind map template input by the user as the selected template for generating the mind map.

[0105] After the template is selected, the system will summarize and summarize the vector data according to the order of the vector data table and the vector data according to a certain level and logical relationship, and fill it into the mind map template to draw the output mind map, which will help participants to summarize the speaker's logic and organize the knowledge, greatly improving the timeliness and accuracy of the transmission of meeting key points information.

[0106] In order to better illustrate the working principle and step flow of the mind map generation method, device and system based on natural language processing of the present invention, it is possible but not limited to refer to the relevant records above.

[0107] Accordingly, see Figure 2 , Figure 2 This is a schematic diagram of the structure of an embodiment of a mind map generation device based on natural language processing provided by the present invention. Figure 2 As shown, the mind map generating device includes a text data vectorization module 201, an information weight calculation module 202 and a mind map generating module 203.

[0108] Among them, the text data vectorization module 201 is used to respond to the mind map generation signal, collect voiceprint data in real time and transcribe the voiceprint data into text data through speech transcription technology, vectorize the text data, and then store the vectorized data into the vector database.

[0109] The information weight calculation module 202 is used to retrieve first vector data from a vector database, and input the first vector data into a preset text weight calculation model for weight calculation, output the corresponding first information weight value, and repeatedly retrieve second vector data from the vector database until each vector data in the vector database has a corresponding information weight value.

[0110] Furthermore, the information weight calculation module 202 inputs the first vector data into a preset text weight calculation model to perform weight calculation, and outputs a corresponding first information weight value, specifically including:

[0111] The first vector data is linearly transformed as the first input sequence and projected to the query matrix, key matrix and value matrix; the query matrix, key matrix and value matrix are respectively divided and distributed to multiple heads, and the attention score is calculated for each head; the probability distribution of the attention score of each head is calculated, and the score value of each head is weighted and summed according to the probability distribution result to obtain a weighted output; the weighted outputs of all heads are integrated, and the integrated weighted output is linearly transformed to obtain the first information weight value.

[0112] Furthermore, the information weight calculation module 202 uses the first vector data as a first input sequence to perform linear transformation and project it to a query matrix, a key matrix and a value matrix, specifically:

[0113] The first vector data is serialized and embedded to obtain the first input sequence, and the query weight matrix, key weight matrix and value weight matrix are retrieved to perform a linear transformation on the first input sequence to obtain the query matrix, the key matrix and the value matrix.

[0114] Furthermore, the information weight calculation module 202 divides and distributes the query matrix, the key matrix and the value matrix to multiple heads respectively, and calculates the attention score for each head, specifically:

[0115] The query matrix, the key matrix and the value matrix are split and reduced in dimension respectively, and the multiple sub-matrices obtained after the split and reduced in dimension are distributed to the multiple heads of the model, so that each head contains at least one query sub-matrix, one key sub-matrix and one value sub-matrix; each subset in the first input sequence is distributed to each head of the model for processing, so that each head processes a corresponding subset, and the attention score of each head is calculated through the query sub-matrix and the key sub-matrix corresponding to each head.

[0116] The mind map generation module 203 is used to sort the vector data according to the information weight value to obtain a vector data table, and fill the vector data in the vector data table into a preset mind map template in sequence to obtain and output a corresponding mind map.

[0117] Furthermore, the mind map generation module 203 sorts each of the vector data according to the information weight value to obtain a vector data table, which specifically includes:

[0118] The vector data in the vector database are sorted according to the information weight value to obtain an initial vector data table; and the weight value of each vector data in all the vector data in the vector database is calculated by the TF-IDF algorithm;

[0119] The information weight value of each of the vector data is weightedly calculated according to the specific weight value to obtain the weight value of the vector data, and then the vector data is sorted according to the weight value to output the vector data table.

[0120] Furthermore, the mind map generation module 203 sequentially fills the vector data in the vector data table into a preset mind map template, specifically including:

[0121] Performing a rationality test on the vector data table, outputting a corresponding test result and storing the vector data table as a first data table in a sample library;

[0122] If the detection result is that the rationality test of the vector data table passes, the corresponding mind map template is obtained by matching in the mind map template library according to the vector data table, and the vector data in the vector data table is filled into the mind map template according to the table order; if the detection result is that the rationality test of the vector data table fails, the process returns to retrieving the first vector data from the vector database until the detection result is that the rationality test of the vector data table passes.

[0123] Correspondingly, an embodiment of the present invention also provides a mind map generation system based on natural language processing, the mind map generation system includes a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor processes the computer program, it implements any of the above-mentioned mind map generation methods based on natural language processing.

[0124] Correspondingly, an embodiment of the present invention further provides a storage medium, on which a computer program is stored. The computer program is called and executed by a processor to implement any of the above-mentioned mind map generation methods based on natural language processing.

[0125] In summary, the embodiments of the present invention provide a method, device and system for generating a mind map based on natural language processing, which collects voiceprint data in real time and transcribes it into text data through speech transcription technology, and performs vectorization processing and database storage on the text data; retrieves the first vector data from the database and inputs it into the text weight calculation model based on the multi-head attention mechanism, calculates and obtains the first information weight value, repeats the above retrieval and calculation steps until all vector data in the database have their corresponding information weight values, sorts the vector data according to the information weight value to obtain a vector data table, and fills the data into the mind map template according to the vector data table to generate the corresponding mind map. The efficiency, accuracy, completeness and real-time performance of speech content recording are improved through speech collection and speech transcription technology; the speech content mind map generated according to the information weight value improves the timeliness and accuracy of the transmission of the core information of the speech.

[0126] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A mind map generation method based on natural language processing, characterized in that: The following steps are involved: In response to a signal generated by the mind map, voiceprint data is collected in real time, the voiceprint data is transcribed into text data through speech transcription technology, the text data is vectorized, and then the vectorized data is stored in a vector database; Retrieving first vector data from a vector database, inputting the first vector data into a text weight calculation model based on a multi-head attention mechanism for weight calculation, outputting a corresponding first information weight value, and repeatedly retrieving second vector data from the vector database until each vector data in the vector database has a corresponding information weight value; The vector data are sorted according to the information weight value to obtain a vector data table, which is specifically: The vector data in the vector database are sorted according to the information weight value to obtain an initial vector data table; the weight value of each vector data in all the vector data in the vector database is calculated by the TF-IDF algorithm; the information weight value of each vector data is weightedly calculated according to the weight value to obtain the weight value of the vector data, and then the vector data is sorted according to the weight value, and the vector data table is output; The vector data in the vector data table are sequentially filled into a preset mind map template to obtain and output a corresponding mind map.

2. A mind map generation method based on natural language processing as claimed in claim 1, characterized in that: The step of inputting the first vector data into a text weight calculation model based on a multi-head attention mechanism to perform weight calculation and outputting a corresponding first information weight value specifically includes: The first vector data is used as a first input sequence to be linearly transformed and projected into a query matrix, a key matrix and a value matrix; Split and distribute the query matrix, key matrix, and value matrix to multiple heads respectively, and calculate the attention score for each head; Calculate the probability distribution of the attention score of each head respectively, and perform weighted summation of the score values ​​of each head according to the probability distribution result to obtain the weighted output; The weighted outputs of all heads are integrated, and a linear transformation is performed on the integrated weighted outputs to obtain the first information weight value.

3. A mind map generation method based on natural language processing as claimed in claim 2, characterized in that: The first vector data is used as a first input sequence for linear transformation and projected to a query matrix, a key matrix and a value matrix, specifically: The first vector data is serialized and embedded to obtain the first input sequence, and the query weight matrix, key weight matrix and value weight matrix are retrieved to perform a linear transformation on the first input sequence to obtain the query matrix, the key matrix and the value matrix.

4. A mind map generation method based on natural language processing as claimed in claim 2, characterized in that: The query matrix, key matrix and value matrix are divided and distributed to multiple heads respectively, and the attention score is calculated for each head, specifically: Performing a segmentation and dimensionality reduction operation on the query matrix, the key matrix, and the value matrix respectively, and distributing a plurality of sub-matrices obtained after the segmentation and dimensionality reduction to a plurality of heads of the model, so that each head contains at least one query sub-matrix, one key sub-matrix, and one value sub-matrix; Each subset in the first input sequence is respectively assigned to each head of the model for processing, so that each head processes a corresponding subset, and the attention score of each head is calculated through the query submatrix and the key submatrix corresponding to each head.

5. The method for generating a mind map based on natural language processing according to claim 1, characterized in that: The step of sequentially filling the vector data in the vector data table into a preset mind map template specifically includes: Performing a rationality test on the vector data table, outputting a corresponding test result and storing the vector data table as a first data table in a sample library; If the detection result is that the rationality detection of the vector data table passes, then the corresponding mind map template is obtained by matching in the mind map template library according to the vector data table, and the vector data in the vector data table is filled into the mind map template according to the table order; If the detection result is that the rationality check of the vector data table fails, the process returns to retrieving the first vector data from the vector database until the detection result is that the rationality check of the vector data table passes.

6. A mind map generation device based on natural language processing, characterized in that: The mind map generation device includes a text data vectorization module, an information weight calculation module and a mind map generation module; The text data vectorization module is used to respond to the mind map generation signal, collect voiceprint data in real time, transcribe the voiceprint data into text data through speech transcription technology, vectorize the text data, and then store the vectorized data in the vector database; The information weight calculation module is used to retrieve first vector data from the vector database, input the first vector data into a preset text weight calculation model for weight calculation, output a corresponding first information weight value, and repeatedly retrieve second vector data from the vector database until each vector data in the vector database has a corresponding information weight value; The mind map generation module is used to sort each of the vector data according to the information weight value to obtain a vector data table, specifically: sort each of the vector data in the vector database according to the information weight value to obtain an initial vector data table, calculate the proportion weight value of each of the vector data in all the vector data in the vector database by the TF-IDF algorithm, perform weighted calculation on the information weight value of each of the vector data according to the proportion weight value to obtain the weight value of the vector data, then sort the vector data according to the weight value, output the vector data table, and fill the vector data in the vector data table into the preset mind map template in turn to obtain and output the corresponding mind map.

7. A mind map generation device based on natural language processing as claimed in claim 6, characterized in that: The information weight calculation module inputs the first vector data into a preset text weight calculation model to perform weight calculation, and outputs a corresponding first information weight value, specifically including: The first vector data is used as a first input sequence to be linearly transformed and projected into a query matrix, a key matrix and a value matrix; Split and distribute the query matrix, key matrix, and value matrix to multiple heads respectively, and calculate the attention score for each head; Calculate the probability distribution of the attention score of each head respectively, and perform weighted summation of the score values ​​of each head according to the probability distribution result to obtain the weighted output; The weighted outputs of all heads are integrated, and a linear transformation is performed on the integrated weighted outputs to obtain the first information weight value.

8. A mind map generation system based on natural language processing, characterized in that: The mind map generation system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor processes the computer program, it implements a mind map generation method based on natural language processing as described in any one of claims 1 to 5.

9. A storage medium, characterized in that: The storage medium stores a computer program, which is called and executed by a processor to implement a mind map generation method based on natural language processing as described in any one of claims 1 to 5.

Images