Text similarity analysis method and device
By generating text set vectors and using bidirectional long and short-term memory network encoding, combining the attention between the sentences themselves, the problem of inaccurate text similarity in the prior art is solved, and higher accuracy is achieved.
Patent Information
- Application Number
- CN202411739981.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-18
AI Technical Summary
In determining text similarity, prior art only focuses on the attention of the sentence itself, resulting in inaccurate text similarity.
By generating text set vectors, converting them into two-dimensional matrix using word2vec word vector model, and then re-encoding the text set vector matrix using a bidirectional long and short-term memory network to obtain the target text sequence. Then, based on the target text sequence, the sentence's own attention and the attention between the sentences are spliced to determine the similarity between the texts.
Using two-way semantic information, we capture the dependence of words in the sentence and the weight of words, and perform forward and backward calculations on the input at each moment, which significantly improves the accuracy of text similarity.
Smart Images
Figure CN119646185A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of data analysis, and in particular to a text similarity analysis method and device. Background Art
[0002] The rapid development of financial technology has led to an increasing demand for technology in the financial industry. More and more financial institutions are also applying artificial intelligence technology to improve their competitiveness and customer service levels. With the use of intelligent question-answering tools in the field of customer service, the intelligent question-answering system has also helped commercial banks achieve 24-hour intelligent customer service, becoming a key step in the digital transformation of commercial banks.
[0003] The bank knowledge base is a collection of various resources inside and outside the banking industry, including business rules, product introductions, risk compounding, customer service and business management knowledge, which can effectively improve the efficiency and competitiveness of banks, meet the diverse needs of customers, and promote the innovation and development of banks. Text similarity methods can help banks realize the construction, management, retrieval, and recommendation of knowledge bases. For example, the text similarity between customer questions and knowledge base questions can be calculated to realize intelligent customer service functions; the credit risk assessment function can be realized by calculating the text similarity between the information provided by users when applying for loans and the credit rules in the knowledge base; the news recommendation function can be realized by calculating the text similarity between the news browsed by users and the news in the knowledge base.
[0004] By using artificial intelligence technology, we can introduce intelligent question-answering systems in commercial banks, and automatically identify and answer user questions by calculating text similarity, thereby improving the service efficiency and customer satisfaction of commercial banks. The intelligent question-answering system can quickly match answers in the knowledge base or generate new answers based on user questions to meet user information needs; it can also provide personalized wealth management suggestions based on user risk preferences, consumption habits and life scene characteristics; it can also handle some highly professional, logical and standardized business issues, and complete account inquiries and credit approvals.
[0005] In the prior art, convolutional neural networks and multi-attention mechanisms are used to extract features of text and classify it. It includes a training set acquisition step, a text segmentation step, a word vector training step, a sentence splitting step, a word vector conversion step, a sentence-level convolutional neural network step; a sentence-level attention mechanism step; a sentence-level full connection step, a document processing step, and a classification step, including linearly transforming the document feature vector obtained in the document processing step, and then using a softmax function to generate the probability of each classification on the class set C. The model is designed into two parts, namely the sentence level and the document level. Sentence features are first extracted at the sentence level, and then document features are extracted at the document level for classification. However, this method only focuses on the attention of the sentence itself, resulting in inaccurate text similarity.
[0006] In summary, how to improve the accuracy of confirming text similarity is a problem that technical personnel in this field need to solve urgently. Summary of the invention
[0007] In view of this, the present application provides a text similarity analysis method and device, aiming to improve the accuracy of confirming text similarity.
[0008] In a first aspect, the present application provides a text similarity analysis method, comprising:
[0009] Generate text set vectors based on knowledge base data;
[0010] Through the word2vec word vector model, the one-dimensional vector text set vector is converted into a two-dimensional text set vector matrix;
[0011] Re-encode the text set vector matrix using a bidirectional long short-term memory network to obtain a target text sequence;
[0012] Based on the target text sequence, the attention of the sentences themselves and the attention between the sentences in the text are concatenated to determine the similarity between the texts.
[0013] Optionally, generating a text set vector according to the knowledge base data includes:
[0014] Get knowledge base data;
[0015] The text in the knowledge base data is digitized to obtain a text set vector.
[0016] Optionally, before digitizing the text in the knowledge base data to obtain the text set vector, the method further includes:
[0017] Cleaning the knowledge base data to obtain cleaned knowledge base data;
[0018] The step of digitizing the text in the knowledge base data to obtain a text set vector includes:
[0019] The text in the cleaned knowledge base data is digitized to obtain the text set vector.
[0020] Optionally, after generating the text set vector according to the knowledge base data, the method further includes:
[0021] The data whose data dimension in the text set vector exceeds a preset length is removed.
[0022] Optionally, the step of re-encoding the text set vector matrix using a bidirectional long short-term memory network to obtain a target text sequence includes:
[0023] Use the hidden layer to extract features from the LSTM forward input sequence and the LSTM reverse input sequence to obtain the forward latent vector and the reverse latent vector;
[0024] The forward latent vector and the reverse latent vector are concatenated to obtain the target text sequence.
[0025] In a second aspect, the present application provides a text similarity analysis device, comprising:
[0026] A generation module, used to generate text set vectors based on knowledge base data;
[0027] The conversion module is used to convert the one-dimensional vector text set vector into a two-dimensional text set vector matrix through the word2vec word vector model;
[0028] An encoding module, used for re-encoding the text set vector matrix using a bidirectional long short-term memory network to obtain a target text sequence;
[0029] A splicing module is used to splice the attention of the sentences themselves and the attention between sentences in the text based on the target text sequence to determine the similarity between the texts.
[0030] Optionally, the generating module includes:
[0031] An acquisition unit, used for acquiring knowledge base data;
[0032] The digitization unit digitizes the text in the knowledge base data to obtain a text set vector.
[0033] Optionally, the device further comprises:
[0034] A cleaning unit, used for cleaning the knowledge base data to obtain cleaned knowledge base data;
[0035] The digitization unit comprises:
[0036] The digitization subunit is used to digitize the text in the cleaned knowledge base data to obtain the text set vector.
[0037] Optionally, the device further comprises:
[0038] The elimination module is used to eliminate the data whose data dimension exceeds the preset length in the text set vector.
[0039] Optionally, the encoding module includes:
[0040] A feature extraction unit is used to extract features of the LSTM forward input sequence and the LSTM reverse input sequence using the hidden layer to obtain a forward latent vector and a reverse latent vector;
[0041] A concatenation unit is used to concatenate the forward latent vector and the reverse latent vector to obtain the target text sequence.
[0042] In a third aspect, an embodiment of the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the text similarity analysis method described in any one of the implementation methods in the first aspect of the embodiment of the present application is implemented.
[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the text similarity analysis method introduced in any one of the implementation methods in the first aspect of the embodiment of the present application.
[0044] The present application provides a method for analyzing text similarity. When executing the method, a text set vector is first generated based on the knowledge base data, and then the text set vector of the one-dimensional vector is converted into a two-dimensional text set vector matrix through the word2vec word vector model, and then the text set vector matrix is re-encoded using a bidirectional long short-term memory network to obtain a target text sequence, and finally, based on the target text sequence, the attention of the sentences themselves in the text and the attention between the sentences are spliced to determine the similarity between the texts. In this way, the dependency relationship and weight of words in the sentence are captured by using bidirectional semantic information, and the input at each moment is calculated in the reverse forward and backward directions, which improves the accuracy of confirming the text similarity. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0046] Figure 1 A flowchart of a text similarity analysis method provided in an embodiment of the present application;
[0047] Figure 2 A BiLSTM structure diagram provided for an embodiment of the present application;
[0048] Figure 3 A schematic diagram of the structure of a text similarity analysis device provided in an embodiment of the present application;
[0049] Figure 4A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. The present application provides a method and device for analyzing text similarity, which are used in the field of data analysis technology. The above is only an example and does not limit the application field of the method and device provided by the present application.
[0051] The rapid development of financial technology has led to an increasing demand for technology in the financial industry. More and more financial institutions are also applying artificial intelligence technology to improve their competitiveness and customer service levels. With the use of intelligent question-answering tools in the field of customer service, the intelligent question-answering system has also helped commercial banks achieve 24-hour intelligent customer service, becoming a key step in the digital transformation of commercial banks.
[0052] The bank knowledge base is a collection of various resources inside and outside the banking industry, including business rules, product introductions, risk compounding, customer service and business management knowledge, which can effectively improve the efficiency and competitiveness of banks, meet the diverse needs of customers, and promote the innovation and development of banks. Text similarity methods can help banks realize the construction, management, retrieval, and recommendation of knowledge bases. For example, the text similarity between customer questions and knowledge base questions can be calculated to realize intelligent customer service functions; the credit risk assessment function can be realized by calculating the text similarity between the information provided by users when applying for loans and the credit rules in the knowledge base; the news recommendation function can be realized by calculating the text similarity between the news browsed by users and the news in the knowledge base.
[0053] By using artificial intelligence technology, we can introduce intelligent question-answering systems in commercial banks, and automatically identify and answer user questions by calculating text similarity, thereby improving the service efficiency and customer satisfaction of commercial banks. The intelligent question-answering system can quickly match answers in the knowledge base or generate new answers based on user questions to meet user information needs; it can also provide personalized wealth management suggestions based on user risk preferences, consumption habits and life scene characteristics; it can also handle some highly professional, logical and standardized business issues, and complete account inquiries and credit approvals.
[0054] In the prior art, convolutional neural networks and multi-attention mechanisms are used to extract features of text and classify it. It includes a training set acquisition step, a text segmentation step, a word vector training step, a sentence splitting step, a word vector conversion step, a sentence-level convolutional neural network step; a sentence-level attention mechanism step; a sentence-level full connection step, a document processing step, and a classification step, including linearly transforming the document feature vector obtained in the document processing step, and then using a softmax function to generate the probability of each classification on the class set C. The model is designed into two parts, namely the sentence level and the document level. Sentence features are first extracted at the sentence level, and then document features are extracted at the document level for classification. However, this method only focuses on the attention of the sentence itself, resulting in inaccurate text similarity.
[0055] After research, the inventor proposed the technical solution of this application. First, a text set vector is generated based on the knowledge base data. Then, the one-dimensional vector text set vector is converted into a two-dimensional text set vector matrix through the word2vec word vector model. Then, the text set vector matrix is re-encoded using a bidirectional long short-term memory network to obtain a target text sequence. Finally, based on the target text sequence, the attention of the sentences themselves and the attention between sentences in the text are spliced to determine the similarity between texts. In this way, the dependency relationship and weight of words in the sentence are captured by using bidirectional semantic information, and the input at each moment is calculated in reverse forward and backward, which improves the accuracy of confirming text similarity.
[0056] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. It should be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.
[0057] See also Figure 1 , Figure 1 A flowchart of a text similarity analysis method provided in an embodiment of the present application includes:
[0058] S101: Generate a text set vector based on knowledge base data.
[0059] Obtain knowledge base data. The bank's knowledge base data includes the bank's policies and regulations, business processes, special financial services, and user question and answer data. There is a large amount of redundant data, and the knowledge database needs to be cleaned to remove symbol information and unify upper and lower case. Then, the text is digitized, the data entity information is extracted, and converted into a text set vector of a word index. Then, the data in the vector whose data dimension exceeds the maximum length is removed to reduce the impact of text noise.
[0060] S102: Using the word2vec word vector model, the one-dimensional vector text set vector is converted into a two-dimensional text set vector matrix.
[0061] Word2vec is a machine learning model that maps words in a text into a vector space. For the equal-length vector sequence obtained after preprocessing, a pre-trained word2vec model is used to convert the one-dimensional database entity text set vector into a two-dimensional database entity text set matrix, that is, converting the text into a word vector matrix to extract high-dimensional and sparse vector features and convert them into continuous low-dimensional dense vectors.
[0062] Among them, word vector: digitally represent words in the language through modeling, realize text vectorization, and enable computers to process natural language. There are two common representation methods: discrete representation represents words with vectors composed of 0 and 1, and its length is related to the number of vocabulary. Only one position is 1, corresponding to the current word, and the rest are 0; distributed representation represents words as a low-dimensional vector composed of numbers, which can overcome the shortcomings of high dimensionality and sparsity of discrete representation, so that similar words have similar positions in the vector space.
[0063] The word2vec model is trained using the CBOW model, with a window size of 5 and a vector dimension of 100. First, word2vec is used to train the knowledge base entity information to generate word vectors for each word. Then, the semantic similarity of two adjacent sentences is calculated. After adding and weighting, the semantic similarity of the sentences is obtained by mixing them with the Jaccard method according to the experimental weights. The Jaccard calculation method is as follows:
[0064]
[0065] The hybrid method of Jaccard and word vector combines the term information of co-occurring words and the semantic information of non-co-occurring words to obtain better results.
[0066] S103: Use a bidirectional long short-term memory network to re-encode the text set vector matrix to obtain a target text sequence.
[0067] BiLSTM combines forward LSTM and backward LSTM, so that it can capture both the forward and backward information of the input sequence. The prediction result is determined based on the previous inputs and the following inputs, which effectively improves the accuracy of the result. Figure 2 shown. Figure 2 A BiLSTM structure diagram provided in an embodiment of the present application, for a text sequence M (m1, m2, ...m n ), the forward LSTM is fed with a positive input sequence (m1,m2,...m n ), the backward LSTM reverse input sequence (m n ,m n-1 ,...m n ), after the hidden layer feature extraction, the forward hidden vector {k L1 ,k L2 ,...k Ln} and the inverse hidden vector {k R1 ,k R2 ,...k Rn}, concatenate to get {[k L1 k Rn ],[k L1 k Rn ],...[k L1 k Rn ]}, that is, the final result {k1,k2,...k n}.
[0068] S104: Based on the target text sequence, the attention of the sentences themselves in the text is concatenated with the attention between the sentences to determine the similarity between the texts.
[0069] In order to highlight the words in the key parts, the attention mechanism is introduced to screen the relevance of the data and focus on the important features. The bidirectional LSTM outputs the tensor of the time series by default. Combined with the hierarchical attention, attention is used on the words of the sentence and the sentences of the text respectively, so as to distinguish the irrelevant parts from the key parts. Attention is a method of dynamically assigning different weights to the words in the text according to the importance of the input data. For the two sentences R1 and R2 of length n, the attention is first calculated for the sentence itself, and the corresponding weight of the vector is calculated through the softmax function:
[0070]
[0071] The final attention K1 of the sentence is accumulated:
[0072]
[0073] Then calculate the attention for the interactive sentences and multiply the dot product of R1 and R2:
[0074] S(i, j) = h i T h j ;
[0075] The attention K2 after sentence interaction is obtained through Softmax from the row dimension and column dimension respectively:
[0076]
[0077] Finally, the attention of these two steps is concatenated to get the final attention K:
[0078] K = [K1, K2];
[0079] The above are some specific implementations of the text similarity analysis method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding device. The device provided in the embodiment of the present application will be introduced from the perspective of functional modularization.
[0080] See also Figure 3 , Figure 3 The present invention provides a schematic diagram of a text similarity analysis device according to an embodiment of the present invention. The text similarity analysis device 300 includes:
[0081] A generating module 310, used to generate a text set vector according to the knowledge base data;
[0082] The conversion module 320 is used to convert the one-dimensional vector text set vector into a two-dimensional text set vector matrix through the word2vec word vector model;
[0083] An encoding module 330, configured to re-encode the text set vector matrix using a bidirectional long short-term memory network to obtain a target text sequence;
[0084] The splicing module 340 is used to splice the attention of the sentences themselves and the attention between sentences in the text based on the target text sequence to determine the similarity between the texts.
[0085] Optionally, the generating module 310 includes:
[0086] An acquisition unit, used for acquiring knowledge base data;
[0087] The digitization unit digitizes the text in the knowledge base data to obtain a text set vector.
[0088] Optionally, the device 300 further includes:
[0089] A cleaning unit, used for cleaning the knowledge base data to obtain cleaned knowledge base data;
[0090] The digitization unit comprises:
[0091] The digitization subunit is used to digitize the text in the cleaned knowledge base data to obtain the text set vector.
[0092] Optionally, the device 300 further includes:
[0093] The elimination module is used to eliminate the data whose data dimension exceeds the preset length in the text set vector.
[0094] Optionally, the encoding module 330 includes:
[0095] A feature extraction unit is used to extract features of the LSTM forward input sequence and the LSTM reverse input sequence using the hidden layer to obtain a forward latent vector and a reverse latent vector;
[0096] A concatenation unit is used to concatenate the forward latent vector and the reverse latent vector to obtain the target text sequence.
[0097] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the solutions provided by the embodiments of the present application.
[0098] like Figure 4 As shown, the computer device 01 is in the form of a general computing device. The components of the computer device 01 may include but are not limited to: one or more processors or processing units 03, a system memory 08, and a bus 04 connecting different system components (including the system memory 08 and the processing unit 03).
[0099] Bus 04 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.
[0100] The computer device 01 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 01, including volatile and non-volatile media, removable and non-removable media.
[0101] The system memory 08 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 09 and / or cache memory 10. The computer device 01 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 11 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 04 via one or more data medium interfaces. The memory 08 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.
[0102] A program / utility 12 having a set (at least one) of program modules 13 may be stored, for example, in the memory 08, such program modules 13 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may include an implementation of a network environment. The program modules 13 generally perform the functions and / or methods of the embodiments described herein.
[0103] The computer device 01 may also communicate with one or more external devices 02 (e.g., keyboard, pointing device, display 07, etc.), one or more devices that enable a user to interact with the computer device 01, and / or any device that enables the computer device 01 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed via input / output (I / O) interface 06. Furthermore, the computer device 01 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) via network adapter 05. Figure 4 As shown, the network adapter 05 communicates with other modules of the computer device 01 via the bus 04. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with computer device 01, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0104] The processor unit 03 executes various functional applications and data processing by running the programs stored in the system memory 08, for example, implementing a text similarity analysis method provided in an embodiment of the present application.
[0105] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0106] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.
[0107] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.
[0108] The above description is merely an exemplary embodiment of the present application and is not intended to limit the protection scope of the present application.
Claims
1. A text similarity analysis method, characterized in that: include: Generate text set vectors based on knowledge base data; Through the word2vec word vector model, the one-dimensional vector text set vector is converted into a two-dimensional text set vector matrix; Re-encode the text set vector matrix using a bidirectional long short-term memory network to obtain a target text sequence; Based on the target text sequence, the attention of the sentences themselves and the attention between the sentences in the text are concatenated to determine the similarity between the texts.
2. The method according to claim 1, characterized in that Generating a text set vector according to the knowledge base data includes: Get knowledge base data; The text in the knowledge base data is digitized to obtain a text set vector.
3. The method according to claim 2, characterized in that Before digitizing the text in the knowledge base data to obtain the text set vector, the method further includes: Cleaning the knowledge base data to obtain cleaned knowledge base data; The step of digitizing the text in the knowledge base data to obtain a text set vector includes: The text in the cleaned knowledge base data is digitized to obtain the text set vector.
4. The method according to claim 1, characterized in that After generating the text set vector according to the knowledge base data, the method further includes: The data whose data dimension in the text set vector exceeds a preset length is removed.
5. The method according to claim 1, characterized in that: The step of re-encoding the text set vector matrix using a bidirectional long short-term memory network to obtain a target text sequence includes: Use the hidden layer to extract features from the LSTM forward input sequence and the LSTM reverse input sequence to obtain the forward latent vector and the reverse latent vector; The forward latent vector and the reverse latent vector are concatenated to obtain the target text sequence.
6. A text similarity analysis device, characterized in that: include: A generation module, used to generate text set vectors based on knowledge base data; The conversion module is used to convert the one-dimensional vector text set vector into a two-dimensional text set vector matrix through the word2vec word vector model; An encoding module, used for re-encoding the text set vector matrix using a bidirectional long short-term memory network to obtain a target text sequence; The splicing module is used to splice the attention of the sentences themselves and the attention between sentences in the text based on the target text sequence to determine the similarity between the texts.
7. The device according to claim 6, characterized in that The generating module comprises: An acquisition unit, used for acquiring knowledge base data; The digitization unit digitizes the text in the knowledge base data to obtain a text set vector.
8. The device according to claim 7, characterized in that The device also includes: A cleaning unit, used for cleaning the knowledge base data to obtain cleaned knowledge base data; The digitization unit comprises: The digitization subunit is used to digitize the text in the cleaned knowledge base data to obtain the text set vector.
9. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the text similarity analysis method according to any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the text similarity analysis method according to any one of claims 1 to 6.