A knowledge sparse distributed coding method and system

The semantic space is constructed through TF-IDF and TSNE models, which solves the problem of low natural language coding accuracy of the HTM model, and realizes efficient natural language sparse distributed coding, which improves the encoding capability of the HTM model.

CN114925657BActive Publication Date: 2024-11-15AEROSPACE INFORMATION RES INST CAS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210349150.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-01
Publication Date
2024-11-15
Estimated Expiration
2042-04-01

AI Technical Summary

Technical Problem

The existing HTM model encoder has poor encoding accuracy for natural language, and has defects such as high computational complexity and similarity calculation bottlenecks, resulting in limited encoding capabilities.

Method used

The text data is processed using the TF-IDF statistical model and the TSNE dimensionality reduction model, high-dimensional vectors are obtained and dimensionality reduction is reduced to low-dimensionality, semantic space is constructed, sparse distributed encoding of natural language, and knowledge encoding results are output.

Benefits of technology

It effectively reduces the complexity of encoding processing, improves the coding accuracy of natural language, and improves the encoding ability of HTM models for natural language.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114925657B_ABST
    Figure CN114925657B_ABST
Patent Text Reader

Abstract

The present invention provides a knowledge sparse distributed coding method and system. The method includes: obtaining text data to be processed from a data set; cutting the text data to obtain all sentences contained in the text data; encoding the words in the sentences based on the TF-IDF statistical model to obtain high-dimensional vectors corresponding to each sentence; reducing the dimension of the high-dimensional vectors corresponding to each sentence based on the TSNE dimensionality reduction model to obtain low-dimensional vectors corresponding to each sentence, and constructing a corresponding semantic space based on the low-dimensional vectors; wherein the low-dimensional vector is a two-dimensional sentence vector or a three-dimensional sentence vector; based on the semantic space, sparse distributed coding of natural language is performed on the input knowledge, and the corresponding knowledge coding result is output. The knowledge sparse distributed coding method provided by the present invention can effectively reduce the complexity of coding processing, improve the accuracy of natural language coding, and thus enhance the coding ability of natural language.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a knowledge sparse distributed coding method and system. In addition, the present invention also relates to an electronic device and a processor-readable storage medium. Background Art

[0002] The Hierarchical Temporal Memory (HTM) model is an imitation of the human brain's processing mechanism. This model can combine biology with computer science and simulate human learning and prediction behaviors through programming. The HTM model is a memory-based system that emphasizes the combination of the temporal and spatial characteristics of information. Regardless of the relationship between the data to be learned, the input knowledge data is a time-based sequence. When processing information, Figure 2As shown in the figure, it mainly goes through four steps: encoder, spatial pooling, temporal memory and classifier. The HTM model uses sparse distributed representation (SDR) to represent the input knowledge data. It has four characteristics: vector, high dimension, binary and sparse. It is similar to the mechanism of human brain neurons processing information. At present, the HTM model can be applied to scenarios such as behavioral anomaly detection, geospatial application tracking and financial monitoring. For the human brain, if you want to obtain data, you naturally get it from your senses. Neuroscience research shows that the senses will convert the received information into sparse data, which has two major characteristics: sparseness and redundancy. The encoder of the HTM model imitates the human sensory input. It mainly processes the input knowledge data into data that the HTM model can use. The input is raw data, and the output data is a binary vector or matrix. The output of the encoder has the following requirements: (1) For semantically similar data, binary vectors with overlapping activation sites should be generated; (2) The binary vectors generated by the same input should be the same; (3) For all input data, the output binary vectors should have the same dimension; (4) For all input data, the binary vectors should have similar sparsity. Based on the above conditions, four methods of HTM encoders have been proposed: numerical encoding, classification encoding, geographic space encoding, and natural language encoding. Although the proposed Semantic Folding natural language encoding method realizes the encoding of natural language by the HTM model, so that the encoded natural language can be processed by the HTM model, it provides potential for tasks such as relationship extraction, knowledge representation, and knowledge mining. However, the Webber method processes the corpus and constructs a 128*128 two-dimensional semantic space based on sentences. Each point in the matrix represents a defined semantic space. The higher the similarity between two sentences, the smaller the distance in the semantic space, and the greater the similarity, the greater the distance. The information contained in the input knowledge data will be directly mapped into the matrix. The cell containing the semantic information of the segment will be marked as 1, and the other positions will be marked as 0. In this way, the input natural language can generate a two-dimensional SDR matrix through the semantic space for subsequent processing by the HTM model.

[0003] However, the semantic folding theory (SFT) constructs a two-dimensional semantic space by processing the corpus and uses sparse distributed coding to represent natural language. However, it has defects such as high computational complexity and similarity calculation bottleneck, which has high limitations and makes the natural language encoding accuracy poor, resulting in the HTM model encoder's encoding ability for natural language. Therefore, how to design an efficient and accurate encoding scheme for the HTM model encoder is of great practical significance. Summary of the invention

[0004] To this end, the present invention provides a knowledge sparse distributed coding method and system to solve the defect that the coding scheme for the HTM model encoder in the prior art has high limitations, resulting in poor natural language coding accuracy.

[0005] In a first aspect, the present invention provides a knowledge sparse distributed coding method, comprising:

[0006] Get the text data to be processed from the dataset;

[0007] Cutting the text data to obtain all sentences contained in the text data;

[0008] Encode the words in the sentence based on the TF-IDF statistical model to obtain a high-dimensional vector corresponding to each sentence;

[0009] Based on the TSNE dimensionality reduction model, the high-dimensional vectors corresponding to the sentences are reduced in dimension to obtain low-dimensional vectors corresponding to the sentences, and a corresponding semantic space is constructed based on the low-dimensional vectors; wherein the low-dimensional vectors are two-dimensional sentence vectors or three-dimensional sentence vectors;

[0010] Based on the semantic space, the input knowledge is sparsely distributed encoded in natural language, and the corresponding knowledge encoding result is output.

[0011] Furthermore, the high-dimensional vectors corresponding to the sentences are reduced in dimension based on the TSNE dimensionality reduction model to obtain low-dimensional vectors corresponding to the sentences, and the corresponding semantic space is constructed based on the low-dimensional vectors, specifically including: reducing the high-dimensional vectors corresponding to the sentences to low-dimensional vectors based on the TSNE dimensionality reduction model, scaling and rounding the low-dimensional vectors to proportionally place the low-dimensional vectors into a preset 128*128 original semantic space to obtain the semantic space; and storing the semantic space in a database; the semantic space contains the coordinates of the sentences in the semantic space and the attribute information of the sentences.

[0012] Furthermore, after obtaining all the sentences included in the text data, the method further includes: counting all the words in the sentences and recording the number of occurrences of each word to generate a corresponding dictionary.

[0013] Furthermore, the sparse distributed encoding of the input knowledge in natural language based on the semantic space and outputting the corresponding knowledge encoding result specifically includes:

[0014] Obtain original input knowledge to be encoded; preprocess the original input knowledge, and compare and match the preprocessed input knowledge with each sentence in the database. If there is a target sentence corresponding to the word or phrase contained in the input knowledge, read the coordinates of the target sentence in the semantic space, and mark the value corresponding to the corresponding coordinate position in the 128*128 original semantic space as 1, so as to realize sparse distributed encoding of the input knowledge based on the semantic space and output the corresponding knowledge encoding result.

[0015] Furthermore, the text data is segmented to obtain all the sentences contained in the text data, specifically including: segmenting the preprocessed text data, and performing noise reduction processing on the sentences obtained after segmentation to eliminate invalid characters in the sentences and obtain all the sentences contained in the text data.

[0016] Furthermore, encoding the words in the sentence based on the TF-IDF statistical model to obtain the high-dimensional vector corresponding to each sentence specifically includes: encoding the words in the sentence based on the TF-IDF statistical model to determine the importance and inverse document frequency of the words, and determining the TF-IDF value based on the importance and inverse document frequency of the words to obtain the high-dimensional vector corresponding to each sentence.

[0017] In a second aspect, the present invention further provides a knowledge sparse distributed coding system, comprising:

[0018] A text data acquisition unit, used for acquiring text data to be processed from a data set;

[0019] A sentence generating unit, used for segmenting the text data to obtain all sentences contained in the text data;

[0020] A high-dimensional vector encoding unit, used to encode the words in the sentence based on the TF-IDF statistical model to obtain a high-dimensional vector corresponding to each sentence;

[0021] A dimensionality reduction processing unit, used to reduce the dimension of the high-dimensional vectors corresponding to the sentences based on the TSNE dimensionality reduction model to obtain low-dimensional vectors corresponding to the sentences, and construct a corresponding semantic space based on the low-dimensional vectors; wherein the low-dimensional vectors are two-dimensional sentence vectors or three-dimensional sentence vectors;

[0022] The knowledge encoding unit is used to perform sparse distributed encoding of the input knowledge in natural language based on the semantic space and output the corresponding knowledge encoding result.

[0023] Furthermore, the dimensionality reduction processing unit is specifically used to: reduce the high-dimensional vectors corresponding to the sentences to low-dimensional vectors based on the TSNE dimensionality reduction model, scale and round the low-dimensional vectors to proportionally place the low-dimensional vectors into a preset 128*128 original semantic space to obtain the semantic space; and store the semantic space in a database; the semantic space contains the coordinates of the sentences in the semantic space and the attribute information of the sentences.

[0024] Furthermore, after obtaining all the sentences included in the text data, it also includes: a dictionary generation unit, which is used to count all the words in the sentences and record the number of occurrences of each word to generate a corresponding dictionary.

[0025] Furthermore, the knowledge encoding unit is specifically used for:

[0026] Obtain original input knowledge to be encoded; preprocess the original input knowledge, and compare and match the preprocessed input knowledge with each sentence in the database. If there is a target sentence corresponding to the word or phrase contained in the input knowledge, read the coordinates of the target sentence in the semantic space, and mark the value corresponding to the corresponding coordinate position in the 128*128 original semantic space as 1, so as to realize sparse distributed encoding of the input knowledge based on the semantic space and output the corresponding knowledge encoding result.

[0027] Furthermore, the sentence generation unit is specifically used to: segment the preprocessed text data, and perform noise reduction processing on the sentences obtained after segmentation, so as to remove invalid characters in the sentences and obtain all the sentences contained in the text data.

[0028] Furthermore, the high-dimensional vector encoding unit is specifically used to: encode the words in the sentence based on the TF-IDF statistical model to determine the importance and inverse document frequency of the words, and determine the TF-IDF value based on the importance and inverse document frequency of the words to obtain the high-dimensional vector corresponding to each sentence.

[0029] In a third aspect, the present invention further provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the knowledge sparse distributed coding method as described in any one of the above items are implemented.

[0030] In a fourth aspect, the present invention further provides a processor-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the knowledge sparse distributed coding method as described in any one of the above items are implemented.

[0031] The knowledge sparse distributed coding method provided by the present invention obtains the text data to be processed from the data set, and cuts the text data to obtain all the sentences contained in the text data; then, encodes the words in the sentences based on the TF-IDF statistical model to obtain the high-dimensional vectors corresponding to each sentence, and reduces the dimension of the high-dimensional vectors corresponding to each sentence based on the TSNE dimensionality reduction model to obtain the low-dimensional vectors corresponding to each sentence, and constructs the corresponding semantic space; uses the semantic space to perform sparse distributed coding of natural language on the input knowledge, and outputs the corresponding knowledge coding result. It can effectively reduce the complexity of the coding process, improve the natural language coding accuracy, and thus enhance the natural language coding ability of the HTM model encoder. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 It is a flowchart of a knowledge sparse distributed coding method provided by an embodiment of the present invention;

[0034] Figure 2 It is a schematic diagram of the overall architecture of the HTM model provided by an embodiment of the present invention;

[0035] Figure 3 It is a schematic diagram of a complete processing flow of the knowledge sparse distributed coding method provided by an embodiment of the present invention;

[0036] Figure 4 It is a schematic diagram of a process of performing dimensionality reduction processing on a high-dimensional vector provided by an embodiment of the present invention;

[0037] Figure 5 is a flow chart of knowledge sparse distributed coding provided by an embodiment of the present invention;

[0038] Figure 6 is a schematic diagram of the structure of a knowledge sparse distributed coding system provided by an embodiment of the present invention;

[0039] Figure 7 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] In order to improve the encoding ability of the encoder in the HTM model for natural language, this application proposes a knowledge sparse distributed encoding scheme based on the TF-IDF statistical model (TF-IDF statistical algorithm) and the TSNE dimensionality reduction model (TSNE nonlinear dimensionality reduction algorithm). First, a semantic space is constructed based on the TF-IDF statistical model and the TSNE dimensionality reduction model to determine the semantic space for application to knowledge encoding based on the HTM model, and knowledge sparse distributed encoding is performed based on the semantic space, which improves the natural language encoding of the encoder in the HTM model and improves the natural language encoding accuracy of the HTM model.

[0042] Based on the knowledge sparse distributed coding method of the present invention, the embodiments are described in detail below. Figure 1 As shown, it is a flow chart of the knowledge sparse distributed coding method provided by an embodiment of the present invention, and the specific implementation process includes the following steps:

[0043] Step 101: Obtain text data to be processed from a data set.

[0044] Specifically, in the embodiment of the present invention, it is necessary to pre-process the data in the data set by noise reduction. That is, before encoding to obtain a high-dimensional vector, it is necessary to first remove the icons, formulas, tables, titles, etc. in the data that do not contain information or are difficult to extract information, so as to obtain more effective text data.

[0045] Step 102: Segment the text data to obtain all sentences contained in the text data.

[0046] like Figure 3As shown, in this step, the valid text data after preprocessing can be cut into sentences, and then the sentences are subjected to noise reduction processing, and the sequence numbers, punctuation marks, stop words, special characters and other contents in the sentences are removed, and the final data will obtain a relatively pure sentence library. Specifically, it is necessary to cut the preprocessed text data in advance, and perform noise reduction processing on the sentences obtained after cutting, so as to remove the invalid characters in the sentences and obtain all the sentences contained in the text data. After obtaining all the sentences contained in the text data, it also includes: counting all the words in the sentences and recording the number of occurrences of each word, so as to count all the words expected to appear, and generate a corresponding dictionary for subsequent use.

[0047] Step 103: Encode the words in the sentence based on the TF-IDF statistical model to obtain a high-dimensional vector corresponding to each sentence.

[0048] The TF-IDF statistical algorithm can determine the metric or rating basis for evaluating the relevance of keywords. In this statistical algorithm, the importance of keywords increases with the increase of their frequency of occurrence, and also decreases inversely with the frequency of occurrence in the corpus, that is, the more times a word appears in an article, and the fewer times it appears in all documents, the more representative it is of the article. Among them, the term frequency (TF) represents the frequency of the entry in the document. This number is usually normalized (generally the term frequency divided by the total number of articles) to prevent it from being biased towards longer documents, resulting in some words with less frequent occurrences being able to express the theme of the article. Therefore, it is not appropriate to use TF alone. Therefore, the weight design must meet the following requirements: the stronger the ability of a word to represent a document, the greater the weight. In addition, the inverse document frequency (IDF) represents the commonness of the keyword. For a certain word, if the fewer documents containing the word, the greater the IDF, it means that the entry has a good ability to distinguish categories. In the embodiment of the present invention, the TF-IDF statistical model is used to obtain the high-dimensional vector of the sentence, and the TF*IDF value is used as the word weight to indicate its importance in the entire corpus.

[0049] In an embodiment of the present invention, the words in the sentence are encoded based on the TF-IDF statistical model to determine the importance and inverse document frequency of the words, and the TF-IDF value is determined based on the importance and inverse document frequency of the words to obtain a high-dimensional vector corresponding to each sentence. Specifically, in the process of obtaining the high-dimensional vector of the sentence, the Vector Space Model (VSM) represents the semantic information of the text through the vector of the sentence, and uses the standardized inner product between two sentences to represent the similarity between the two sentences; the TF-IDF statistical model is used to encode the words to obtain the high-dimensional vector corresponding to each sentence. It should be noted that the TF-IDF statistical model can take into account both word frequency and importance, filter common words, and retain important words that can provide more information. For a word t in a particular file i For example, its importance can be expressed as:

[0050]

[0051] Among them, n i,j The word is in file D j The denominator is the number of times it appears in file D j The sum of the occurrences of all words in .

[0052] Inverse Document Frequency (IDF) reflects the prevalence of keywords. The more common a word is (i.e., a large number of documents contain this word), the lower its IDF value is; conversely, the higher its IDF value is. IDF is defined as follows:

[0053]

[0054] Where N is the total number of documents in the corpus; |{j:w i ∈D j}| contains the word w i If the word is not in the corpus, the denominator will be 0, so a smoothing factor of 1 is added. At this time, the TF-IDF value of keyword w in document d is:

[0055] TF-IDF=tf i,j *idf i

[0056] Cut the sentence into a set of words and sort them by the first letter. Because useless characters and stop words have been removed in the preprocessing part, the remaining words all contain valid information. Generate a key-value pair {x:y} for each word, where x is the subscript of the word in the dictionary, and y is the sum of the weights of the same word in the sentence (here the weight is the TF-IDF value). The high-dimensional vector of the sentence (such as an n-dimensional vector) is the set of encoding vectors of the n words in the sentence.

[0057] Step 104: Based on the TSNE dimensionality reduction model, the high-dimensional vectors corresponding to the sentences are reduced in dimension to obtain low-dimensional vectors corresponding to the sentences, and a corresponding semantic space is constructed based on the low-dimensional vectors; wherein the low-dimensional vectors are two-dimensional sentence vectors or three-dimensional sentence vectors.

[0058] The TSNE (t-distributed Stochastic Neighbor Embedding) dimensionality reduction algorithm is a nonlinear machine learning algorithm for dimensionality reduction. It can map high-dimensional vectors (n-dimensional data) to two-dimensional or three-dimensional data and visualize them. It can well preserve the local features of the data. In an embodiment of the present invention, the obtained high-dimensional vector is reduced to a two-dimensional space by using the dimensionality reduction algorithm to achieve the construction of a semantic space. The dimensionality reduction algorithm is developed from the SNE algorithm. SNE maps data points to a probability distribution through an affine transformation. It mainly includes two steps: (1) SNE constructs a probability distribution between high-dimensional objects so that similar objects have a higher probability of being selected, while dissimilar objects have a lower probability of being selected; (2) SNE constructs the probability distribution of these points in a low-dimensional space so that the two probability distributions are as similar as possible. Compared with SNE, the TSNE dimensionality reduction algorithm uses symmetric SNE to simplify the gradient formula. In a low-dimensional space, t distribution is used instead of Gaussian distribution to represent the similarity between two points. It is an unsupervised dimensionality reduction, so that the data distribution after dimensionality reduction is also relatively uniform.

[0059] In an embodiment of the present invention, the high-dimensional vectors corresponding to the sentences can be reduced to low-dimensional vectors based on the TSNE dimensionality reduction model, and the low-dimensional vectors can be scaled and rounded to achieve the purpose of placing the low-dimensional vectors proportionally into a preset 128*128 original semantic space to obtain the semantic space. Then, the semantic space is stored in a database for subsequent use. The semantic space contains the coordinates of the sentences in the semantic space and the attribute information of the sentences. After obtaining the high-dimensional vector of the sentence, the high-dimensional vector needs to be represented as a two-dimensional vector or a three-dimensional vector to construct a semantic space. Specifically, in an embodiment of the present invention, the TSNE dimensionality reduction algorithm is used. The TSNE dimensionality reduction process is as follows: Figure 4As shown. The TSNE dimensionality reduction algorithm is used to map data points to corresponding probability distributions through affine transformation to reduce the dimensionality of high-dimensional vectors to two-dimensional vectors or three-dimensional vectors and visualize them. If the method of simply projecting the data onto an axis for dimensionality reduction will lose a lot of information. The TSNE dimensionality reduction algorithm can first measure the distance of a point relative to other points. Instead of directly processing these distances, it maps them to a probability distribution. Use tools such as KL divergence and cross entropy to measure whether the two distances are close enough. It should be noted that KL divergence is asymmetric, and the penalty weights corresponding to different distances in low-dimensional mapping are different. Specifically, two data points that are farther apart will generate a larger cost to express two data points that are closer. On the contrary, the cost generated by using two closer data points to express two data points that are farther apart is relatively small. Therefore, TSNE tends to retain local features in the data. TSNE first converts the Euclidean distance into conditional probability to express the similarity between data points, that is, given n high-dimensional vectors of data x 1 ,...,x n , using the TSNE dimensionality reduction algorithm, the first step is to calculate the probability p ij , which is proportional to x i and x j The similarity between them is:

[0060]

[0061] Different data points have different σ i , P i The entropy of i The TSNE dimensionality reduction algorithm uses the concept of perplexity to find an optimal σ using a binary search. The perplexity refers to:

[0062]

[0063] Here H(P i ) is P i The entropy is:

[0064]

[0065] The perplexity can be the number of valid neighboring data points near a data point. The TSNE dimensionality reduction algorithm is relatively robust to the adjustment of the perplexity. For example, it is usually selected between 5 and 50. After it is given, a binary search method is used to find a suitable σ. In addition, p is set x∣x= 0, because the embodiment of the present invention focuses on the similarity between two pairs. For the low-dimensional yi, the t distribution is used to represent it. The t distribution is less affected by outliers, the fitting result is more reasonable, and it better captures the overall characteristics of the data. The probability q ij for:

[0066]

[0067] Similarly, set q i∣i = 0. If the dimensionality reduction effect is good and the local features are preserved intact, then p i∣j =q i∣j , so we optimize the distance between the two distributions - KL divergence (Kullback-Leibler divergences), and use the joint probability distribution to replace the conditional probability distribution, that is, P is the joint probability distribution of each point in the high-dimensional space, Q is in the low-dimensional space, and the objective function (cost function) is as follows:

[0068]

[0069] Here the p ij is 0, because it is assumed that for any i, p ij =p ji ,q ij =q ji , so the probability distribution can be rewritten as:

[0070]

[0071] The objective function is equivalent to the formula ∑∑-plog(q) and the objective function ∑-ylogp of softmax, and the corresponding gradient is yp (y represents label, p represents estimated value). Similarly, it can be deduced that the gradient of the conditional probability of i under j in the objective function of TSNE is 2(p i∣j -q i∣j )(y i -y j ), and similarly, the gradient of the conditional probability of j under i is 2(p j∣i -q j∣i )(y i -y j ), and finally the complete gradient formula is as follows:

[0072]

[0073] It should be noted that the above formula has the problem of outliers. For example, if xi is an outlier, then ||x i -x j || 2 will be very large, corresponding to all j, pij will be very small, resulting in y under low-dimensional mapping i The impact on cost is small. In order to solve this problem, the joint probability distribution definition is modified as follows in the embodiment of the present invention: ij =(p i∣j +p j∣i ) / 2, thus ensuring that ∑p ij >1 / (2n), so that each point will have a certain contribution to the cost. The gradient is calculated as follows:

[0074]

[0075] During initialization, a Gaussian distribution with a small σ can be used for initialization. In order to speed up the optimization process and avoid falling into a local optimal solution, a relatively large momentum is required in the gradient. That is, in addition to the current gradient, the exponential decay term of the previous gradient accumulation should also be introduced in the parameter update, as follows:

[0076]

[0077] Among them, Y(t) represents the solution of iteration t times; η represents the learning rate; α(t) represents the momentum of iteration t times. The number of iterations is T, the learning rate is η, and the momentum α(t) are all predetermined hyperparameters. Then, according to the set parameters, iterate from t=1 to T times, and finally obtain low-dimensional data that meets the requirements. Further, after the high-dimensional vector is reduced in dimension by TSNE, the two-dimensional vector or three-dimensional vector of the sentence is scaled and rounded so that each value in the vector is an integer in (0, 128), and finally the coordinates of the sentence in the semantic space are obtained, and the coordinates and the attribute information of the sentence are stored in the database for subsequent use.

[0078] Step 105: Perform sparse distributed encoding of natural language on the input knowledge based on the semantic space, and output corresponding knowledge encoding results.

[0079] In an embodiment of the present invention, it is first necessary to obtain the original input knowledge to be encoded, and then preprocess the original input knowledge to obtain input knowledge, and compare and match the preprocessed input knowledge with each sentence in the database. If there is a target sentence corresponding to the word or phrase contained in the input knowledge, the coordinates of the target sentence in the semantic space are read, and the value corresponding to the corresponding coordinate position in the 128*128 original semantic space is marked as 1, so as to implement sparse distributed encoding of the input knowledge based on the semantic space and output the corresponding knowledge encoding result. Specifically, Figure 5As shown in the figure, in the process of acquiring knowledge encoding, after the semantic space is constructed, when the input knowledge needs to be encoded, the semantic space encoding of the corresponding input knowledge can be obtained through the interface. Through the input knowledge (generally words or phrases), the input knowledge is denoised and stop words are removed, split into words, and each word is compared with each sentence in the database. If the sentence contains the corresponding word, the coordinates of the sentence in the semantic space are read, and its semantic space coordinates can be output. At the same time, the coordinates are marked as 1 in the output 128*128 matrix, and finally the 128*128 sparse knowledge encoding of the value is output, that is, the knowledge encoding of the sentence is obtained.

[0080] The knowledge sparse distributed coding method described in the embodiment of the present invention obtains the text data to be processed from the data set, and cuts the text data to obtain all the sentences contained in the text data; then, encodes the words in the sentences based on the TF-IDF statistical model to obtain the high-dimensional vectors corresponding to each sentence, and reduces the dimension of the high-dimensional vectors corresponding to each sentence based on the TSNE dimensionality reduction model to obtain the low-dimensional vectors corresponding to each sentence, and constructs the corresponding semantic space; uses the semantic space to perform sparse distributed coding of natural language on the input knowledge, and outputs the corresponding knowledge coding result. It can effectively reduce the complexity of the coding process, improve the accuracy of natural language coding, and thus enhance the natural language coding ability of the HTM model encoder.

[0081] Corresponding to the above-mentioned knowledge sparse distributed coding method, the present invention also provides a knowledge sparse distributed coding system. Since the embodiment of the system is similar to the above-mentioned method embodiment, the description is relatively simple. For the relevant parts, please refer to the description of the above-mentioned method embodiment. The embodiment of the knowledge sparse distributed coding system described below is only illustrative. Please refer to Figure 6 As shown, it is a structural diagram of a knowledge sparse distributed coding system provided by an embodiment of the present invention.

[0082] The knowledge sparse distributed coding system of the present invention specifically includes:

[0083] The text data acquisition unit 601 is used to acquire the text data to be processed from the data set;

[0084] A sentence generating unit 602, configured to segment the text data to obtain all sentences contained in the text data;

[0085] A high-dimensional vector encoding unit 603 is used to encode the words in the sentence based on the TF-IDF statistical model to obtain a high-dimensional vector corresponding to each sentence;

[0086] A dimensionality reduction processing unit 604 is used to reduce the dimension of the high-dimensional vectors corresponding to the sentences based on the TSNE dimensionality reduction model to obtain low-dimensional vectors corresponding to the sentences, and construct a corresponding semantic space based on the low-dimensional vectors; wherein the low-dimensional vectors are two-dimensional sentence vectors or three-dimensional sentence vectors;

[0087] The knowledge encoding unit 605 is used to perform sparse distributed encoding of the input knowledge in natural language based on the semantic space, and output the corresponding knowledge encoding result.

[0088] Furthermore, the dimensionality reduction processing unit is specifically used to: reduce the high-dimensional vectors corresponding to the sentences to low-dimensional vectors based on the TSNE dimensionality reduction model, scale and round the low-dimensional vectors to proportionally place the low-dimensional vectors into a preset 128*128 original semantic space to obtain the semantic space; and store the semantic space in a database; the semantic space contains the coordinates of the sentences in the semantic space and the attribute information of the sentences.

[0089] Furthermore, after obtaining all the sentences included in the text data, it also includes: a dictionary generation unit, which is used to count all the words in the sentences and record the number of occurrences of each word to generate a corresponding dictionary.

[0090] Furthermore, the knowledge encoding unit is specifically used for:

[0091] Obtain original input knowledge to be encoded; preprocess the original input knowledge, and compare and match the preprocessed input knowledge with each sentence in the database. If there is a target sentence corresponding to the word or phrase contained in the input knowledge, read the coordinates of the target sentence in the semantic space, and mark the value corresponding to the corresponding coordinate position in the 128*128 original semantic space as 1, so as to realize sparse distributed encoding of the input knowledge based on the semantic space and output the corresponding knowledge encoding result.

[0092] Furthermore, the sentence generation unit is specifically used to: segment the preprocessed text data, and perform noise reduction processing on the sentences obtained after segmentation, so as to remove invalid characters in the sentences and obtain all the sentences contained in the text data.

[0093] Furthermore, the high-dimensional vector encoding unit is specifically used to: encode the words in the sentence based on the TF-IDF statistical model to determine the importance and inverse document frequency of the words, and determine the TF-IDF value based on the importance and inverse document frequency of the words to obtain the high-dimensional vector corresponding to each sentence.

[0094] The knowledge sparse distributed coding system described in the embodiment of the present invention obtains the text data to be processed from the data set, and cuts the text data to obtain all the sentences contained in the text data; then, encodes the words in the sentences based on the TF-IDF statistical model to obtain the high-dimensional vectors corresponding to each sentence, and reduces the dimension of the high-dimensional vectors corresponding to each sentence based on the TSNE dimensionality reduction model to obtain the low-dimensional vectors corresponding to each sentence, and constructs the corresponding semantic space; uses the semantic space to perform sparse distributed coding of natural language on the input knowledge, and outputs the corresponding knowledge coding result. It can effectively reduce the complexity of the coding process, improve the accuracy of natural language coding, and thus enhance the natural language coding ability of the HTM model encoder.

[0095] Corresponding to the knowledge sparse distributed coding method provided above, the present invention also provides an electronic device. Since the embodiment of the electronic device is similar to the embodiment of the method described above, the description is relatively simple. For the relevant parts, please refer to the description of the embodiment of the method described above. The electronic device described below is only exemplary. Figure 7 As shown, it is a schematic diagram of the physical structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may include: a processor 701, a memory 702 and a communication bus 703, wherein the processor 701 and the memory 702 complete mutual communication through the communication bus 703, and communicate with the outside through the communication interface 704. The processor 701 can call the logic instructions in the memory 702 to execute the knowledge sparse distributed coding method, which includes: obtaining the text data to be processed from the data set; cutting the text data to obtain all the sentences contained in the text data; encoding the words in the sentence based on the TF-IDF statistical model to obtain the high-dimensional vectors corresponding to each sentence; reducing the high-dimensional vectors corresponding to each sentence based on the TSNE dimensionality reduction model to obtain the low-dimensional vectors corresponding to each sentence, and constructing the corresponding semantic space based on the low-dimensional vector; wherein the low-dimensional vector is a two-dimensional sentence vector or a three-dimensional sentence vector; based on the semantic space, the input knowledge is sparsely distributed coded in natural language, and the corresponding knowledge coding result is output.

[0096] In addition, the logic instructions in the above-mentioned memory 702 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: a storage chip, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes.

[0097] On the other hand, an embodiment of the present invention further provides a computer program product, the computer program product comprising a computer program stored on a processor-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the computer can execute the knowledge sparse distributed coding method provided by the above-mentioned method embodiments. The method comprises: obtaining text data to be processed from a data set; cutting the text data to obtain all sentences contained in the text data; encoding the words in the sentences based on the TF-IDF statistical model to obtain high-dimensional vectors corresponding to each sentence; reducing the high-dimensional vectors corresponding to each sentence based on the TSNE dimensionality reduction model to obtain low-dimensional vectors corresponding to each sentence, and constructing a corresponding semantic space based on the low-dimensional vector; wherein the low-dimensional vector is a two-dimensional sentence vector or a three-dimensional sentence vector; performing sparse distributed coding of natural language on the input knowledge based on the semantic space, and outputting the corresponding knowledge coding result.

[0098] On the other hand, an embodiment of the present invention further provides a processor-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor, it is implemented to execute the knowledge sparse distributed coding method provided by the above embodiments. The method includes: obtaining text data to be processed from a data set; cutting the text data to obtain all sentences contained in the text data; encoding the words in the sentences based on the TF-IDF statistical model to obtain high-dimensional vectors corresponding to each sentence; reducing the dimension of the high-dimensional vector corresponding to each sentence based on the TSNE dimensionality reduction model to obtain a low-dimensional vector corresponding to each sentence, and constructing a corresponding semantic space based on the low-dimensional vector; wherein the low-dimensional vector is a two-dimensional sentence vector or a three-dimensional sentence vector; based on the semantic space, sparse distributed coding of natural language is performed on the input knowledge, and the corresponding knowledge coding result is output.

[0099] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0100] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A knowledge sparse distributed coding method, characterized in that: include: Get the text data to be processed from the dataset; Cutting the text data to obtain all sentences contained in the text data; Encode the words in the sentence based on the TF-IDF statistical model to obtain a high-dimensional vector corresponding to each sentence; The encoding of the words in the sentence based on the TF-IDF statistical model to obtain the high-dimensional vector corresponding to each sentence specifically includes: encoding the words in the sentence based on the TF-IDF statistical model to determine the importance and inverse document frequency of the words, and determining the TF-IDF value based on the importance and inverse document frequency of the words to obtain the high-dimensional vector corresponding to each sentence; Based on the TSNE dimensionality reduction model, the high-dimensional vectors corresponding to the sentences are reduced in dimension to obtain low-dimensional vectors corresponding to the sentences, and the corresponding semantic space is constructed based on the low-dimensional vectors; wherein the low-dimensional vectors are two-dimensional sentence vectors or three-dimensional sentence vectors; based on the TSNE dimensionality reduction model, the high-dimensional vectors corresponding to the sentences are reduced in dimension to obtain low-dimensional vectors corresponding to the sentences, and the corresponding semantic space is constructed based on the low-dimensional vectors, specifically including: based on the TSNE dimensionality reduction model, the high-dimensional vectors corresponding to the sentences are reduced in dimension to low-dimensional vectors, and the low-dimensional vectors are scaled and rounded to put the low-dimensional vectors into a preset 128*128 original semantic space in proportion to obtain the semantic space; and the semantic space is stored in a database; the semantic space contains the coordinates of the sentences in the semantic space and the attribute information of the sentences; Based on the semantic space, the input knowledge is sparsely distributed encoded in natural language, and the corresponding knowledge encoding result is output; The method of performing sparse distributed encoding of natural language on input knowledge based on the semantic space and outputting corresponding knowledge encoding results specifically includes: obtaining original input knowledge to be encoded; preprocessing the original input knowledge, and comparing and matching the preprocessed input knowledge with each sentence in a database, and if there is a target sentence corresponding to the words or phrases contained in the input knowledge, reading the coordinates of the target sentence in the semantic space, and marking the value corresponding to the corresponding coordinate position in the 128*128 original semantic space as 1, so as to achieve sparse distributed encoding of the input knowledge based on the semantic space and output corresponding knowledge encoding results.

2. The knowledge sparse distributed coding method according to claim 1, characterized in that: After all the sentences included in the text data are obtained, the method further includes: counting all the words in the sentences and recording the number of occurrences of each word to generate a corresponding dictionary.

3. The knowledge sparse distributed coding method according to claim 1, characterized in that: The text data is segmented to obtain all sentences contained in the text data, specifically comprising: segmenting the preprocessed text data, and performing noise reduction processing on the sentences obtained after segmentation to eliminate invalid characters in the sentences, and obtain all sentences contained in the text data.

4. A system for implementing the knowledge sparse distributed coding method according to claim 1, characterized in that: include: A text data acquisition unit, used for acquiring text data to be processed from a data set; A sentence generating unit, used for segmenting the text data to obtain all sentences contained in the text data; A high-dimensional vector encoding unit is used to encode the words in the sentence based on the TF-IDF statistical model to obtain a high-dimensional vector corresponding to each sentence; the high-dimensional vector encoding unit is specifically used to: encode the words in the sentence based on the TF-IDF statistical model to determine the importance and inverse document frequency of the words, and determine the TF-IDF value based on the importance and inverse document frequency of the words to obtain a high-dimensional vector corresponding to each sentence; A dimensionality reduction processing unit is used to reduce the dimension of the high-dimensional vectors corresponding to the sentences based on the TSNE dimensionality reduction model to obtain low-dimensional vectors corresponding to the sentences, and construct a corresponding semantic space based on the low-dimensional vectors; wherein the low-dimensional vectors are two-dimensional sentence vectors or three-dimensional sentence vectors; the dimensionality reduction processing unit is specifically used to: reduce the dimension of the high-dimensional vectors corresponding to the sentences to low-dimensional vectors based on the TSNE dimensionality reduction model, scale and round the low-dimensional vectors to achieve proportional placement of the low-dimensional vectors into a preset 128*128 original semantic space to obtain the semantic space; and store the semantic space in a database; the semantic space contains the coordinates of the sentences in the semantic space and the attribute information of the sentences; A knowledge encoding unit is used to perform sparse distributed encoding of input knowledge in natural language based on the semantic space, and output corresponding knowledge encoding results; the knowledge encoding unit is specifically used to: obtain original input knowledge to be encoded; preprocess the original input knowledge, and compare and match the preprocessed input knowledge with each sentence in the database. If there is a target sentence corresponding to the words or phrases contained in the input knowledge, the coordinates of the target sentence in the semantic space are read, and the values ​​corresponding to the corresponding coordinate positions in the 128*128 original semantic space are marked as 1, so as to realize sparse distributed encoding of the input knowledge based on the semantic space and output corresponding knowledge encoding results.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the knowledge sparse distributed coding method as described in any one of claims 1 to 3 are implemented.

6. A processor-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the knowledge sparse distributed coding method as described in any one of claims 1 to 3 are implemented.