Natural language processing method, device, equipment, medium and program product

By generating a sparse coding matrix and using the HTM model to process the text to be processed, the problem that the HTM model cannot output predicted text is solved, and effective encoding and accurate prediction of text are achieved.

CN114997160BActive Publication Date: 2024-11-26AEROSPACE INFORMATION RES INST CAS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210404668.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-18
Publication Date
2024-11-26
Estimated Expiration
2042-04-18

AI Technical Summary

Technical Problem

Existing hierarchical real-time memory (HTM) models are unable to effectively encode the text to be processed, resulting in the inability to output predicted text.

Method used

By acquiring the target words in the text to be processed, a sparse coding matrix is ​​generated using a preset coding matrix and two-dimensional coordinates of a pre-constructed semantic space. This matrix is ​​then processed by an HTM model to output the predicted text.

Benefits of technology

The HTM model was implemented for efficient encoding and prediction of the text to be processed, thereby improving the accuracy of the predicted text.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114997160B_ABST
    Figure CN114997160B_ABST
Patent Text Reader

Abstract

The present invention provides a natural language processing method, device, equipment, medium and program product, the method comprising: obtaining a target word included in a text to be processed; determining a sparse coding matrix corresponding to the target word according to the target word, a preset coding matrix and two-dimensional coordinates of a plurality of preset texts included in a pre-obtained semantic space; processing the sparse coding matrix through a hierarchical real-time memory HTM model to obtain a predicted text corresponding to the text to be processed. The natural language processing method, device, equipment, medium and program product provided by the present invention are used to enable the HTM model to output a predicted text corresponding to the text to be processed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical fields of machine learning, natural language processing (NLP), and more particularly to a natural language processing method, device, equipment, medium, and program product (invention name). Background Art

[0002] The Hierarchical Temporal Memory (HTM) model can process data information with time sequence and obtain the prediction information corresponding to the data information.

[0003] In the related art, the HTM model includes an encoder. The encoder is used to output encoding information based on data information. Other processors in the HTM model, except the encoder, output prediction information corresponding to the data information based on the encoding information. At present, the above data information can be numerical, classified, geographic space and other data, rather than the text to be processed, so it is impossible to obtain the encoding information corresponding to the text to be processed (that is, the above encoder cannot process the text to be processed), resulting in the HTM model being unable to output the predicted text corresponding to the text to be processed. Therefore, how to enable the HTM model to output the predicted text corresponding to the text to be processed becomes a problem to be solved urgently. Summary of the invention

[0004] The present invention provides a natural language processing method, device, equipment, medium and program product, which are used to solve the defect that the HTM model in the prior art cannot output the predicted text corresponding to the text to be processed, and achieve the purpose of the HTM model being able to output the predicted text corresponding to the text to be processed.

[0005] The present invention provides a natural language processing method, comprising: obtaining a target word included in a text to be processed; determining a sparse coding matrix corresponding to the target word according to the target word, a preset coding matrix and two-dimensional coordinates of a plurality of preset texts included in a pre-obtained semantic space; processing the sparse coding matrix through a hierarchical real-time memory HTM model to obtain a predicted text corresponding to the text to be processed.

[0006] According to a natural language processing method provided by the present invention, a sparse coding matrix corresponding to a target word is determined based on the target word, a preset coding matrix and the two-dimensional coordinates of multiple preset texts included in a pre-obtained semantic space, including: obtaining the target coordinates of the preset text including the target word from the two-dimensional coordinates of the multiple preset texts; setting the numerical value at the position corresponding to the target coordinate in the preset coding matrix to a first preset value, and obtaining the sparse coding matrix corresponding to the target word.

[0007] According to a natural language processing method provided by the present invention, target words included in a text to be processed are obtained, including: performing abnormal text style elimination processing on the text to be processed to obtain an intermediate text; performing symbol elimination processing and stop word elimination processing on the intermediate text to obtain a target text; performing word splitting processing on the target text to obtain the target words included in the text to be processed.

[0008] A natural language processing method provided by the present invention also includes: obtaining multiple preset texts included in an original text set; performing word splitting processing and statistical sorting processing on the multiple preset texts to obtain a dictionary; the dictionary includes multiple words included in the multiple preset texts, and the multiple words are arranged in the dictionary according to the order of appearance in the original text set; determining the unique hot vectors of each of the multiple words in the dictionary; and determining the two-dimensional coordinates of the multiple preset texts included in the semantic space based on the unique hot vectors of each of the multiple words.

[0009] According to a natural language processing method provided by the present invention, the two-dimensional coordinates of a plurality of preset texts included in a semantic space are determined according to the unique hot vectors of a plurality of words, including:

[0010] According to the unique hot vectors of each of the multiple words, the model parameters of the continuous bag of words CBOW model are adjusted to obtain a weight matrix from the input layer to the middle layer of the CBOW model; according to the weight matrix, the encoding vectors of each of the multiple words are determined; for each preset text among the multiple preset texts, the sum of the encoding vectors of the words included in the preset text is determined as the initial semantic vector of the preset text; through a preset dimensionality reduction model, the initial semantic vector is dimensionally reduced to obtain a two-dimensional semantic vector of the preset text; the two-dimensional semantic vector is scaled and rounded to obtain a two-dimensional coordinate of the preset text.

[0011] According to a natural language processing method provided by the present invention, the dimension of the weight matrix is ​​equal to the total number of multiple words; based on the weight matrix, the encoding vectors of each of the multiple words are determined, including: for the i-th word among the multiple words, the i-th dimension vector in the weight matrix is ​​determined as the encoding vector of the i-th word; i is an integer greater than or equal to 1 and less than or equal to the total number.

[0012] The present invention also provides a natural language processing device, comprising:

[0013] An acquisition module, used for acquiring target words included in the text to be processed;

[0014] A determination module, used to determine a sparse coding matrix corresponding to a target word according to the target word, a preset coding matrix and two-dimensional coordinates of a plurality of preset texts included in a pre-obtained semantic space;

[0015] The processing module is used to process the sparse coding matrix through the hierarchical real-time memory HTM model to obtain the predicted text corresponding to the text to be processed.

[0016] According to the natural language processing device provided by the present invention, the determination module is specifically used to: obtain the target coordinates of the preset text including the target word from the two-dimensional coordinates of multiple preset texts; set the value at the position corresponding to the target coordinate in the preset coding matrix to the first preset value, and obtain the sparse coding matrix corresponding to the target word

[0017] According to the natural language processing device provided by the present invention, the acquisition module is specifically used for: performing abnormal text style elimination processing on the text to be processed to obtain an intermediate text; performing symbol elimination processing and stop word elimination processing on the intermediate text to obtain a target text; performing word splitting processing on the target text to obtain target words included in the text to be processed.

[0018] According to the natural language processing device provided by the present invention, the determination module is also used to: obtain multiple preset texts included in the original text set; perform word splitting processing and statistical sorting processing on the multiple preset texts to obtain a dictionary; the dictionary includes multiple words included in the multiple preset texts, and the multiple words are arranged in the dictionary according to the order of appearance in the original text set; determine the unique hot vectors of each of the multiple words in the dictionary; and determine the two-dimensional coordinates of the multiple preset texts included in the semantic space based on the unique hot vectors of each of the multiple words.

[0019] According to the natural language processing device provided by the present invention, the determination module is also specifically used to: adjust the model parameters of the continuous bag-of-words CBOW model according to the unique hot vectors of each of the multiple words, and obtain the weight matrix from the input layer to the middle layer of the CBOW model; determine the encoding vectors of each of the multiple words according to the weight matrix; for each of the multiple preset texts, determine the sum of the encoding vectors of the words included in the preset text as the initial semantic vector of the preset text; perform dimensionality reduction processing on the initial semantic vector through a preset dimensionality reduction model to obtain a two-dimensional semantic vector of the preset text; perform scaling and rounding processing on the two-dimensional semantic vector to obtain the two-dimensional coordinates of the preset text.

[0020] According to the natural language processing device provided by the present invention, the dimension of the weight matrix is ​​equal to the total number of multiple words; the determination module is also specifically used to: for the i-th word among the multiple words, determine the i-th dimension vector in the weight matrix as the encoding vector of the i-th word; i is an integer greater than or equal to 1 and less than or equal to the total number.

[0021] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the above-mentioned natural language processing methods is implemented.

[0022] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements any of the above-mentioned natural language processing methods when executed by a processor.

[0023] The present invention also provides a computer program product, including a computer program, which implements any of the above-mentioned natural language processing methods when executed by a processor.

[0024] The natural language processing method, apparatus, device, medium and program product provided by the present invention obtain a sparse coding matrix corresponding to the target word and having an SDR data structure through the target word included in the text to be processed, the preset coding matrix and the two-dimensional coordinates of multiple preset texts included in the pre-obtained semantic space, so that the HTM model can process the sparse coding matrix to obtain the predicted text corresponding to the text to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is a schematic diagram of an application scenario provided by the present invention;

[0027] Figure 2 It is a flowchart of the natural language processing method provided by the present invention;

[0028] Figure 3 It is a schematic diagram of a flow chart of obtaining a semantic space provided by the present invention;

[0029] Figure 4 is a schematic diagram of the structure of the natural language processing device provided by the present invention;

[0030] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] In the related art, the HTM model includes an encoder. The encoder is used to output coded information based on data information. For example, the data information can be data such as numerical values, categories (such as Monday to Sunday), and geographic space (such as geographic space coordinates). At present, the data information cannot be the text to be processed, so it is impossible to obtain the coded information corresponding to the text to be processed (that is, the above encoder cannot process the text to be processed), which causes the HTM model to be unable to output the predicted text corresponding to the text to be processed. Therefore, how to enable the HTM model to output the predicted text corresponding to the text to be processed becomes a problem to be solved urgently.

[0033] In practical applications, the data structure used by the HTM model is a sparse distributed representation (SDR). Therefore, in this application, in order to enable the HTM model to output the predicted text corresponding to the text to be processed, the inventors thought of obtaining a sparse coding matrix corresponding to the target word and having a data structure of SDR based on the target words included in the text to be processed, the preset coding matrix and the two-dimensional coordinates of multiple preset texts included in the pre-obtained semantic space, so that the HTM model can output the predicted text corresponding to the text to be processed based on the sparse coding matrix.

[0034] Combine the following Figure 1 The application scenarios of the natural language processing method in the invention are explained.

[0035] Figure 1 is a schematic diagram of the application scenario provided by the present invention. Figure 1 As shown, it includes: database and HTM model.

[0036] The database includes a semantic space, which includes two-dimensional coordinates of multiple preset texts and words included in each of the multiple preset texts. The database may also include indexes of multiple preset texts, etc.

[0037] When it is necessary to obtain the predicted text corresponding to the text to be processed, the sparse coding matrix of the target word can be obtained according to the two-dimensional coordinates of multiple preset texts and the target words included in the text to be processed, and the sparse coding matrix can be processed by the HTM model to obtain the predicted text corresponding to the text to be processed, so that the HTM model can output the predicted text corresponding to the text to be processed.

[0038] The following is combined with specific Figure 2-5 The embodiment illustrates the natural language processing method provided by the present invention.

[0039] Figure 2 Schematic diagram of the natural language processing method provided by the present invention. Figure 2 As shown, the method includes:

[0040] S201, obtaining target words included in the text to be processed.

[0041] The execution subject of the embodiment of the present invention may be an electronic device (for example, a computer, etc.), or a natural language processing device arranged in the electronic device. The natural language processing device may be implemented by a combination of software and / or hardware.

[0042] The text to be processed may be a sentence written in a natural language such as Chinese or English.

[0043] The target words included in the text to be processed are words obtained after preprocessing the text to be processed.

[0044] For example, when the text to be processed is a sentence written in English, if the text to be processed is "I like apple", the target words "like" and "apple" are obtained after preprocessing.

[0045] For example, when the text to be processed is a sentence written in Chinese, if the text to be processed is "I like apples", the target words "like" and "apple" are obtained after preprocessing.

[0046] In the present invention, the following method can be used to obtain the target words included in the text to be processed: the text to be processed is subjected to abnormal text style elimination processing to obtain an intermediate text; the intermediate text is subjected to symbol elimination processing and stop word elimination processing to obtain a target text; the target text is subjected to word splitting processing to obtain the target words included in the text to be processed.

[0047] Abnormal text styles may include, but are not limited to, titles, charts, formulas, data, references, etc.

[0048] In some embodiments, performing abnormal text style elimination processing on the text to be processed to obtain the intermediate text includes: eliminating text with abnormal text style in the text to be processed to obtain the intermediate text.

[0049] In some embodiments, after abnormal text style elimination processing is performed, text that does not contain information (such as be, at, but, etc.) or text from which information is difficult to extract (such as can, come, contain, etc.) can also be eliminated from the text to be processed to obtain intermediate text.

[0050] In some embodiments, the intermediate text is processed by removing symbols and stop words to obtain the target text, including: removing preset symbols and stop words in the intermediate text to obtain the target text.

[0051] Preset symbols may include serial numbers, Latin symbols, punctuation marks, etc.

[0052] Stop words may be words without semantic features, such as I, she, it, etc. Stop words may be words in a pre-stored stop word set.

[0053] In some embodiments, the above method of obtaining the target text can be changed to obtain other methods of obtaining the target text, for example: removing symbols and stop words from the text to be processed to obtain the intermediate text; removing abnormal text styles from the intermediate text to obtain the target text.

[0054] S202, determining a sparse coding matrix corresponding to the target word according to the target word, the preset coding matrix and the two-dimensional coordinates of a plurality of preset texts included in the pre-obtained semantic space.

[0055] In the present invention, the number of target words may be one or more.

[0056] When there are multiple target words, for each target word, a sparse coding matrix corresponding to the target word is determined according to the target word, the preset coding matrix and the two-dimensional coordinates of multiple preset texts included in the pre-obtained semantic space.

[0057] The preset coding matrix may be a P*P matrix, where P may be 128, 256, etc.

[0058] The values ​​in the preset coding matrix are all second preset values. For example, the second preset value may be 0, 1, etc.

[0059] In some embodiments, S202 may specifically include: obtaining the target coordinates of the preset text including the target word from the two-dimensional coordinates of multiple preset texts; setting the numerical value at the position corresponding to the target coordinate in the preset coding matrix to a first preset value, and obtaining a sparse coding matrix corresponding to the target word.

[0060] In the present invention, the target coordinates of a preset text including a target word are obtained from the two-dimensional coordinates of multiple preset texts; the numerical value at the position corresponding to the target coordinate in the preset coding matrix is ​​set to a first preset value to obtain a sparse coding matrix corresponding to the target word. This ensures that when the target word is a low-frequency word in the original text set, the sparse coding matrix corresponding to the target word can be obtained, thereby improving the accuracy of knowledge sparse distributed coding for the HTM model.

[0061] In some embodiments, the preset text including the target word can be determined by the following method: for each preset text among multiple preset texts, determine whether the target exists among the multiple words included in the preset text; if so, the preset text is the preset text including the target word; if not, the preset text is not the preset text including the target word.

[0062] In some embodiments, there is a preset interface between the natural language processing device and the database. When it is necessary to obtain predicted text of the text to be processed, the semantic space can be imported from the database through the preset interface to obtain the two-dimensional coordinates of multiple preset texts and multiple words included in each preset text.

[0063] The first preset value is different from the second preset value. For example, when the second preset value is 0, the first preset value is 1.

[0064] For example, when the text to be processed includes the target word "like" and the preset text is "The red dishes like the sun", since the preset text includes the target word, the target coordinates are the two-dimensional coordinates corresponding to the preset text.

[0065] Furthermore, when the second preset value is 0 and P=128, the preset encoding matrix is ​​a matrix with 128 rows and 128 columns. If the target coordinates are [4, 5], the value at the position of the fourth row and fifth column in the preset coding matrix (corresponding to the position of the target coordinates) is set to the first preset value (1) to obtain the sparse coding matrix corresponding to the target word.

[0066] In the present invention, when the number of target words included in the text to be processed is multiple, for each target word, the number of preset texts including the target word may be multiple.

[0067] When there are multiple preset texts including the target word, the value at the position corresponding to the target coordinate of each preset text in the preset coding matrix is ​​set to a first preset value to obtain a sparse coding matrix corresponding to the target word.

[0068] S203, processing the sparse coding matrix through the HTM model to obtain a predicted text corresponding to the text to be processed.

[0069] exist Figure 2 In the natural language processing method provided by the embodiment, the sparse coding matrix corresponding to the target word is determined according to the target word, the preset coding matrix and the two-dimensional coordinates of multiple preset texts included in the pre-obtained semantic space, wherein the data structure of the sparse coding matrix is ​​SDR, so the sparse coding matrix can be processed by the HTM model, so that the HTM model can output the predicted text corresponding to the text to be processed.

[0070] In the present invention, an intermediate text is obtained by performing abnormal text style elimination processing on the text to be processed; a symbol elimination processing and a stop word elimination processing are performed on the intermediate text to obtain a target text; a word splitting processing is performed on the target text to obtain the target words included in the text to be processed, so that the sparse coding matrix can be a sparse coding matrix of the target words with actual meanings, thereby improving the accuracy of the predicted text output by the HTM model.

[0071] In addition, the text to be processed in the present invention can be a sentence written in a natural language such as Chinese or English, so that the natural language processing method provided by the present invention has a good application prospect.

[0072] Based on the above embodiments, the present invention also provides a method for obtaining a semantic space. Figure 3 The embodiment describes a method for obtaining a semantic space.

[0073] Figure 3 FIG. 1 is a flow chart of obtaining a semantic space provided by the present invention. Figure 3 As shown, the method includes:

[0074] S301, obtaining a plurality of preset texts included in an original text set.

[0075] The original text set may be a text set in any field (eg, general field, earth observation field, earth system field). For a certain field, the original text set includes relevant books and literature materials in the field.

[0076] In some embodiments, the plurality of preset texts included in the original text set may be obtained by the following method:

[0077] The texts with abnormal text styles, texts that do not include information, and texts that are difficult to extract information from the original text set are eliminated to obtain a first text set (i.e., the text set is obtained after noise elimination); sentence segmentation is performed on the first text set to obtain multiple texts to be processed; preset symbols and stop words in the multiple texts to be processed are eliminated to obtain multiple preset texts.

[0078] In some embodiments, the above method of obtaining multiple preset texts can also be modified to obtain a new method. For example: remove texts with abnormal text styles, texts that do not contain information, and texts that are difficult to extract information from the original text set to obtain a first text set; remove preset symbols and stop words in the first text set to obtain a second text set; perform sentence segmentation processing on the second text set to obtain multiple preset texts.

[0079] S302, performing word splitting processing and statistical sorting processing on multiple preset texts to obtain a dictionary; the dictionary includes multiple words included in the multiple preset texts, and the multiple words are arranged in the dictionary according to the order of appearance in the original text set.

[0080] Among them, the multiple words included in the dictionary are different from each other.

[0081] For example, when the plurality of preset texts include "like apple" and "eat orange", the following dictionary may be obtained by performing word splitting processing and statistical sorting processing on the plurality of preset texts: like, apple, eat, orange.

[0082] S303, determining the unique hot vectors of the multiple words in the dictionary.

[0083] In some embodiments, S303 may include: determining a preset coding vector according to the total number of the plurality of words included in the dictionary; the length of the preset coding vector is equal to the total number, and the values ​​in the preset coding vector are all second preset values;

[0084] For each word, the value at the position corresponding to the index of the word in the dictionary in the preset encoding vector is set to a first preset value to obtain a one-hot vector of the word.

[0085] For example, the dictionary includes like, apple, eat, orange, with a total of 4 words, then the one-hot vector of like is [1, 0, 0, 0].

[0086] S304, determining two-dimensional coordinates of a plurality of preset texts according to the unique hot vectors of the plurality of words.

[0087] In some embodiments, according to the unique hot vectors of the plurality of words, the model parameters of the Continuous Bag-of-Words (CBOW) model are adjusted to obtain a weight matrix from the input layer to the middle layer of the CBOW model;

[0088] According to the weight matrix, determine the encoding vectors of the multiple words respectively;

[0089] For each preset text among multiple preset texts, the sum of the encoding vectors of the words included in the preset text is determined as the initial semantic vector of the preset text; the initial semantic vector is dimensionally reduced through a preset dimensionality reduction model to obtain a two-dimensional semantic vector of the preset text; the two-dimensional semantic vector is scaled and rounded to obtain a two-dimensional coordinate of the preset text.

[0090] In some embodiments, the dimension of the weight matrix is ​​equal to the total number of multiple words; according to the weight matrix, the encoding vectors of each of the multiple words are determined, including: for the i-th word among the multiple words, the i-th dimension vector in the weight matrix is ​​determined as the encoding vector of the i-th word; i is an integer greater than or equal to 1 and less than or equal to the total number.

[0091] The CBOW model includes an input layer, an intermediate layer and an output layer. Since the weight matrix from the input layer to the intermediate layer is used to measure the encoding information, and the weight matrix from the intermediate layer to the output layer is used to measure the prediction information, the encoding vectors of the multiple words are determined according to the weight matrix from the input layer to the intermediate layer in the present invention.

[0092] In some embodiments, the preset dimensionality reduction model may be a t-SNE algorithm model.

[0093] In the present invention, the initial semantic vector of the preset text is dimensionality reduced by a preset dimensionality reduction model to obtain a two-dimensional semantic vector of the preset text. This can make semantic vectors that are closer in high-dimensional space still closer in low-dimensional space, and the encodings of semantically similar words still have similar semantics, thereby realizing knowledge sparse distributed coding.

[0094] In some embodiments, for an nth preset text among a plurality of (eg, N) preset texts, the two-dimensional coordinates of the nth preset text may be obtained by the following formula:

[0095]

[0096] Among them, (x p ,y p ) n is the two-dimensional coordinate of the nth preset text, [] represents rounding operation, [x n ,y n ] is the two-dimensional semantic vector of the nth preset text, μ is a small preset value greater than 0, x max is the largest first value among the two-dimensional semantic vectors of N preset texts, x min is the smallest first value among the two-dimensional semantic vectors of N preset texts, y max is the second largest value among the two-dimensional semantic vectors of N preset texts, y min is the smallest second value among the two-dimensional semantic vectors of N preset texts. n ,y n ], the first value is x n , the second value is y n .

[0097] S305: Determine the two-dimensional coordinates of the plurality of preset texts and the words included in each of the plurality of preset texts as a semantic space.

[0098] In some embodiments, word splitting processing is performed on each preset text among a plurality of preset texts to obtain the words included in the preset text.

[0099] In the present invention, the original text set can be a text set in any field. When the original text set is a text set in a certain field, according to Figure 3 The method shown in the embodiment can obtain the semantic space corresponding to the field, so that the construction process of the semantic space is flexible and scalable and can be applied to different fields.

[0100] Figure 4 Schematic diagram of the structure of the natural language processing device provided by the present invention. Figure 4 As shown, the natural language processing device includes:

[0101] An acquisition module 410 is used to acquire a target word included in the text to be processed;

[0102] A determination module 420, configured to determine a sparse coding matrix corresponding to a target word according to the target word, a preset coding matrix and two-dimensional coordinates of a plurality of preset texts included in a pre-obtained semantic space;

[0103] The processing module 430 is used to process the sparse coding matrix through the HTM model to obtain the predicted text corresponding to the text to be processed.

[0104] In some embodiments, the determination module 420 is specifically used to: obtain the target coordinates of the preset text including the target word from the two-dimensional coordinates of multiple preset texts; set the numerical value at the position corresponding to the target coordinate in the preset coding matrix to a first preset value, and obtain the sparse coding matrix corresponding to the target word.

[0105] In some embodiments, the acquisition module 410 is specifically used to: perform abnormal text style elimination processing on the text to be processed to obtain an intermediate text; perform symbol elimination processing and stop word elimination processing on the intermediate text to obtain a target text; perform word splitting processing on the target text to obtain target words included in the text to be processed.

[0106] In some embodiments, the determination module 420 is also used to: obtain multiple preset texts included in the original text set; perform word splitting and statistical sorting on the multiple preset texts to obtain a dictionary; the dictionary includes multiple words included in the multiple preset texts, and the multiple words are arranged in the dictionary according to the order of appearance in the original text set; determine the unique hot vectors of each of the multiple words in the dictionary; and determine the two-dimensional coordinates of the multiple preset texts included in the semantic space based on the unique hot vectors of each of the multiple words.

[0107] In some embodiments, the determination module 420 is also specifically used to: adjust the model parameters of the continuous bag-of-words CBOW model according to the unique hot vectors of each of the multiple words, and obtain the weight matrix from the input layer to the middle layer of the CBOW model; determine the encoding vectors of each of the multiple words according to the weight matrix; for each of the multiple preset texts, determine the sum of the encoding vectors of the words included in the preset text as the initial semantic vector of the preset text; perform dimensionality reduction processing on the initial semantic vector through a preset dimensionality reduction model to obtain a two-dimensional semantic vector of the preset text; perform scaling and rounding processing on the two-dimensional semantic vector to obtain the two-dimensional coordinates of the preset text.

[0108] In some embodiments, the dimension of the weight matrix is ​​equal to the total number of multiple words; the determination module 420 is also specifically used to: for the i-th word among the multiple words, determine the i-th dimension vector in the weight matrix as the encoding vector of the i-th word; i is an integer greater than or equal to 1 and less than or equal to the total number.

[0109] Figure 5 Schematic diagram of the structure of the electronic device provided by the present invention. The schematic diagram is a physical structure diagram of the electronic device. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the natural language processing method, which includes: obtaining a target word included in the text to be processed; determining a sparse coding matrix corresponding to the target word according to the target word, the preset coding matrix and the two-dimensional coordinates of multiple preset texts included in the pre-obtained semantic space; processing the sparse coding matrix through the hierarchical real-time memory HTM model to obtain a predicted text corresponding to the text to be processed.

[0110] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0111] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the natural language processing method provided by the above methods, which includes: obtaining a target word included in the text to be processed; determining a sparse coding matrix corresponding to the target word based on the target word, a preset coding matrix and two-dimensional coordinates of multiple preset texts included in a pre-obtained semantic space; processing the sparse coding matrix through a hierarchical real-time memory HTM model to obtain a predicted text corresponding to the text to be processed.

[0112] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the natural language processing method provided by the above-mentioned methods, the method comprising: obtaining a target word included in the text to be processed; determining a sparse coding matrix corresponding to the target word according to the target word, a preset coding matrix and two-dimensional coordinates of multiple preset texts included in a pre-obtained semantic space; processing the sparse coding matrix through a hierarchical real-time memory HTM model to obtain a predicted text corresponding to the text to be processed.

[0113] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0114] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A natural language processing method, characterized in that: include: Obtaining target words included in the text to be processed; Determine a sparse coding matrix corresponding to the target word according to the target word, the preset coding matrix and the two-dimensional coordinates of a plurality of preset texts included in the pre-obtained semantic space; The sparse coding matrix is ​​processed by a hierarchical real-time memory (HTM) model to obtain a predicted text corresponding to the text to be processed; The step of determining the sparse coding matrix corresponding to the target word according to the target word, the preset coding matrix and the two-dimensional coordinates of the plurality of preset texts included in the pre-obtained semantic space comprises: Acquire the target coordinates of the preset text including the target word from the two-dimensional coordinates of the plurality of preset texts; The numerical value at the position corresponding to the target coordinate in the preset coding matrix is ​​set to a first preset value to obtain a sparse coding matrix corresponding to the target word; The method further comprises: Acquire the plurality of preset texts included in the original text set; Performing word splitting and statistical sorting processing on the plurality of preset texts to obtain a dictionary; the dictionary includes the plurality of words included in the plurality of preset texts, and the plurality of words are arranged in the dictionary according to the order of appearance in the original text set; Determining a one-hot vector for each of the plurality of words in the dictionary; Determining two-dimensional coordinates of a plurality of preset texts included in the semantic space according to the respective one-hot vectors of the plurality of words; The determining, according to the unique hot vectors of the plurality of words, the two-dimensional coordinates of the plurality of preset texts included in the semantic space comprises: According to the unique hot vectors of the multiple words, the model parameters of the continuous bag of words CBOW model are adjusted to obtain a weight matrix from the input layer to the middle layer of the CBOW model; Determining encoding vectors of the respective words according to the weight matrix; For each preset text among multiple preset texts, the sum of the encoding vectors of the words included in the preset text is determined as the initial semantic vector of the preset text; the initial semantic vector is subjected to dimensionality reduction processing through a preset dimensionality reduction model to obtain a two-dimensional semantic vector of the preset text; the two-dimensional semantic vector is scaled and rounded to obtain the two-dimensional coordinates of the preset text.

2. The method according to claim 1, characterized in that The step of obtaining the target words included in the text to be processed includes: Performing abnormal text style elimination processing on the text to be processed to obtain an intermediate text; Performing symbol removal processing and stop word removal processing on the intermediate text to obtain a target text; The target text is subjected to word splitting processing to obtain target words included in the text to be processed.

3. The method according to claim 1, characterized in that: The dimension of the weight matrix is ​​equal to the total number of the plurality of words; Determining the encoding vectors of the respective words according to the weight matrix includes: For an i-th word among the multiple words, determining an i-th dimension vector in the weight matrix as a coding vector of the i-th word; The i is an integer greater than or equal to 1 and less than or equal to the total number.

4. A natural language processing device for executing the natural language processing method according to any one of claims 1 to 3, characterized in that: include: An acquisition module, used for acquiring target words included in the text to be processed; A determination module, configured to determine a sparse coding matrix corresponding to the target word according to the target word, a preset coding matrix and two-dimensional coordinates of a plurality of preset texts included in a pre-obtained semantic space; The processing module is used to process the sparse coding matrix through a hierarchical real-time memory HTM model to obtain a predicted text corresponding to the text to be processed.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the natural language processing method according to any one of claims 1 to 3 is implemented.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the natural language processing method according to any one of claims 1 to 3 is implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the natural language processing method according to any one of claims 1 to 3 is implemented.