Logical reasoning problem-oriented anthropomorphic thinking-driven embedding enhancement method

Through anthropomorphic thinking-driven embedding enhancement method, the logical semantic filtering vectors and semantic unit relationship diagrams are used to solve the problem of low accuracy in logical reasoning problems, and achieve higher semantic attention and long-distance relationship modeling. Improved the accuracy of multi-choice logical reasoning.

CN119025646AActive Publication Date: 2024-11-26UNIV OF ELECTRONICS SCI & TECH OF CHINA

Patent Information

Application Number
CN202411102286.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-26
Estimated Expiration
2044-08-12

AI Technical Summary

Technical Problem

Existing neural network models perform poorly in logical reasoning problems, especially inadequate attention to the problem-assignment method and insufficient logical performance capabilities in text semantic unit modeling, resulting in low accuracy on multiple-choice logical reasoning problems.

Method used

Adopting anthropomorphic thinking-driven embedding enhancement method, through data cleaning and formatting processing, using pre-trained language models and self-attention neural networks to generate logical semantic filtering vectors, perform embedding representation enhancement of the question stem and options, and establish a semantic unit relationship diagram , realize information interaction and embedding update, and finally calculate the correct probability of the option.

Benefits of technology

It improves the accuracy of neural networks in logical reasoning problems, enhances attention to semantic information and modeling capabilities of long-distance semantic unit relationships, and improves the ability to solve multiple-choice logical reasoning problems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025646A_ABST
    Figure CN119025646A_ABST
Patent Text Reader

Abstract

The invention discloses a logical reasoning question-oriented human problem solving thinking-driven embedding enhancement method, and relates to a multi-option logical reasoning question and answer question in the field of natural language processing. The method aims at solving the technical problems that in an existing logical reasoning question and answer problem solving method, text structure analysis is not clear, problem related elements are insufficient in utilization, and logical reasoning problem semantic mining difficulty is large. The method mainly comprises the steps of preprocessing data, arranging examples into a text sequence suitable for neural network model processing, and designing and driving a question meaning-based embedding and updating module based on human question solving thinking, and comprises the following steps of: firstly, initializing character embedding representation, and performing vectorization representation on characters in the examples by using a pre-training language model; then, a logic information filtering vector is constructed by using an embedded sequence of a question part, then the filtering vector is used for updating embedded representation of question stems and options, and a basic semantic unit of a neural network is analyzed by referring to a semantic relationship modeling module of a human reading habit;
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the application of neural networks and deep learning technologies in the field of human natural language reading comprehension, and in particular to an anthropomorphic thinking-driven embedding enhancement method for logical reasoning problems. Background Art

[0002] Natural language is an important tool for human expression and communication. Training machines to understand natural language and perform natural language processing tasks is a long-term goal of the development of artificial intelligence technology. Especially with the continuous deepening of deep learning technology research, natural language processing technology has brought great convenience to users in the fields of information retrieval, machine translation and language recognition.

[0003] The development of technology has given rise to higher requirements for artificial intelligence systems. From the perspective of machine reading comprehension research, with the birth of Transformer and a series of pre-trained language models, the machine model's ability to understand and generate text has been significantly improved. However, when faced with some problems that require reasoning ability, the performance of current neural network models is not satisfactory, and even makes jokes about "artificial mental retardation". Researchers have proposed a variety of reasoning question and answer data sets including common sense reasoning, mathematical reasoning, and logical reasoning to examine the reasoning ability of the model. Among them, the question form of the logical reasoning data set is the most complex and relatively close to the user's usage habits, which is of great reference significance for the study of reasoning question and answering technology.

[0004] The present invention is based on a neural network method for natural language processing. By designing a problem-guided embedding representation enhancement method, the embedding representation capability of the neural network model for natural language text is enhanced, and then the semantic units of the question stem and options are logically divided to achieve further information updating, thereby realizing an effective and novel multiple-choice logical reasoning problem-solving method.

[0005] With the development of artificial intelligence technology, the logical reasoning ability of neural network models is becoming more and more important. For such problems, the current mainstream research ideas are: 1. Improve the reading comprehension ability of the model. 2. Model the relationship graph structure of the samples to be solved, and use the neural network to train and update the embedded information according to the modeled structure. 3. Perform data enhancement on the samples according to certain rules, and then give them to the neural network model for reasoning and prediction.

[0006] However, the above methods all have the problem of insufficient attention to the logical reasoning problem itself. Most methods rigidly divide the text into blocks, ignoring the fact that in such logical reasoning datasets, "problems" have different types, which is crucial for understanding semantics when humans solve problems. At the same time, some modeling methods lack the ability to model long-distance semantic unit relationships. Summary of the invention

[0007] Based on neural networks, the present invention proposes an embedding enhancement method driven by human problem-solving thinking for logical reasoning problems to address the above shortcomings. The method aims to solve the problem of insufficient attention to the way of asking questions in the prior art, as well as the technical problems of insufficient logical expression ability and insufficient association ability in the process of modeling text semantic units.

[0008] To achieve the above object, the present invention provides the following technical solution: an anthropomorphic thinking-driven embedding enhancement method for logical reasoning problems, comprising the following steps:

[0009] Step 1: Clean the data of the sample to be solved to ensure the correctness of the input question itself, then combine the stem, question and options according to the number of options, complete the data formatting of the input data, and record the sentence component to which each character in the sample belongs;

[0010] Step 2: Use the pre-trained language model to vectorize the input natural language text sequence to obtain the initial embedding representation of each character, and then obtain the embedding representation of the character sequence corresponding to the question part based on the marked position information. Use the embedding representation of the question part to encode the logical semantic filter vector using the self-attention neural network structure, and then use the logical semantic filter vector to perform logical enhancement on the embedding representation of the question stem and options.

[0011] Step 3: Divide the input natural language text sequence into logical semantic units, and establish a relationship graph structure between the semantic units in the input text; calculate the embedded representation of the entire semantic unit according to different weights for each character in a semantic unit; and realize information interaction between semantic units based on the graph structure of the semantic units of the input text sequence.

[0012] Step 4: Calculate the correct probability of the option. Based on the embedded representation of the updated text sequence obtained in the previous step, use the fully connected neural network layer to calculate the correct probability of the option. Finally, give the answer judgment of the input question sample.

[0013] Preferably, the step 1 comprises the following steps:

[0014] Step 1.1: Perform grammar check on the stem, question, and options of the input sample, and use a generative large-scale language model to perform semantic check;

[0015] Step 1.2: According to the number of options in the input sample, copy the question stem and question several times, and splice them with each option to form a text sequence input into the neural network model.

[0016] Step 1.3: Format the data in the form of a text sequence and record it in a structure array. For example, mark the positions of the characters in the text sequence, specifically record which characters belong to the stem, question, and options, and record them in their respective tag arrays.

[0017] Preferably, the step 2 comprises the following steps:

[0018] Step 2.1, set the parameters of the pre-trained language model, vectorize the input text sequence, and obtain the initialization embedding representation corresponding to the text sequence;

[0019] Step 2.2, extract the embedded representation of the characters contained in the text sequence of the problem part according to the position marker array;

[0020] Step 2.3: Construct a sequence encoding neural network based on the self-attention mechanism, encode the features of the question sequence, and generate a filter vector for the question stem information based on the question;

[0021] Step 2.4: Reconstruct a sequence encoding neural network with the same structure, perform feature encoding on the question sequence, and generate a filter vector for option information based on the question;

[0022] Step 2.5: Use the logical information filter vector to calculate the character embedding representation in the question stem or option sequence to obtain the updated embedding;

[0023] Step 2.6: Use the residual neural network to add the initial embedding representation and the updated embedding representation together according to the learned weights.

[0024] Preferably, the step 3 comprises the following steps:

[0025] Step 3.1: Divide the text sequence of the input sample into semantic units according to the rules, and record the association relationship between the semantic units;

[0026] Step 3.2, based on the relationship between the semantic units, establish a logical relationship diagram reflecting the semantic units in the text sequence;

[0027] Step 3.3: Calculate the embedding representation of each semantic unit based on the embedding representations of the characters contained in the semantic unit.

[0028] Step 3.4: Calculate the overall embedding representation of the question stem and options based on the embedding representation of the characters contained in the question stem and options.

[0029] Step 3.5: According to the logical relationship diagram of semantic units, information interaction is performed between semantic units; information interaction is performed between the question stem and the semantic units before the options; and the embedding update of semantic units is completed.

[0030] Step 3.6: Based on the correspondence between characters and semantic units, the question-enhanced character embedding representation obtained in step 2 and the semantic unit embedding representation obtained in step 3 are used to further update the embedding representation of the characters in the text sequence.

[0031] Preferably, the step 4 comprises the following steps:

[0032] Step 4.1, using a linear neural network, based on the embedding representation of the characters in the text sequence, calculate the weight vector of each character in the text sequence;

[0033] Step 4.2: According to the position information of the question stem and options recorded in the position information, extract the weights of the corresponding parts in the weight vector, and normalize them to obtain the question stem information extraction weight vector and the option information extraction weight vector.

[0034] Step 4.3: Pool the embedded representations of the characters at corresponding positions in the text sequence according to their respective weight vectors, and concatenate the pooled vectors to obtain the evaluation feature vector.

[0035] Step 4.4: Set the random dropout layer, linear neural network layer, regularization layer, activation function layer, and linear neural network layer in sequence, integrate and evaluate the feature vector information, and give the calculation results.

[0036] Step 4.4: According to the calculation results of the evaluation feature vectors of the four options, softmax normalization processing is performed to obtain the final correct probability of the option.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] In view of the common problem of lack of attention to "question" text information in current mainstream research methods, this method proposes a problem-based embedding representation enhancement method from the perspective of human beings solving logical reasoning problems. By extracting and re-encoding the word embedding representation of the "question" sentence sequence, a filter vector based on the word embedding of the "question" sequence is constructed, and the word embedding representation of the question stem and the option part is enhanced respectively, thereby achieving targeted enhancement of semantic information related to problem solving. At the same time, the discourse graph structure modeling method is improved, which improves the problem of difficult information exchange between long-distance units in the traditional discourse graph structure modeling process. Based on deep learning technology, the present invention conducts in-depth research on logical reasoning in natural language processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The present invention will be further described below in conjunction with the accompanying drawings:

[0042] Example 1

[0043] The data source of the example is the Chinese National Civil Service Examination test questions and the open domain of the English Internet. It contains the question stem information, question information, and option information. The specific input example structure and example content are as follows:

[0044]

[0045] like Figure 1 As shown in the figure, it is the overall flow chart of the whole method. The present invention includes four major steps: data preprocessing, question-guided embedding update, discourse-level embedding update, and option probability prediction. Each major step contains several small steps. Preprocessing is to prepare for the subsequent steps; question-guided embedding update is to construct a filtering vector based on the question text in the input sample to enhance the embedding of the question stem and options; discourse-level embedding update is to model the logical relationship of the text sequence at the text discourse level, and on this basis, to perform information interaction and embedding update; option probability prediction is to extract the evaluation vector of the final calculation score based on the final embedding representation of the text sequence, and normalize it to a specific probability value.

[0046] First, clean the source data to ensure that the input question format is complete and there are no interfering characters. Then format the data to meet the input requirements of subsequent steps and record the position information of each character in the character sequence. The details are as follows:

[0047] 1.1.1 Format verification: confirm that the data structure of the input instance is complete and there are no incorrect characters.

[0048] 1.1.2 Use a generative large-scale language model to rewrite the input text and correct any semantic errors, logical errors, and typos in the text.

[0049] 1.2 Model input construction: The question stem, question, and options are concatenated and processed into a text sequence suitable for the model to receive. For example, for an input instance with four options, the input is constructed into four input text sequences in the following format: <s> Title< / s> Question Option A; <s> Title< / s> Question option B;<s> Title< / s> Question option C; <s> Title< / s> Question option D.

[0050] in, <s>、< / s> Specific characters preset for the RoBERTa pre-trained language model. Set the maximum length of text to 256, and input sequences longer than 256 will be truncated.

[0051] 1.3 Format the information of the input text sequence and store it in a structure. According to the RoBERTa dictionary, record the numbers of the characters in the text sequence in the dictionary and the character positions contained in different sentence components.

[0052]

[0053] In step 2, the present invention designs a problem-guided representation enhancement method according to the specific characteristics of the logical reasoning problem. First, the text sequence is initialized for vectorized expression. Secondly, the character embedding representation of each sequence is extracted according to different sentence components. Then, the logical semantic filtering vector is obtained based on the embedding representation of the problem part, thereby realizing the embedding representation enhancement of the instance text sequence, as shown below:

[0054] 2.1 According to the input text sequence number and character position mask, use the RoBERTa pre-trained language model, and choose to use the RoBERTa-Large version based on the hardware conditions to obtain the embedded representation of each character in the input text sequence.

[0055] 2.2 Based on the obtained embedding representation of each character in the text sequence and the problem position information mask in the data information, the neural network design is used to encode it to obtain the problem-based representation enhancement filter vector. The specific steps are:

[0056] 2.2.1 Read the problem position code in the input data, which is a one-dimensional vector of length 256, in which the value is 1 or 0. 1 represents that the character belongs to the "problem" part.

[0057] 2.2.2 According to the value of the problem position code, the embedded representation of the characters contained in the problem part is taken out from the text sequence character code.

[0058] 2.2.3 Input the character embedding of the question into the "question stem filter vector generation module". It contains a Transformer encoder architecture network based on the self-attention mechanism. Set the network structure to two layers. Obtain the intermediate representation of the question stem filter vector.

[0059] 2.2.4 Use the intermediate representation of the option filter vector and the information encoded in the stem position to normalize the intermediate representation of the filter vector at the position of the characters contained in the stem to obtain a stem embedding representation filter vector with the same length as the stem text sequence.

[0060] 2.2.5 Input the character embedding of the question part into the "option filter vector generation module". It contains a Transformer encoder architecture network based on the self-attention mechanism. Set the network structure to two layers. Obtain the intermediate representation of the option filter vector.

[0061] 2.2.6 Using the intermediate representation of the option filter vector, normalize the intermediate representation of the filter vector at the position of the characters contained in the option according to the information encoded in the option position, and obtain the option embedding representation filter vector with the same length as the option text sequence.

[0062] 2.3 Use the obtained logical semantic filter vector to enhance the embedding representation of the characters in the question stem and options based on the basic embedding representation of the characters obtained during the initialization process. The specific steps are as follows:

[0063] 2.3.1 Based on the value of the stem position code, extract the embedded representation of the characters contained in the stem part from the text sequence character code.

[0064] 2.3.2 Use the question stem embedding representation to multiply the filter vector by the embedding representation of the question stem characters to obtain the question stem embedding representation of the logical filtering part.

[0065] 2.3.3 Input the original stem character embedding representation and the stem embedding into the residual neural network together, set the weight value between the two to a total of 1, and obtain the final updated stem character embedding representation, which integrates the basic semantic information and the semantic information enhanced by logical semantics.

[0066] 2.3.4 Based on the value of the option position code, extract the embedded representation of the characters contained in the option part from the text sequence character code.

[0067] 2.3.5 Use option embedding to represent the filter vector and multiply the embedding representation of the option character to obtain the option embedding representation of the logical filter part.

[0068] 2.3.6 Input the original option character embedding representation and the option embedding into the residual neural network together, set the weight value between the two to a sum of 1, and obtain the final updated option character embedding representation.

[0069] 2.3.7 Finally, the updated character embedding representation of the question stem, the embedding representation of the question part obtained during the initialization process, and the updated character embedding representation of the options are concatenated together to obtain the embedding representation of the entire text sequence after logical semantic enhancement based on the question.

[0070] When the logical semantic embedding representation enhancement process in step 2 is performed to obtain the updated embedding representation, the present invention enters the step of learning the embedding representation based on the semantic association relationship at the discourse structure level. The entire embedding representation update process is divided into three steps: text sequence structure division, discourse structure modeling and information interaction. The specific implementation of each step is as follows:

[0071] 3.1 Text sequence structure division: This step is to further divide the text sequence structure and annotate the characteristics of each structure based on the basic information obtained during the data initialization process. The specific steps are as follows:

[0072] 3.1.1 Read the original text data of the text sequence, and annotate the keywords that divide different sentence components according to the specified specific keywords, such as the keywords "if", "but" and "because" and similar prepositional keywords, and mark the subject and object of the relationship according to the relationship type, record the type of relationship, and obtain the position and attribute information of the relationship words.

[0073] 3.1.2 Read the punctuation marks in the text sequence, record their positions, and obtain the punctuation positions.

[0074] 3.1.3 Divide the text sequence into several different semantic units according to the position of relational words and punctuation marks, for example: "If it doesn't rain tomorrow, we'll go hiking." Among them, "It doesn't rain tomorrow" is a semantic unit, and "We'll go hiking" is another semantic unit. After dividing all the semantic units, record the number of semantic units and the character sequence tags contained in each semantic unit.

[0075] 3.1.4 According to the number of semantic units, punctuation position information, and the position and attribute information of relational words, a relationship matrix between different semantic units is established, where the value of the relationship matrix is ​​determined by whether there is an association and the type of relational words.

[0076] 3.2 After completing the division of each semantic unit in the text sequence, a discourse structure diagram based on the text sequence is constructed according to the division results of the semantic units. During the construction process, the basic nodes are the semantic units obtained in the previous step. The association between the semantic units is represented by the value of the relationship matrix. For character sequences of different sentence components, a long-distance information interaction mechanism is implemented to ensure that the information of the semantic units with the same subject can be interacted. The specific steps are as follows:

[0077] 3.2.1 According to the number of semantic units, the embedded representation of each semantic unit is established one by one. Specifically, the embedded representation of the characters contained in each semantic unit is read, and then the same number of learning weight values ​​are set, the sum of these weights is 1, and then the embedded representation of the characters in the semantic unit is multiplied by their respective weights and added up to form the embedded representation of the semantic unit.

[0078] 3.2.2 Abstract each semantic unit as a node in the graph, based on the relationship matrix between different semantic units obtained in 3.1.4. If the attributes of the relationship edge are determined according to the type of relationship word, and different connection edges are set according to the type of punctuation marks, the graph structure of the semantic unit is obtained.

[0079] 3.2.3 For the character sequences contained in different sentence components, they are further divided into three major partitions in the graph structure of semantic units, namely the stem, the question, and the options. Each part contains all the basic semantic units belonging to that part.

[0080] 3.2.4 For the partitioning of the question stem and options, use all the basic semantic units contained therein, set a number of learned weight values, the sum of which is 1, and then multiply the embedded representation of the semantic unit with their respective weights and add them up to form the embedded representation of the partitioning of the question stem and options.

[0081] 3.3 The above steps complete the establishment of the discourse graph structure of the text sequence and represent the embedded representation of each node and each partition. Furthermore, based on the modeling results of the graph structure of the text sequence, this method performs an overall update of the embedded representation of all characters in the text sequence at the discourse structure level. The specific implementation steps are as follows:

[0082] 3.3.1 Read the embedded representation of each semantic unit obtained in step 3.2.1.

[0083] 3.3.2 Based on the graph structure of semantic units obtained in step 3.2.2, calculate the degree to which each semantic unit is affected by other semantic units. Consider each semantic unit as a node, calculate its neighboring nodes according to the relationship matrix, and use a linear neural network to calculate the intermediate calculation value of the weight based on the embedded representation of the node.

[0084] 3.3.3 Based on the embedded representation of the question stem and option partitions obtained in step 3.2.4, use a layer of linear neural network to calculate the degree of influence of each node in another partition and obtain the intermediate value of the weight of the partition influence.

[0085] 3.3.4 Based on the median value of the weight of a node affected by other nodes or partitions obtained in the above two sub-steps, use the normalization function to calculate the weight value of the node affected by other nodes or partitions.

[0086] 3.3.5 Based on the calculated weight values, use the embedding representations obtained in steps 3.2.1 and 3.2.4 to calculate the impact of each neighbor node and another partition on the embedding update of the current node.

[0087] 3.3.6 The embedding representation of the current node is encoded by a layer of linear neural network, and then added to the embedding update vectors brought by other nodes and partitions obtained in step 3.3.5, and the activation function is executed to introduce nonlinear changes to the embedding update process, and finally the vector representation of the current semantic unit after the embedding update based on the text structure is obtained.

[0088] After completing the 3-step semantic unit embedding update based on the text sequence structure, in the last step, the model integrates the embedding vectors of different dimensions obtained in steps 2 and 3, performs pooling on the character embedding representation of the text sequence, reduces its feature dimension through a multi-layer neural network, and finally obtains the calculation score of the text sequence. After obtaining the calculation scores of the text sequences of all options, the normalization function is used to calculate the correct probability of each option and give the model output. The specific steps are as follows:

[0089] 4.1 Obtain the embedding representation of the overall updated character based on the embedding vectors of different dimensions obtained in step 2 and step 3. The sub-steps are as follows:

[0090] 4.1.1 In step 2, the model obtains the embedding of each character in the text sequence after being enhanced by the question semantic representation. Here, each character is retrieved to which semantic unit established in step 3 it belongs.

[0091] 4.1.2 Set a unified semantic unit embedding representation to update the weight values ​​during training.

[0092] 4.1.3 Add the embedding representation of the semantic unit to which the character belongs to the character according to the weight value in step 4.1.2 to the embedding representation of the character itself.

[0093] 4.1.4 Obtain a vector representation of each character in the text sequence that combines the character-level embedding representation and the text sequence-level embedding representation.

[0094] 4.2 Perform a pooling operation on the embedded representation of the characters in the text sequence, compress the hidden layer features layer by layer through a multi-layer neural network, and finally obtain a calculation score that reflects the logical correctness of the option. The sub-steps are as follows:

[0095] 4.2.1 The text sequence embedding representation obtained in step 4.1.4 is passed through a layer of linear neural network, and the vector representation of the characters in the sequence is initially reduced in dimension through matrix multiplication.

[0096] 4.2.2 Set the activation function for the one-dimensional vector representation after dimensionality reduction.

[0097] 4.2.3 The vector processed by the activation function is input into a linear neural network again, and the dimension of the feature vector is reduced again to obtain the logical score of the option sequence.

[0098] 4.2.4 According to the above method, obtain the logical scores of all option sequences in turn.

[0099] 4.2.5 Based on the obtained logical scores of all options, use the normalization function to express the logical scores at the computational level as the correct probabilities of each option, and the sum of their probabilities is 1.

[0100] 4.3 Based on the final calculated probabilities of each option, the model outputs a judgment answer and gives the correct probability calculated for each option.

[0101] The present invention performs exception handling and data formatting on the data through preprocessing to make it meet the input results of the subsequent steps. Then, by referring to the starting point of humans when facing logical reasoning problems, modeling is performed based on the semantics of the question, and the embedded representation of relevant characters in the text sequence is enhanced, thereby realizing the intuitive thinking of humans to think about the problem from the perspective of the question. This ensures that the obtained embedded representation is more targeted for logical reasoning problems. The division of the text sequence as a whole into logical semantic units is based on the research of machine reading comprehension, and is aimed at the problem characteristics of logical reasoning problems, and further improves the problem of long-distance text dependency and logical relationship reinforcement. The subsequent option probability score calculation is performed by pooling and compressing the embedded representation of the sequence characters, and finally encodes the information contained in the hidden layer vector into the final evaluation score, thereby obtaining the final answer.

[0102] Based on neural networks, this invention uses deep learning technology to represent the natural language text information in multiple-choice formal logical reasoning question-answering questions from the perspective of human problem solving, and further strengthens it, solving the problem that some previous methods only focus on text structure splitting and cannot effectively model long-distance dependencies. This makes the results more accurate and effective. Experiments have shown that this method can improve the accuracy of neural network models in solving multiple-choice logical reasoning problems.

[0103] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0104] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An anthropomorphic thinking-driven embedding enhancement method for logical reasoning problems, characterized by: The following steps are involved: Step 1: Clean the data of the sample to be solved to ensure the correctness of the input question itself, then combine the stem, question and options according to the number of options, complete the data formatting of the input data, and record the sentence component to which each character in the sample belongs; Step 2: Use the pre-trained language model to vectorize the input natural language text sequence to obtain the initial embedding representation of each character, and then obtain the embedding representation of the character sequence corresponding to the question part based on the marked position information. Use the embedding representation of the question part to encode the logical semantic filter vector using the self-attention neural network structure, and then use the logical semantic filter vector to perform logical enhancement on the embedding representation of the question stem and options. Step 3: Divide the input natural language text sequence into logical semantic units, establish a relationship graph structure between the semantic units in the input text, calculate the embedded representation of the entire semantic unit according to different weights for each character in a semantic unit, and realize information interaction between semantic units based on the graph structure of the semantic units of the input text sequence. Step 4: Calculate the correct probability of the option. Based on the embedded representation of the updated text sequence obtained in the previous step, use the fully connected neural network layer to calculate the correct probability of the option, and finally give the answer judgment of the input question sample.

2. According to claim 1, the anthropomorphic thinking-driven embedding enhancement method for logical reasoning problems is characterized by: The step 1 comprises the following steps: Step 1.1: Perform grammar check on the stem, question, and options of the input sample, and use a generative large-scale language model to perform semantic check; Step 1.2: According to the number of options in the input sample, copy the question stem and question several times, and splice them with each option to form a text sequence input into the neural network model. Step 1.3: Format the data in the form of a text sequence and record it in a structure array. For example, mark the positions of the characters in the text sequence, specifically record which characters belong to the stem, question, and options, and record them in their respective tag arrays.

3. The method of embedding enhancement driven by anthropomorphic thinking for logical reasoning problems according to claim 1, characterized in that: The step 2 comprises the following steps: Step 2.1, set the parameters of the pre-trained language model, vectorize the input text sequence, and obtain the initialization embedding representation corresponding to the text sequence; Step 2.2, extract the embedded representation of the characters contained in the text sequence of the problem part according to the position marker array; Step 2.3: Construct a sequence encoding neural network based on the self-attention mechanism, perform feature encoding on the question sequence, and generate a filter vector for the question stem information based on the question; Step 2.4: Reconstruct a sequence encoding neural network with the same structure, perform feature encoding on the question sequence, and generate a filter vector for option information based on the question; Step 2.5: Use the logical information filter vector to calculate the character embedding representation in the question stem or option sequence to obtain the updated embedding; Step 2.6: Use the residual neural network to add the initial embedding representation and the updated embedding representation together according to the learned weights.

4. The method of embedding enhancement driven by anthropomorphic thinking for logical reasoning problems according to claim 1, characterized in that: The step 3 comprises the following steps: Step 3.1: Divide the text sequence of the input sample into semantic units according to the rules, and record the association relationship between the semantic units; Step 3.2, based on the relationship between the semantic units, establish a logical relationship diagram reflecting the semantic units in the text sequence; Step 3.3: Calculate the embedding representation of each semantic unit based on the embedding representations of the characters contained in the semantic unit. Step 3.4: Calculate the overall embedding representation of the question stem and options based on the embedding representation of the characters contained in the question stem and options. Step 3.5: According to the logical relationship diagram of semantic units, information interaction is performed between semantic units; information interaction is performed between the question stem and the semantic units before the options; and the embedding update of semantic units is completed. Step 3.6: Based on the correspondence between characters and semantic units, the question-enhanced character embedding representation obtained in step 2 and the semantic unit embedding representation obtained in step 3 are used to further update the embedding representation of the characters in the text sequence.

5. The method of embedding enhancement driven by anthropomorphic thinking for logical reasoning problems according to claim 1, characterized in that: The step 4 comprises the following steps: Step 4.1, using a linear neural network, based on the embedding representation of the characters in the text sequence, calculate the weight vector of each character in the text sequence; Step 4.2: According to the position information of the question stem and options recorded in the position information, extract the weights of the corresponding parts in the weight vector, and normalize them to obtain the question stem information extraction weight vector and the option information extraction weight vector. Step 4.3: Pool the embedded representations of the characters at corresponding positions in the text sequence according to their respective weight vectors, and concatenate the pooled vectors to obtain the evaluation feature vector. Step 4.4: Set the random dropout layer, linear neural network layer, regularization layer, activation function layer, and linear neural network layer in sequence, integrate and evaluate the feature vector information, and give the calculation results. Step 4.4: According to the calculation results of the evaluation feature vectors of the four options, softmax normalization processing is performed to obtain the final correct probability of the option.

Citation Information

Patent Citations

  • Intelligent question-answer method and device, equipment and computer readable storage medium

    CN113761154A

  • Time sequence knowledge graph question and answer method based on virtual time sequence constraint enhancement

    CN117521795A

  • Telecommunications - enabled semitransparent thought- directed cognitive and conceptual communication linkage method for a workplace brain / cognitive education, training, and augmentation program

    US20200234605A1

Cited By

  • Automatic driving large model framework based on 3D space-time perception and human-like decision-making reasoning

    CN120182938A