A human-like thought-driven embedded augmentation method for logical reasoning problems

Abstract

A human problem-solving thinking driven embedded enhancement method for logical reasoning problems, the present application relates to a multiple-choice logical reasoning question and answer problem in the field of natural language processing, which aims to solve the technical problems of unclear text structure analysis, insufficient use of problem related elements and difficult logical reasoning problem semantic mining in the existing logical reasoning question and answer problem solving method. The method mainly includes data preprocessing, arranging the sample into a text sequence suitable for neural network model processing, designing a question-based embedding update module based on human problem-solving thinking, first initializing character embedding representation, using a pre-trained language model to vectorize the characters in the sample, then using the embedding sequence of the question part to construct a logical information filtering vector, and then updating the embedding representation of the stem and options using the filtering vector, and a semantic relationship modeling module referring to human reading habits, analyzing the basic semantic unit of the neural network.

Description

Technical Field

[0001] This invention relates to the application of neural networks and deep learning technologies in the field of human natural language reading comprehension, and in particular to an embedding enhancement method driven by anthropomorphic thinking for logical reasoning problems. Background Technology

[0002] Natural language is an important tool for human expression and communication. Training machines to understand natural language and perform natural language processing tasks is a long-term goal of artificial intelligence technology development. In particular, with the continuous advancement of deep learning technology research, natural language processing technology has brought tremendous convenience to users in the fields of information retrieval, machine translation, and language recognition.

[0003] Technological advancements have spurred higher demands on artificial intelligence systems. From the perspective of machine reading comprehension research, the emergence of Transformers and a series of pre-trained language models has significantly improved machine models' ability to understand and generate text. However, when faced with questions requiring reasoning ability, current neural network models do not perform satisfactorily, sometimes even producing laughable "artificial stupidity." Researchers have proposed various reasoning question-answering datasets, including common sense reasoning, mathematical reasoning, and logical reasoning, to examine the reasoning capabilities of these models. Among these, the logical reasoning dataset presents the most complex question formats and is relatively closest to user habits, making it highly valuable for researching reasoning question-answering technology.

[0004] This invention is based on a neural network method of natural language processing. By designing a question-guided embedding representation enhancement method, the neural network model is enhanced to embed the representation of natural language text. Then, the semantic units of the question stem and options are logically divided to achieve further information updates, thereby realizing an effective and novel solution to multiple-choice logical reasoning problems.

[0005] With the development of artificial intelligence technology, the logical reasoning ability of neural network models is becoming increasingly important. Current mainstream research approaches for this type of problem include: 1. Improving the model's reading comprehension ability. 2. Modeling the relational graph structure of the examples to be solved, and then training and updating the embedded information of the neural network according to the modeled structure. 3. Performing data augmentation on the examples according to certain rules, and then feeding the data back into the neural network model for reasoning and prediction.

[0006] However, all of the above methods suffer from insufficient attention to the logical reasoning problem itself. Most methods rigidly segment the text, ignoring the fact that in such logical reasoning datasets, "problems" have different types, which is crucial for understanding semantics and is essential for human problem-solving. Furthermore, some modeling methods lack the ability to model long-distance semantic unit relationships. Summary of the Invention

[0007] Based on neural networks, this invention proposes an embedding enhancement method driven by human problem-solving thinking for logical reasoning problems, which aims to address the problem of insufficient attention to the questioning method in existing technologies, as well as the technical problems of insufficient logical expression and correlation ability in the process of text semantic unit modeling.

[0008] To achieve the above objectives, the present invention provides the following technical solution: an embedding enhancement method driven by anthropomorphic thinking for logical reasoning problems, comprising the following steps:

[0009] Step 1: Clean the data of the sample to be solved to ensure the correctness of the input question itself. Then, according to the number of options, concatenate the question stem, question and options to complete the data formatting of the input data, and record the sentence component to which each character in the sample belongs.

[0010] Step 2: Use a pre-trained language model to vectorize the input natural language text sequence to obtain the initial embedding representation of each character. Then, based on the labeled position information, obtain the embedding representation of the character sequence corresponding to the question part. Using the embedding representation of the question part, use a self-attention neural network structure to encode a logical semantic filtering vector. Then, use the logical semantic filtering vector to perform logical enhancement on the embedding representations of the question stem and option parts.

[0011] Step 3: Divide the input natural language text sequence into logical semantic units and establish a relationship graph structure between the semantic units in the input text; calculate the embedding representation of the overall semantic unit by assigning different weights to each character within a semantic unit; realize information interaction between semantic units based on the graph structure of the semantic units in the input text sequence.

[0012] Step 4: Calculate the correct probability of each option. Based on the updated text sequence embedding obtained in the previous step, a fully connected neural network layer is used to calculate the correct probability of each option. Finally, the answer to the input question sample is given.

[0013] Preferably, step 1 includes the following steps:

[0014] Step 1.1: Perform grammatical checks on the stem, question, and options of the input sample, and perform semantic checks using a generative large-scale language model;

[0015] Step 1.2: Based on the number of options in the input sample, copy the question stem and question several times, and concatenate them with each option to form the text sequence input to the neural network model.

[0016] Step 1.3: Format the data in the form of a text sequence and record it in a structure array. For example, mark the position of the characters in the text sequence, specifically record which characters belong to the stem, question, and option parts, and record them in their respective mark arrays.

[0017] Preferably, step 2 includes the following steps:

[0018] Step 2.1: Set the parameters of the pre-trained language model, vectorize the input text sequence, and obtain the initial embedding representation corresponding to the text sequence;

[0019] Step 2.2: Extract the embedded representations of the characters contained in the text sequence of the problem part based on the position marker array;

[0020] Step 2.3: Construct a sequence encoding neural network based on the self-attention mechanism to encode the features of the question sequence and generate a filter vector for the question stem information based on the question.

[0021] Step 2.4: Construct a sequence encoding neural network with the same structure to encode the features of the question sequence and generate a filter vector for the option information based on the question.

[0022] Step 2.5: Calculate the updated embedding by using the logical information filtering vector and the character embedding representation in the question stem or option sequence;

[0023] Step 2.6: Use a residual neural network to combine the initial embedding representation and the updated embedding representation according to the learned weights.

[0024] Preferably, step 3 includes the following steps:

[0025] Step 3.1: Divide the text sequence of the input sample into semantic units according to the rules, and record the relationships between the semantic units;

[0026] Step 3.2: Based on the relationships between semantic units, establish a logical relationship diagram that reflects the semantic units in the text sequence;

[0027] Step 3.3: Calculate the embedding representation of each semantic unit based on the embedding representation of the characters contained in the semantic unit.

[0028] Step 3.4: Calculate the overall embedded representation of the question stem and options based on the embedded representation of the characters contained in the question stem and options.

[0029] Step 3.5: Based on the logical relationship diagram of semantic units, perform information interaction between semantic units; and perform information interaction between semantic units before the question stem and options; complete the embedding and updating of semantic units.

[0030] Step 3.6: Based on the correspondence between characters and semantic units, use the question-enhanced character embedding representation obtained in Step 2 and the semantic unit embedding representation obtained in Step 3 to further update the character embedding representation in the text sequence.

[0031] Preferably, step 4 includes the following steps:

[0032] Step 4.1: Using a linear neural network, calculate the weight vector of each character in the text sequence based on the character embedding representation of the text sequence;

[0033] Step 4.2: Based on the location information of the question stem and the options recorded in the location information, extract the weights of the corresponding parts in the weight vector and perform normalization processing to obtain the weight vector extracted from the question stem information and the weight vector extracted from the options information.

[0034] Step 4.3: Based on their respective weight vectors, pool the embedding representations of the characters at the corresponding positions in the text sequence, and concatenate the pooled vectors to obtain the evaluation feature vector.

[0035] Step 4.4: Set up the random dropout layer, linear neural network layer, regularization layer, activation function layer, and linear neural network layer in sequence, integrate and evaluate the feature vector information, and give the calculation results.

[0036] Step 4.4: Based on the calculation results of the evaluation feature vectors of the four options, perform softmax normalization to obtain the final correct probability of the option.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] Addressing the common problem in current mainstream research methods of neglecting the textual information of the "question," this method proposes a question-based embedding representation enhancement approach from the perspective of human problem-solving in logical reasoning. By extracting and re-encoding the word embedding representations of the "question" sentence sequence, a filter vector based on the word embeddings of the "question" sequence is constructed. This enhances the word embedding representations of both the question stem and the answer choices, thereby achieving targeted enhancement of semantic information related to problem-solving. Simultaneously, it improves the discourse graph structure modeling method, addressing the problem of information exchange difficulties between long-distance units in traditional discourse graph structure modeling. This invention is based on deep learning technology and conducts in-depth research on logical reasoning in natural language processing. Attached Figure Description

[0039] Figure 1 This is the overall flowchart of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] The present invention will be further described below with reference to the accompanying drawings:

[0042] Example 1

[0043] The example data comes from Chinese national civil service examination questions and the English-language open domain. It includes question stem information, question information, and answer choices. The specific input example structure and content are shown below:

[0044]

[0045] like Figure 1 The diagram shows the overall flowchart of the entire method. This invention includes four main steps: data preprocessing, question-guided embedding update, discourse-level embedding update, and option probability prediction. Each main step contains several sub-steps. Preprocessing prepares the material for subsequent steps. Question-guided embedding update constructs a filtering vector based on the question text in the input samples to enhance the representation of the question stem and options. Discourse-level embedding update models the logical relationships of the text sequence at the discourse level and performs information interaction and embedding update based on this. Option probability prediction extracts the final score evaluation vector from the final embedding representation of the text sequence and normalizes it into specific probability values.

[0046] First, the source data is cleaned to ensure the input question format is complete and free of interfering characters. Then, the data is formatted to meet the input requirements of subsequent steps, and the position information of each character in the character sequence is recorded. Specifically:

[0047] 1.1.1 Format check to confirm that the data structure of the input instance is complete and there are no erroneous characters.

[0048] 1.1.2 Use a generative large-scale language model to rewrite the input text and correct semantic errors, logical errors and typos in the questions.

[0049] 1.2 Model Input Construction: The stem, question, and options are concatenated and processed into a text sequence suitable for the model's reception. For example, for an input instance with four options, the input is constructed into four input text sequences in the following format: <s> Question stem< / s> Option A for the question; <s> Question stem< / s> Option B for the question;<s> Question stem< / s> Option C is correct. <s> Question stem< / s> Option D is correct.

[0050] in, <s>、< / s> Specific characters pre-defined for the RoBERTa pre-trained language model. The maximum text length is set to 256; input sequences exceeding 256 characters will be truncated.

[0051] 1.3 The information of the input text sequence is formatted and stored in a structure. The characters in the text sequence are recorded in the dictionary according to the RoBERTa dictionary, and the positions of the characters contained in different sentence components are recorded.

[0052]

[0053] In step 2, based on the specific characteristics of logical reasoning problems, this invention designs a question-guided representation enhancement method. First, the text sequence is initialized with a vectorized representation. Second, character embedding representations of each sequence are extracted based on different sentence components. Then, a logical semantic filtering vector is obtained based on the embedding representation of the question part, thereby enhancing the embedding representation of the instance text sequence, as detailed below:

[0054] 2.1 Based on the input text sequence number and character position mask, the RoBERTa pre-trained language model is used. The RoBERTa-Large version is selected according to the hardware conditions to obtain the embedding representation of each character in the input text sequence.

[0055] 2.2 Based on the embedded representations of each character in the obtained text sequence and the question location information mask in the data information, a neural network is used to encode them to obtain a question-based representation enhancement filtering vector. The specific steps are as follows:

[0056] 2.2.1 Read the problem location code from the input data. It is a one-dimensional vector of length 256, where the value is either 1 or 0. 1 represents that the character belongs to the "problem" part.

[0057] 2.2.2 Based on the value of the problem location encoding, extract the embedded representation of the characters contained in the problem part from the character encoding of the text sequence.

[0058] 2.2.3 Embed the characters from the question section into the "Question Stem Filtering Vector Generation Module". This module contains a Transformer encoder architecture based on a self-attention mechanism. The network structure is set to two layers. This yields an intermediate representation of the question stem filtering vector.

[0059] 2.2.4 Using the intermediate representation of the option filtering vector and the information of the question stem position encoding, the intermediate representation of the filtering vector is normalized at the position of the character contained in the question stem to obtain the question stem embedding representation filtering vector with the same length as the question stem text sequence.

[0060] 2.2.5 Embed the characters from the question section into the input "Option Filter Vector Generation Module". This module contains a Transformer encoder architecture network based on a self-attention mechanism. The network structure is set to two layers. This yields an intermediate representation of the option filter vector.

[0061] 2.2.6 Using the intermediate representation of the option filtering vector, based on the information of the option position encoding, the intermediate representation of the filtering vector is normalized at the position of the character contained in the option, to obtain the option embedding representation filtering vector with the same length as the option text sequence.

[0062] 2.3 Using the obtained logical semantic filtering vector, the embedding representations of characters in the question stem and answer choices are enhanced based on the basic character embedding representations obtained during initialization. The specific steps are as follows:

[0063] 2.3.1 Based on the value of the position code of the question stem, extract the embedded representation of the characters contained in the question stem from the character code of the text sequence.

[0064] 2.3.2 The question stem embedding representation is multiplied by the embedding representation of the question stem characters to obtain the question stem embedding representation of the logical filtering part.

[0065] 2.3.3 Input the original question stem character embedding representation and the question stem embedding into the residual neural network, and set a weight value with a sum of 1 between the two to obtain the final updated question stem character embedding representation, which integrates basic semantic information and logical semantic enhancement semantic information.

[0066] 2.3.4 Based on the value of the option position encoding, extract the embedded representation of the characters contained in the option part from the text sequence character encoding.

[0067] 2.3.5 The option embedding representation is multiplied by the option character embedding representation to obtain the option embedding representation of the logical filtering part.

[0068] 2.3.6 Input the original option character embedding representation and the option embedding together into the residual neural network, and set a weight value with a sum of 1 between the two to obtain the final updated option character embedding representation.

[0069] 2.3.7 Finally, the updated embedding representation of the question stem characters, the embedding representation of the question part obtained during the initialization process, and the updated embedding representation of the option characters are concatenated to obtain the overall text sequence embedding representation after logical semantic enhancement based on the question.

[0070] During the logical semantic embedding representation enhancement process in step 2, when the updated embedding representation is obtained, this invention enters the step of embedding representation learning at the discourse structure level based on semantic association. The entire embedding representation update process is divided into three steps: text sequence structure partitioning, discourse structure modeling, and information interaction. The specific implementation of each step is as follows:

[0071] 3.1 Text Sequence Structure Decomposition: This step involves further decomposing the text sequence structure and annotating the characteristics of each structure based on the basic information obtained during data initialization. The specific steps are as follows:

[0072] 3.1.1 Read the original text data of the text sequence, mark the keywords that divide the sentence into different components according to the specified keywords, such as the keywords "if", "but" and "because" and similar preposition keywords, and mark the subject and object of the relationship according to the relationship type, record the relationship type, and obtain the position and attribute information of the relationship words.

[0073] 3.1.2 Read the punctuation marks in the text sequence, record their positions, and obtain the punctuation marks positions.

[0074] 3.1.3 Based on the positions of relational words and punctuation marks, divide the text sequence into several different semantic units. For example, in the sentence "If it doesn't rain tomorrow, we'll go hiking," "If it doesn't rain tomorrow" is one semantic unit, and "We'll go hiking" is another semantic unit. After dividing all the semantic units, record the number of semantic units and the character sequence markers contained in each semantic unit.

[0075] 3.1.4 Based on the number of semantic units, punctuation position information, and the position and attribute information of relation words, establish a relation matrix between different semantic units, where the value of the relation matrix is ​​determined by whether there is an association and the type of relation words.

[0076] 3.2 After dividing the text sequence into semantic units, a discourse structure graph based on the text sequence is constructed. During construction, the basic nodes are the semantic units obtained in the previous step. The relationships between semantic units are represented by the values ​​of the relation matrix. For character sequences with different sentence components, a long-distance information exchange mechanism is implemented to ensure that information between semantic units with the same subject can interact. The specific steps are as follows:

[0077] 3.2.1 Based on the number of semantic units, establish the embedding representation of each semantic unit one by one. Specifically, read the embedding representation of the characters contained in each semantic unit, then set the same number of learned weight values, the sum of these weights is 1, then multiply the embedding representation of the characters in the semantic unit by their respective weights, and sum them up to form the embedding representation of the semantic unit.

[0078] 3.2.2 Each semantic unit is abstracted as a node in a graph. Based on the relation matrix between different semantic units obtained in 3.1.4, the attributes of relation edges are determined according to the type of relation words, and different connection edges are set according to the type of punctuation marks, thereby obtaining the graph structure of semantic units.

[0079] 3.2.3 For the character sequences contained in different sentence components, they are further divided into three major parts in the graph structure of semantic units: stem, question, and option. Each part contains all the basic semantic units belonging to that part.

[0080] 3.2.4 For the partitions of the question stem and options, use all the basic semantic units contained therein, set several learned weight values, the sum of these weights is 1, and then multiply the embedding representation of the semantic unit with its respective weight, and sum them up to form the embedding representation of the partition of the question stem and options.

[0081] 3.3 The above steps complete the establishment of the discourse graph structure for the text sequence and represent the embedding representations of each node and partition. Furthermore, based on the modeling results of the graph structure of the text sequence, this method performs a discourse-structure-level overall update of the embedding representations of all characters in the text sequence. The specific implementation steps are as follows:

[0082] 3.3.1 Read the embedded representations of each semantic unit obtained in step 3.2.1.

[0083] 3.3.2 Based on the graph structure of the semantic units obtained in step 3.2.2, calculate the degree to which each semantic unit is influenced by other semantic units. Treat each semantic unit as a node, calculate its neighboring nodes according to the relation matrix, and use a linear neural network to calculate the intermediate values ​​of the weights based on the embedding representation of the nodes.

[0084] 3.3.3 Based on the embedded representations of the question stem and option partitions obtained in step 3.2.4, a linear neural network is used to calculate the degree of influence of each node in another partition, and the median weight of the partition influence is obtained.

[0085] 3.3.4 Based on the intermediate weights of a node affected by other nodes or partitions obtained from the above two sub-steps, use the normalization function to calculate the weight value of the node affected by other nodes or partitions.

[0086] 3.3.5 Based on the calculated weight values, use the embedding representations obtained in steps 3.2.1 and 3.2.4 to calculate the embedding update impact of each neighboring node and another partition on the current node.

[0087] 3.3.6 The embedding representation of the current node is encoded through a linear neural network layer, and then added to the embedding update vectors obtained from other nodes and partitions in step 3.3.5. An activation function is then executed to introduce a non-linear change into the embedding update process, ultimately obtaining the vector representation of the current semantic unit after embedding update based on the text structure.

[0088] After completing the three-step semantic unit embedding update based on the text sequence structure, the final step involves the model integrating the embedding vectors of different dimensions obtained in steps 2 and 3. The character embedding representations of the text sequence are pooled, and the feature dimensionality is reduced through a multi-layer neural network to finally obtain the calculated score for the text sequence. After obtaining the calculated scores for all options, a normalization function is used to calculate the correct probability of each option, and the model output is given. The specific steps are as follows:

[0089] 4.1 Based on the embedding vectors of different dimensions obtained in steps 2 and 3, obtain the overall updated character embedding representation. The sub-steps are as follows:

[0090] 4.1.1 In step 2, the model obtains the embedding of each character in the text sequence after the semantic representation of the problem is enhanced. Here, it retrieves which semantic unit each character belongs to in step 3.

[0091] 4.1.2 Set a uniform semantic unit embedding representation for updating weight values ​​during training.

[0092] 4.1.3 Add the embedding representation of the semantic unit to which the character belongs to the embedding representation of the character itself according to the weight value in step 4.1.2.

[0093] 4.1.4 Obtain a vector representation of each character in the text sequence that combines character-level embedding representation and text sequence-level embedding representation.

[0094] 4.2 Pooling is performed on the character embeddings of the text sequence, and the hidden layer features are compressed layer by layer through a multi-layer neural network to finally obtain a score reflecting the logical correctness of the option. The sub-steps are as follows:

[0095] 4.2.1 The text sequence embedding representation obtained in step 4.1.4 is processed by a linear neural network layer, and the vector representation of the characters in the sequence is initially reduced in dimensionality by matrix multiplication.

[0096] 4.2.2 Set the activation function for the dimensionality-reduced one-dimensional vector representation.

[0097] 4.2.3 The vector processed by the activation function is then input into a linear neural network to further reduce the dimensionality of the feature vector, thus obtaining the logical score of the option sequence.

[0098] 4.2.4 Following the above method, obtain the logical scores of all option sequences in sequence.

[0099] 4.2.5 Based on the logical scores of all the options obtained, a normalization function is used to express the logical scores at the computational level as the correct probabilities of each option, with the sum of their probabilities being 1.

[0100] 4.3 Based on the probabilities of each option obtained from the final calculation, the model outputs the judgment answer and gives the correct probability of each option.

[0101] This invention preprocesses data by performing anomaly handling and data formatting to ensure it meets the input requirements of subsequent steps. Then, by referencing the human approach to logical reasoning problems, it models the semantics of the questions to enhance the embedded representations of relevant characters in the text sequence, thereby enabling humans to intuitively approach problem-solving from the perspective of the questions. This ensures that the obtained embedded representations are more targeted towards logical reasoning problems. Dividing the entire text sequence into logical semantic units is a further improvement on the characteristics of logical reasoning problems, specifically addressing long-distance text dependencies and strengthening logical relationships, based on research in machine reading comprehension. The subsequent calculation of option probability scores involves pooling and compressing the embedded representations of the sequence characters, ultimately encoding the information contained in the hidden layer vectors into the final evaluation score, thus obtaining the final answer.

[0102] This invention, based on neural networks, applies deep learning techniques to represent natural language text information in multiple-choice logical reasoning questions, drawing inspiration from human problem-solving approaches. Furthermore, it enhances the representation of natural language text information in multiple-choice logical reasoning questions, addressing the limitations of previous methods that focused solely on text structure decomposition and failed to effectively model long-distance dependencies. This results in more accurate and effective results. Experiments demonstrate that this method improves the accuracy of neural network models in solving multiple-choice logical reasoning problems.

[0103] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0104] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An embedding enhancement method driven by anthropomorphic thinking for logical reasoning problems, characterized in that: Includes the following steps: Step 1: Clean the data of the sample to be solved to ensure the correctness of the input question itself. Then, according to the number of options, concatenate the question stem, question and options to complete the data formatting of the input data, and record the sentence component to which each character in the sample belongs. Step 2: Use a pre-trained language model to vectorize the input natural language text sequence to obtain the initial embedding representation of each character. Then, based on the marked position information, obtain the embedding representation of the character sequence corresponding to the question part. Using the embedding representation of the question part, use a self-attention neural network structure to encode the logical semantic filtering vector. Then, use the logical semantic filtering vector to perform logical enhancement on the embedding representation of the question stem and option parts. Step 2 includes the following steps: Step 2.1: Set the parameters of the pre-trained language model, vectorize the input text sequence, and obtain the initial embedding representation corresponding to the text sequence; Step 2.2: Extract the embedded representations of the characters contained in the text sequence of the problem part based on the position marker array; Step 2.3: Construct a sequence encoding neural network based on the self-attention mechanism to encode the features of the question sequence and generate a filter vector for the question stem information based on the question. Step 2.4: Construct a sequence encoding neural network with the same structure to encode the features of the question sequence and generate a filter vector for the option information based on the question. Step 2.5: Calculate the updated embedding by using the logical information filtering vector and the character embedding representation in the question stem or option sequence; Step 2.6: Using a residual neural network, combine the initial embedding representation and the updated embedding representation according to the learned weights; Step 3: Divide the input natural language text sequence into logical semantic units, establish a relationship graph structure between the semantic units in the input text, calculate the embedding representation of the overall semantic unit according to different weights of each character in a semantic unit, and realize information interaction between semantic units based on the graph structure of the semantic units of the input text sequence. Step 3 includes the following steps: Step 3.1: Divide the text sequence of the input sample into semantic units according to the rules, and record the relationships between the semantic units; Step 3.2: Based on the relationships between semantic units, establish a logical relationship diagram that reflects the semantic units in the text sequence; Step 3.3: Calculate the embedding representation of each semantic unit based on the embedding representation of the characters contained in the semantic unit; Step 3.4: Calculate the overall embedded representation of the question stem and options based on the embedded representation of the characters contained in the question stem and options; Step 3.5: Based on the logical relationship diagram of semantic units, perform information interaction between semantic units; and perform information interaction between semantic units before the question stem and options; complete the embedding and updating of semantic units; Step 3.6: Based on the correspondence between characters and semantic units, use the question-enhanced character embedding representation obtained in Step 2 and the semantic unit embedding representation obtained in Step 3 to further update the character embedding representation in the text sequence; Step 4: Calculate the correct probability of the option. Based on the updated text sequence embedding representation obtained in the previous step, use a fully connected neural network layer to calculate the correct probability of the option, and finally give the answer judgment for the input question sample.

2. The anthropomorphic thinking-driven embedding enhancement method for logical reasoning problems according to claim 1, characterized in that: Step 1 includes the following steps: Step 1.1: Perform grammatical checks on the stem, question, and options of the input sample, and perform semantic checks using a generative large-scale language model; Step 1.2: Based on the number of options in the input sample, copy the question stem and question several times, and concatenate them with each option to form the text sequence input to the neural network model; Step 1.3: Format the data in the form of a text sequence and record it in a structure array. Mark the position of the characters in the text sequence, specifically recording the characters belonging to the stem, question, and option parts, and recording them in their respective mark arrays.

3. The anthropomorphic thinking-driven embedding enhancement method for logical reasoning problems according to claim 1, characterized in that: Step 4 includes the following steps: Step 4.1: Using a linear neural network, calculate the weight vector of each character in the text sequence based on the character embedding representation of the text sequence; Step 4.2: Based on the location information of the question stem and the options recorded in the location information, extract the weights of the corresponding parts in the weight vector and normalize them to obtain the weight vector extracted from the question stem information and the weight vector extracted from the options information. Step 4.3: Based on their respective weight vectors, pool the embedding representations of the characters at corresponding positions in the text sequence, and concatenate the pooled vectors to obtain the evaluation feature vector; Step 4.4: Sequentially set up the random dropout layer, linear neural network layer, regularization layer, activation function layer, and linear neural network layer; integrate and evaluate the feature vector information; and give the calculation results. Step 4.4: Based on the calculation results of the evaluation feature vectors of the four options, perform softmax normalization to obtain the final correct probability of the option.

Images