Nerve symbol learning-based natural language problem programmed analysis method

Abstract

The invention discloses a neural symbol learning-based natural language problem programmed analysis method, which comprises the following steps of: constructing a problem coding module, a program analysis module, a program representation module, a problem reconstruction module and a joint training and consistency constraint module; a'problem-to-program 'analysis process and a'program-to-problem' reconstruction process are modeled at the same time under a unified framework, and the accuracy and stability of program generation are improved by using bidirectional consistency constraints. An auxiliary learning signal of a program reconstruction problem is introduced in a training process of the system and is used for constraining reasonability of program semantics, so that generation of error program structures is reduced, and a learning effect under a weak supervision or semi-supervision condition is improved. The method does not need to depend on additional manual rules, has the advantages of being clear in structure, high in interpretability, good in generalization ability and the like, and is suitable for program analysis and reasoning process modeling in neural symbol reasoning tasks such as visual reasoning and the like.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and neurosymbolic reasoning technology, specifically relating to a method for program parsing and semantic reconstruction for complex reasoning tasks, and more particularly to a method for programmatic parsing of natural language questions based on neurosymbolic learning, applicable to application scenarios such as visual question answering, multimodal reasoning, semantic parsing, program generation and interpretable reasoning. Background Technology

[0002] Multimodal reasoning tasks, especially visual question answering (VQA), have become an important benchmark for measuring the comprehensive understanding ability of machines since the standard dataset was proposed in the literature [Antol, S.; Agrawal, A.; Lu, J.; Mitchell, M.; Batra, D.; Zitnick, CL; and Parikh, D. 2015. VQA: Visualquestion answering. In Proceedings of the IEEE International Conference on Computer Vision, 2425–2433.]. Early research focused on improving accuracy using end-to-end black-box models. For example, a visual question-answering method, device, electronic device, and storage medium (WO2024164616A1) significantly improved feature representation capabilities through two-stage Deformable DETR and cross-modal contrastive learning. Systems and Methods for Visual Question Answering (US202217934671A) innovatively utilizes BLIP to generate question-oriented image descriptions, which are then answered by a pure text QA model, achieving high performance without training. However, while these high-performance methods achieved breakthroughs in accuracy, they completely abandoned interpretability, making the models difficult to verify—a "black box." In response, the academic community turned to the neuro-symbolic reasoning paradigm. The literature [Andreas, J.; Rohrbach, M.; Darrell, T.; and Klein, D. 2016b. Neural modulenetworks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 39–48.] proposed parsing the problem into an executable program structure; subsequently, the literature [Yi, K.; Wu, J.; Gan, C.; Torralba, A.; Kohli, P.; and Tenenbaum, J. 2018. Neural-symbolic VQA: Disentangling reasoning from vision and language understanding. Advances in Neural Information Processing Systems, 31.] further decoupled visual perception from symbolic reasoning, verifying the effectiveness of this paradigm in complex reasoning tasks.A confidence-based interactive neural symbol visual question answering system (CN119998799A) introduces a confidence assessment mechanism to support user intervention requests when confidence is low; a zero-shot visual question answering method and system based on Boolean prompts (CN119904712A) dynamically generates sub-questions through a Boolean self-judgment mechanism. However, these works still have fundamental limitations: they only focus on the one-way mapping from questions to procedures, lacking reverse constraints on the semantic rationality of the procedures. Although the confidence-based interactive neural symbol visual question answering system attempts to introduce reconstruction loss to assess procedure quality, it is only used as an offline screening tool, rather than being jointly optimized end-to-end with the main model, and therefore cannot fundamentally guarantee the semantic consistency of the generated procedures.

[0003] In recent years, architectural innovation has become a new focus. The literature [Zhao, Z.; Samel, K.; Chen, B.; et al.2021. Proto: Programguided Transformer for program-guided tasks. Advances in Neural Information Processing Systems, 34: 17021–17036.] integrates program structure information into the Transformer encoding process; a visual question answering optimization method and system based on hierarchical multimodal fine-tuning (CN120011547A) proposes multi-level CLIP features and a phased LoRA fine-tuning strategy to achieve fine-tuned visual-text fusion. However, these methods still belong to a unidirectional modeling paradigm and do not solve the problem of semantic drift in programs. It is worth noting that existing neural symbolic methods are prone to generating semantically inconsistent program structures in the early stages of training or under weak supervision, because they lack a closed-loop verification mechanism to verify whether the program truly reflects the intent of the question. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, the present invention aims to provide a method for procedural parsing of natural language questions based on neural symbolic learning. By simultaneously constructing a problem-to-program parsing model and a program-to-problem reconstruction model within a unified framework, and introducing a bidirectional consistency constraint mechanism, the two form a closed-loop feedback during training, suppressing the generation of pseudo-program questions and improving the procedural parsing effect of natural language questions.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: The present invention provides a procedural parsing method for natural language problems based on neural symbol learning, comprising the following steps: A procedural parsing method for natural language questions based on neural symbol learning includes the following steps: Step 1: Construct a problem-program bidirectional modeling framework, which includes at least: The program parsing module is used to map the input natural language question into a structured executable program; The problem reconstruction module is used to reverse map the program generated by the program parsing module into a natural language problem. Step 2: Segment and vectorize the natural language questions to obtain a question sequence; symbolize and serialize the program to obtain a program sequence; wherein the program sequence is a sequence of symbolic operators to be executed in a visual or structured knowledge space; Step 3: Input the problem sequence into the program parsing module to generate the corresponding prediction program sequence; Step 4: Input the prediction program sequence into the problem reconstruction module to generate the corresponding reconstructed problem sequence; Step 5: Simultaneously construct the program parsing loss function and the problem reconstruction loss function, and train the framework end-to-end through a joint optimization strategy to make the reconstruction problem as close as possible to the original input problem in terms of semantics and syntax; Step 6: After training is complete, use only the program parsing module to reason about the input problem, output a structured executable program, and hand it over to the subsequent symbolic execution engine to complete the inference calculation.

[0006] In one embodiment, the program parsing module is divided into two consecutive sub-processes: the problem semantic encoding stage and the program sequence generation stage. In the semantic encoding stage, the input natural language question is first formed into a vector representation through embedding mapping and positional encoding mapping. The vector representations of all symbols constitute the input vector sequence and are fed into a multi-layer stacked sequence modeling network structure for context modeling processing, thereby obtaining the context semantic representation sequence of the question. Each element of the sequence is the semantic representation of the corresponding symbol in the input natural language question under the global semantic context. In the program sequence generation stage, the program parsing module generates the target program sequence step by step in an autoregressive manner. Through conditional probability modeling, the program generation process is modeled as a sequence conditional generation process under the constraints of the semantic representation of the problem. The mapping relationship between natural language problems and programs is learned by maximizing the conditional probability of the complete program sequence. During the model training phase, the program parsing module uses the real program sequences provided in the training dataset as supervision signals and updates the parameters by minimizing the sequence-level difference between the model's predicted program sequence and the target program sequence. During the reasoning phase, the program parsing module relies solely on the input question and its encoded representation to automatically generate the corresponding program sequence through a step-by-step decoding method. This program sequence serves as the input to the question reconstruction module and as the intermediate representation for the symbolic reasoning execution module.

[0007] In one embodiment, the problem reconstruction module is divided into two consecutive sub-processes: the program symbol encoding stage and the problem sequence generation stage. In the program symbol encoding stage, the program sequence is converted into a vector sequence representation through embedding mapping and positional encoding mapping, and then fed into the sequence modeling network for contextual semantic modeling, thereby obtaining the contextual representation sequence of the program; In the problem sequence reconstruction stage, the target problem sequence is generated step by step in an autoregressive manner. Through conditional probability modeling, the reverse generation process from program to natural language problem is modeled as a sequence condition generation problem under the constraints of program semantic representation. The training objective of the problem reconstruction module is to make the generated reconstruction problem as close as possible to the original problem in terms of semantics and structure, given a sequence of input programs. Under the joint training framework, the parameters of the problem reconstruction module and the parameters of the program parsing module jointly participate in the optimization process, enabling them to gradually form a stable and consistent mapping relationship in the shared semantic representation space.

[0008] In one embodiment, the problem reconstruction module employs an encoder-decoder structure, first processing the input program sequence... Encode to obtain the program context semantic representation. Then by the decoder Under the constraints of the conditions, a sequence of reconstruction problems is generated step by step. ;or, The problem reconstruction module adopts a decoder-only structure, directly based on the vector representation of the program sequence, to construct an autoregressive generation process from the program to the problem.

[0009] In one embodiment, the natural language question is segmented and vectorized to obtain a question sequence, and the program is symbolized and serialized to obtain a program sequence. The implementation method is as follows: The natural language problem is represented as a sequence in symbolic form: in, Indicates the sequence length. Indicates the first in the problem A discrete symbol unit, wherein the symbol unit includes natural language words and basic symbols that have distinguishing meaning in semantic analysis; The program is represented as a sequence of symbolic operators: in, Indicates the first in the program There are 10 operator nodes, and each operator node can reference the execution results of any one or more previous operators, thus semantically forming an implicit directed acyclic computation graph structure. Indicates the total length of the program sequence. This represents a symbolic unit in the program, which can be an operator name, parameter symbol, index symbol, or structure control symbol; each operator node Formalized as: in, Indicates the operator node name, This indicates the index position of the result of the preceding operator referenced by this operator node. This indicates the first node of the operator. There are 10 parameters, which are used to describe the attribute information of the object.

[0010] In one embodiment, a unified symbol vocabulary is constructed using a byte-pair encoding method based on sub-word units. This converts the problem sequence and program sequence into discrete symbol sequence representations in a shared symbol space and maps them to a unified index sequence form. The implementation method is as follows: First, an initial set of symbols is constructed, which contains the following basic symbol units: Characters and word fragments appearing in natural language problems; Operator names, parameter names, and attribute names appearing in the program; The index symbols, bracket symbols, delimiters, and end markers used in the program structure; Numerical symbols and numerical symbols that represent semantics; Secondly, the frequency of symbol fragments in all problem sequences and program sequences in the training dataset is statistically analyzed, and a word merging operation based on frequency statistics is performed on the entire training corpus, using character-level or smallest symbol units as the initial vocabulary. The word merging operation follows the BPE encoding strategy, that is, in each iteration, the adjacent symbol pairs with the highest frequency in the current corpus are selected and merged into new word units. This process is repeated until the preset vocabulary size limit is reached, forming a unified word vocabulary that simultaneously covers natural language problem and program symbols. Finally, the vocabulary is solidified into a unified symbol-encoded dictionary, and a unique integer index is assigned to each sub-word unit.

[0011] In one embodiment, the bidirectional modeling framework combines mapping relationships: Problem → Program Program → Problem and constraints An approximation identity mapping is used to achieve a semantic consistency closed-loop constraint, which is incorporated into the model training objective function system.

[0012] In one embodiment, when the training samples simultaneously contain natural language questions and its corresponding annotation program At the same time, the following two types of training objectives are constructed: First, the forward program parsing supervision target is used to constrain the program parsing module to satisfy: This objective is achieved by minimizing the difference between the model's predicted program sequence and the actual program sequence, and the corresponding loss function is expressed as: Secondly, the reverse consistency reconstruction objective is used to constrain: This objective is achieved by minimizing the difference between the problem generated by the model based on the real program and the original problem. The corresponding loss function is expressed as: Introduce a semantic consistency closed-loop constraint based on the model prediction procedure, which requires: The loss function corresponding to the semantic consistency closed-loop constraint is expressed as follows: By combining the above three types of objective functions, the following joint optimization objective function is constructed for scenarios with labeled data: in, and These are hyperparameters used to balance the weights of different training objectives.

[0013] In one embodiment, when only natural language questions are provided in the training samples... Without providing a corresponding annotation program, the prediction program is first generated using the program parsing module under the current parameter state: Reuse the problem refactoring module to generate refactoring problems: And by minimizing and The model parameters are updated based on the differences between them, and the corresponding training objective function is: .

[0014] In one embodiment, under a semi-supervised training mode: For data that contains only questions but no program annotations, a pseudo-program is first generated using a pre-trained program parsing module; then, the pseudo-program is reconstructed and trained using a question reconstruction module, thereby improving model performance without relying on manual program annotations.

[0015] In one embodiment, under a semi-supervised training mode: Data samples containing annotation procedures and data samples containing only question text are included in the same training framework, and different methods are used for each. and By jointly optimizing the model parameters, unified modeling and collaborative utilization of multi-source heterogeneous data can be achieved.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: 1. Introducing the reverse reconstruction constraint of "program → problem" enables the model training process to form a semantic closed loop, effectively suppressing the problem of generating pseudo-programs that are semantically incorrect but formally correct.

[0017] 2. In the early stages of model training, the problem reconstruction error can amplify the feedback of the program parsing error, thereby significantly accelerating model convergence and improving training stability.

[0018] 3. The method is naturally applicable to semi-supervised learning scenarios and can make full use of a large amount of unlabeled problem data, significantly reducing data labeling costs.

[0019] 4. The proposed bidirectional modeling framework does not depend on the specific form of downstream tasks, has good versatility, and can be widely applied to scenarios such as visual reasoning, multimodal reasoning, semantic parsing, and complex logical reasoning. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the method of the present invention.

[0021] Figure 2 This is a model architecture diagram of the method of the present invention.

[0022] Figure 3 This is a schematic diagram of the program parsing module in this invention. Detailed Implementation

[0023] The technical solution of the procedural parsing method for natural language problems based on neural symbol learning proposed in this invention will be further described in detail below with reference to specific embodiments.

[0024] This invention presents a natural language problem-based procedural parsing method based on neural symbolic learning, aiming to address the problems of distorted results and low inference efficiency in existing methods, which suffer from unidirectional problem-to-program mapping, lack of semantic consistency constraints, susceptibility to parsing errors, and unstable training. By constructing a problem-program joint representation and symbolic modeling module, a program parsing module, a BPE-based unified symbolic encoding module for problems and programs, a problem reconstruction module, and a bidirectional consistency joint training and closed-loop constraint module, the system simultaneously models the "problem-to-program" parsing process and the "program-to-problem" reconstruction process within a unified framework. Bidirectional consistency constraints are used to improve the accuracy and stability of program generation. Simultaneously, the system introduces auxiliary learning signals for program reconstruction during training to constrain the rationality of program semantics, thereby reducing the generation of erroneous program structures and improving learning performance under weakly supervised or semi-supervised conditions.

[0025] In summary, this invention constructs a problem-program bidirectional modeling framework and utilizes this framework to implement the parsing process, while also covering the framework training process. The main flow of the framework includes: Step 1: Preprocessing and word segmentation encoding for natural language problems.

[0026] Step 2: Based on the encoding results, model the semantic features of the problem.

[0027] Step 3: Perform structured analysis from problem to program.

[0028] Step 4: Model the semantic representation of the program.

[0029] Step 5: Perform semantic reconstruction from program to problem.

[0030] Step 6: Parse-reconstruct joint constraint training; Step 7: Output the structured reasoning program.

[0031] This invention does not rely on additional manually generated rules and has advantages such as clear structure, strong interpretability, and good generalization ability. It is suitable for program parsing and reasoning process modeling in neural symbolic reasoning tasks such as visual reasoning. The system as a whole mainly consists of the following five functional modules: Module 1: Problem-Program Joint Representation and Symbolic Modeling Module; Module 2: Unified BPE Symbol Encoding Module; Module 3: Program Parsing Module; Module 4: Problem Refactoring Module; Module 5: Bidirectional Consistency Joint Training and Closed-Loop Constraint Module.

[0032] The modules work collaboratively according to a closed-loop logic of "problem → program → problem," thereby forming bidirectional semantic constraints during the training phase and outputting a structured executable program during the inference phase. The following five parts describe each module in detail.

[0033] Part 1: Problem-Program Joint Representation and Symbolic Modeling Module.

[0034] In this invention, to achieve bidirectional mapping modeling between natural language problems and executable programs, a unified formal definition and sequential modeling of the problem space, program space, and intermediate symbol structure are first performed. This provides a rigorous mathematical object and a unified data representation foundation for the subsequent end-to-end learning of neural network models. The model architecture is referenced... Figure 2 .

[0035] In this embodiment, the natural language problem is represented as a sequence of symbols: in, Indicates the length of the problem sequence. Indicates the first in the problem Each discrete symbol unit includes not only natural language words, but also basic symbols that have distinguishing meaning in semantic parsing, such as numbers, attribute words, relational words, and punctuation marks.

[0036] On the other hand, the "program" in this invention is not code text in the traditional sense, but rather a sequence of symbolic operators to be executed in a visual or structured knowledge space. Each operator... Formalized as: in, Indicates the operator name, This indicates the index position of the result of the preceding operator referenced by this operator. This indicates the first... Each parameter is used to describe the object's attribute information, including but not limited to semantic attributes such as category, color, material, size, and shape.

[0037] A complete program sequence is defined as a sequence of operators: in, Indicates the first in the program Each operator node can reference the execution result of any one or more previous operators, thus semantically forming an implicit directed acyclic computation graph structure.

[0038] In order to adapt the above program structure to the sequence-to-sequence neural network modeling framework, each operator in this invention is further... Expand into the following unified sequence form: The entire program is further flattened into a symbolic sequence representation: in, Indicates the total length of the program sequence. This represents a symbolic unit in the program, which can be an operator name, parameter symbol, index symbol, or structure control symbol.

[0039] In this way, programs that originally had nested and graph-structured semantics are uniformly converted into linear symbolic sequences, thus enabling them to be processed at the representation level using a unified sequence modeling framework, similar to natural language problems.

[0040] Within the overall modeling framework of this invention, the following three spaces are defined: Problem Space : Represents the set of all legal natural language question sequences; Program space : Represents the set of all legal executable program sequences; Reconstructing the problem space : Represents a set of problem sequences generated by the program in reverse.

[0041] And further define two mapping functions: Where, mapping This represents the forward parsing process from a natural language problem to a program, reflecting... This represents the reverse reconstruction process from a program to a natural language problem. The core objective of this invention is to ensure that the following composite mapping relationship holds: .in, Representing the problem space The identity mapping on the problem, that is, through a two-way mapping loop of "problem → program → problem", makes the reconstructed problem... The natural language problem of approximating the original input as closely as possible at the semantic and structural levels. .

[0042] By using the unified formal modeling of problems, programs, and their mapping relationships, this invention constructs a strict correspondence structure between the natural language semantic space and the executable program space at the symbolic level, providing a clear, stable, and computable representation foundation for subsequent neural network-based program parsing and problem reconstruction modules.

[0043] Part Two: Problem and Program Unified Symbol Encoding Module Based on BPE In this invention, to achieve unified modeling of natural language problem sequences and program symbol sequences within the same neural network framework, it is necessary to first convert them into discrete symbol sequence representations in a shared symbol space, and then further map them into a unified index sequence form. To this end, this invention employs a Byte Pair Encoding (BPE) method based on sub-word units to construct a unified symbol vocabulary, and accordingly performs consistent symbolic encoding processing on problems and programs.

[0044] In this embodiment, an initial symbol set is first constructed, which includes the following basic symbol units: Characters and word fragments appearing in natural language problems; Operator names, parameter names, and attribute names appearing in the program; The index symbols, bracket symbols, delimiters, and end markers used in the program structure; Numerical symbols and numerical symbols that represent quantity, numbering, and other semantic meanings.

[0045] Based on this, the frequency of symbol fragments in all question sequences and program sequences in the training dataset is statistically analyzed. Using character-level or smallest symbol units as the initial vocabulary, a word merging operation based on frequency statistics is performed on the entire training corpus. The word merging operation follows the BPE encoding strategy, that is, in each iteration, the most frequent adjacent symbol pairs in the current corpus are selected and merged into new word units. This process is repeated until the preset vocabulary size limit is reached.

[0046] Through the above iterative merging process, a unified sub-vocabulary that simultaneously covers natural language problems and program symbols is gradually formed, so that high-frequency semantic units (such as common attribute words, operator name fragments and structural symbol combinations) can appear in the form of a single sub-vocabulary unit, while low-frequency or highly combinable symbols are still represented in the form of a combination of multiple sub-vocabulary units, thereby achieving a balance between vocabulary size and representational power.

[0047] After constructing the sub-word vocabulary, this vocabulary is solidified into a unified symbol-encoded dictionary, and a unique integer index is assigned to each sub-word unit. Subsequently, for any natural language problem... And any program Each word is decomposed into a sequence of subwords composed of elements from the subword vocabulary, and further mapped to the corresponding integer index sequence representation.

[0048] Through the aforementioned unified BPE symbol encoding module, this invention achieves consistent discretization modeling of natural language question sequences and program sequences, enabling them to be input into the subsequent program parsing module and question reconstruction module in a unified index sequence form for end-to-end joint training and inference processing.

[0049] Part Three: Program Parsing Module refer to Figure 3 In this invention, the program parsing module is used to implement parsing from the natural language problem space. To the executable program space The mapping process, in its function within the overall system, corresponds to the following mathematical mapping relationship: in, This indicates the program's parsing mapping function. This represents the set of trainable parameters corresponding to the mapping function.

[0050] Program parsing module Structurally, it adopts a sequence-to-sequence modeling structure in the form of an encoder-decoder, and its overall function is divided into two continuous sub-processes: the problem semantic encoding stage and the program sequence generation stage.

[0051] In the semantic encoding stage, the input question sequence is first represented as a vector sequence through embedding mapping and positional encoding mapping: in, Symbols The corresponding content embedding vector, This represents the encoded vector indicating the symbol's position within the sequence. The vector representations of all symbols constitute the input vector sequence: This vector sequence is then fed into a multi-layered stacked sequence modeling network structure for context modeling processing, thereby obtaining a sequence of contextual semantic representations of the problem: Each of them Indicates the first input question The semantic representation of a symbol under the global semantic context. The representation not only encodes the local semantic information of the symbol itself, but also integrates the contextual constraint information generated by other symbols in the problem.

[0052] During the program sequence generation phase, the program parsing module generates the target program step by step using an autoregressive approach. At any generation time The model's prediction probability for the current symbol is defined as: in, This represents the sequence of program prefixes generated before the current time. This represents the contextual semantic representation from the problem encoding stage.

[0053] Using the aforementioned conditional probability modeling method, the program parsing module models the program generation process as a sequence conditional generation process under the constraints of the problem's semantic representation, and learns the mapping relationship between natural language problems and symbolic programs by maximizing the conditional probability of the complete program sequence.

[0054] During the model training phase, the program parsing module uses real program sequences provided in the training dataset. As a supervisory signal, parameters are updated by minimizing the sequence-level difference between the model's predicted program sequence and the target program sequence. The training objective can be formally expressed as maximizing the following log-likelihood function: in, This represents the true symbol of the target program sequence at time t.

[0055] During the inference phase, the program parsing module depends only on the input question. and its encoding representation The corresponding program symbol sequence is automatically generated through a step-by-step decoding method. This program sequence will be used as a module for subsequent problem refactoring. The input, and the intermediate representation as the symbolic reasoning execution module.

[0056] In the bidirectional modeling framework of this invention, the program parsing module It not only undertakes the mapping function from problem to program, but also serves as a forward path component in constructing the semantic closed-loop consistency constraint structure of "problem → program → problem". The quality of its output program representation will directly affect the stability of the subsequent problem reconstruction process and the effectiveness of semantic consistency constraints.

[0057] Part Four: Problem Refactoring Module In this invention, the problem refactoring module is used to implement the refactoring from program space. To reconstruct the problem space The reverse mapping process, in its function within the overall system, corresponds to the following mathematical mapping relationship: in, Represents the problem reconstruction mapping function, This represents the set of trainable parameters corresponding to the mapping function. Represents the input sequence of program symbols. This represents the sequence of reconstruction problems output.

[0058] In the bidirectional modeling framework of this invention, the problem reconstruction module is not used to independently generate natural language problems, but rather in conjunction with the program parsing module. Together, they form a semantic closed-loop structure of "problem → procedure → problem," which is used for composite mapping. Approximation of identity mappings on the problem space ,Right now: This closed-loop constraint mechanism ensures that the program parsing module not only meets the structural rationality requirements of the program space when generating the program structure, but also maintains a reversible consistency relationship with the semantics of the original problem, thereby significantly reducing the probability of semantic drift and structural misparsing.

[0059] Similar to the input processing method of the program parsing module, the program symbol sequence is first converted into a vector sequence representation through embedding mapping and positional encoding mapping, and then fed into the sequence modeling network for contextual semantic modeling, thereby obtaining the contextual representation sequence of the program: Each of them Indicates the first in the program The semantic representation of a symbol within the context of the overall program structure.

[0060] During the problem sequence generation phase, the problem reconstruction module generates the target problem symbol sequence step by step in an autoregressive manner: in, Indicates the length of the sequence of reconstruction problems. Indicates the first Problem symbols generated at each time step.

[0061] At any generation time The model's prediction probability for the symbol in the current problem is defined as: in, This represents the sequence of prefixes for reconstruction problems generated before the current time. This represents the contextual semantics from the program coding phase.

[0062] Using the above-mentioned conditional probability modeling method, the problem reconstruction module will model the reverse generation process from program to natural language problem as a sequence condition generation problem under the constraints of program semantic representation.

[0063] In one specific embodiment of the present invention, the problem reconstruction module can be implemented using one of two different structural forms, depending on system resource constraints and application requirements, as shown in the reference. Figure 2 : In the first implementation, the problem reconstruction module adopts a complete encoder-decoder architecture. This architecture first processes the input program sequence... Encode to obtain the program context semantic representation. Then by the decoder Under the constraints of the conditions, a sequence of reconstruction problems is generated step by step. This implementation offers greater flexibility in terms of expressive power and is suitable for application scenarios with high requirements for reconstruction quality.

[0064] In the second implementation, the problem reconstruction module adopts a simplified decoder-only structure. In this structure, the system no longer explicitly constructs a separate program encoder, but instead directly builds an autoregressive generation process from the program to the problem based on the vector representation of the program symbol embedding sequence. This implementation effectively reduces the model parameter size and computational complexity while maintaining basic reconstruction capabilities, making it suitable for applications with high inference efficiency requirements.

[0065] Regardless of the structural form used, the training objective of the problem reconstruction module is the same: given the input program... Under certain conditions, the reconstruction problem of model generation is made possible. To approximate the original problem as closely as possible at the semantic and structural levels. Its training objective function can be formally expressed as maximizing the following conditional log-likelihood: in, Represents the first in the original problem sequence A symbol.

[0066] Within the joint training framework, the problem reconstruction module parameters With program parsing module parameters By jointly participating in the optimization process, the two gradually form a stable and consistent mapping relationship in the shared semantic representation space.

[0067] Through the above structural design, the problem reconstruction module in this invention not only undertakes the function of reverse mapping from program to natural language problem, but more importantly, it serves as a key component module for constructing bidirectional semantic consistency constraints, and together with the program parsing module, it forms a complete closed-loop structure for bidirectional joint modeling of neural symbols.

[0068] Part 5: Bidirectional Consistency Joint Training and Closed-Loop Constraint Module In this invention, to overcome the problems of semantic drift, structural instability, and high dependence on labeled data caused by relying solely on unidirectional "problem → program" mapping training, a bidirectional consistency joint training and closed-loop constraint mechanism is introduced, integrating the program parsing module... With the problem refactoring module By integrating them into the same optimization framework for collaborative training, a stable, reversible, and consistent semantic mapping relationship can be constructed between the problem space and the program space.

[0069] As described in Part I, the following two mapping functions are defined in this invention: in, Corresponding program parsing module , Corresponding problem reconstruction module The core objective of this invention is no longer merely to make It is not enough to map a problem into a structurally correct program; it is further required that the following composite mapping relationship holds in the overall semantic space: in, Representing the problem space The identity mapping on [the graph]. This constraint implies that for any input problem... The program is generated via the program parsing module. Then, the problem is generated in reverse through the problem refactoring module. The resulting reconstruction problem The semantic and structural aspects should be as close as possible to the original input problem. .

[0070] In this invention, the aforementioned consistency constraints are not used as ex-post evaluation indicators, but are directly incorporated into the model training objective function system, forming a joint optimization objective together with the traditional program analysis supervision objective.

[0071] In one specific implementation, when the training samples simultaneously contain natural language questions and its corresponding annotation program At the same time, the system constructs the following two types of training objectives: First, the forward program parsing supervision target is used to constrain the program parsing module to satisfy: This objective is achieved by minimizing the difference between the model's predicted program sequence and the actual program sequence, and the corresponding loss function can be expressed as: Secondly, the reverse consistency reconstruction objective is used to constrain: This objective is achieved by minimizing the difference between the problem generated by the model based on the real program and the original problem. The corresponding loss function can be expressed as: Building upon this, a closed-loop consistency constraint term based on the model prediction procedure is further introduced, which requires: The loss function corresponding to this closed-loop consistency constraint can be expressed as: By combining the above three types of objective functions, this invention constructs the following joint optimization objective function in the case of labeled data: in, and These are hyperparameters used to balance the weights of different training objectives.

[0072] Through the aforementioned joint training mechanism, the program parsing module is not only constrained to generate a structure consistent with the annotation program, but also implicitly constrained to ensure that its output program retains sufficient information to support the reverse reconstruction of the original problem semantics, thereby significantly reducing the loss of semantic information and structural ambiguity at the representation level.

[0073] In a further preferred embodiment of the invention, when only natural language questions are provided in the training samples... Even without providing a corresponding annotation program, the system can still perform weakly supervised or self-supervised training using the closed-loop consistency constraint mechanism. In this case, the system first generates a prediction program using the program parsing module under the current parameter state: Reuse the problem refactoring module to generate refactoring problems: And by minimizing and The model parameters are updated based on the differences between them, and the corresponding training objective function is: In this way, even in the absence of labeled training samples, the system can still continuously optimize model parameters using a large amount of data containing only problem text, thereby significantly reducing its dependence on high-cost labeled data.

[0074] In a semi-supervised training scenario, this invention integrates data samples containing annotation procedures and data samples containing only question text into the same training framework, and uses the aforementioned supervised objective function and unsupervised closed-loop consistency objective function to jointly optimize the model parameters, thereby achieving unified modeling and collaborative utilization of multi-source heterogeneous data.

[0075] Throughout the training process, the program parsing module With the problem refactoring module The parameters are not updated independently, but are optimized collaboratively under the constraints of a shared joint loss function, so that the two gradually form a stable bidirectional consistent mapping relationship in the shared semantic representation space.

[0076] Through the aforementioned bidirectional consistency joint training and closed-loop constraint mechanism, this invention fundamentally changes the training paradigm of traditional program parsing methods that "only focuses on the accuracy of one-way mapping," enabling the learned program representation to be not only executable at the structural level but also highly consistent with the original problem at the semantic level. This significantly improves the system's stability, generalization ability, and robustness in complex combinatorial reasoning problems.

Claims

1. A procedural parsing method for natural language problems based on neural symbol learning, characterized in that, Includes the following steps: Step 1: Construct a problem-program bidirectional modeling framework, which includes at least the following: The program parsing module is used to map the input natural language question into a structured executable program; The problem reconstruction module is used to reverse map the program generated by the program parsing module into a natural language problem. Step 2: Segment and vectorize the natural language questions to obtain a question sequence; symbolize and serialize the program to obtain a program sequence; wherein the program sequence is a sequence of symbolic operators to be executed in a visual or structured knowledge space; Step 3: Input the problem sequence into the program parsing module to generate the corresponding prediction program sequence; Step 4: Input the prediction program sequence into the problem reconstruction module to generate the corresponding reconstructed problem sequence; Step 5: Simultaneously construct the program parsing loss function and the problem reconstruction loss function, and train the framework end-to-end through a joint optimization strategy to make the reconstruction problem as close as possible to the original input problem in terms of semantics and syntax. Step 6: After training is complete, use only the program parsing module to reason about the input problem, output a structured executable program, and hand it over to the subsequent symbolic execution engine to complete the inference calculation.

2. The method for procedural parsing of natural language problems based on neural symbol learning according to claim 1, characterized in that, The program parsing module is divided into two continuous sub-processes: the problem semantic encoding stage and the program sequence generation stage. In the semantic encoding stage, the input natural language question is first formed into a vector representation through embedding mapping and positional encoding mapping. The vector representations of all symbols constitute the input vector sequence and are fed into a multi-layer stacked sequence modeling network structure for context modeling processing, thereby obtaining the context semantic representation sequence of the question. Each element of the sequence is the semantic representation of the corresponding symbol in the input natural language question under the global semantic context. In the program sequence generation stage, the program parsing module generates the target program sequence step by step in an autoregressive manner. Through conditional probability modeling, the program generation process is modeled as a sequence conditional generation process under the constraints of the semantic representation of the problem. The mapping relationship between natural language problems and programs is learned by maximizing the conditional probability of the complete program sequence. During the model training phase, the program parsing module uses the real program sequences provided in the training dataset as supervision signals and updates the parameters by minimizing the sequence-level difference between the model's predicted program sequence and the target program sequence. During the reasoning phase, the program parsing module relies solely on the input question and its encoded representation to automatically generate the corresponding program sequence through a step-by-step decoding method. This program sequence serves as the input to the question reconstruction module and as the intermediate representation for the symbolic reasoning execution module.

3. The method for procedural parsing of natural language problems based on neural symbol learning according to claim 1, characterized in that, The problem reconstruction module is divided into two continuous sub-processes: the program symbol encoding stage and the problem sequence generation stage. In the program symbol encoding stage, the program sequence is converted into a vector sequence representation through embedding mapping and positional encoding mapping, and then fed into the sequence modeling network for context semantic modeling, thereby obtaining the context representation sequence of the program; In the problem sequence reconstruction stage, the target problem sequence is generated step by step in an autoregressive manner. Through conditional probability modeling, the reverse generation process from program to natural language problem is modeled as a sequence condition generation problem under the constraints of program semantic representation. The training objective of the problem reconstruction module is to make the generated reconstruction problem as close as possible to the original problem in terms of semantics and structure, given a sequence of input programs. Under the joint training framework, the parameters of the problem reconstruction module and the parameters of the program parsing module jointly participate in the optimization process, enabling them to gradually form a stable and consistent mapping relationship in the shared semantic representation space.

4. The method for procedural parsing of natural language problems based on neural symbol learning according to claim 1, characterized in that, The problem reconstruction module adopts an encoder-decoder structure, first processing the input program sequence... Encode to obtain the program context semantic representation. Then by the decoder Under the constraints of the conditions, a sequence of reconstruction problems is generated step by step. ; or, The problem reconstruction module adopts a decoder-only structure, directly based on the vector representation of the program sequence, to construct an autoregressive generation process from the program to the problem.

5. The method for procedural parsing of natural language problems based on neural symbol learning according to claim 1, characterized in that, The natural language problem is segmented and vectorized to obtain a problem sequence, and the program is symbolized and serialized to obtain a program sequence. The implementation method is as follows: The natural language problem is represented as a sequence in symbolic form: in, Indicates the sequence length. Indicates the first in the problem A discrete symbol unit, wherein the symbol unit includes natural language words and basic symbols that have distinguishing meaning in semantic analysis; The program is represented as a sequence of symbolic operators: in, Indicates the first in the program There are 10 operator nodes, and each operator node can reference the execution results of any one or more previous operators, thus semantically forming an implicit directed acyclic computation graph structure. Indicates the total length of the program sequence. This represents a symbolic unit in the program, which can be an operator name, parameter symbol, index symbol, or structure control symbol; each operator node Formalized as: in, Indicates the operator node name, This indicates the index position of the result of the preceding operator referenced by this operator node. This indicates the first node of the operator. There are 10 parameters, which are used to describe the attribute information of the object.

6. The method for procedural parsing of natural language problems based on neural symbol learning according to claim 1, characterized in that, A unified symbol vocabulary is constructed using a byte-pair encoding method based on sub-word units. This method converts problem sequences and program sequences into discrete symbol sequence representations in a shared symbol space and maps them to a unified index sequence format. The implementation method is as follows: First, an initial set of symbols is constructed, which contains the following basic symbol units: Characters and word fragments appearing in natural language problems; Operator names, parameter names, and attribute names appearing in the program; The index symbols, bracket symbols, delimiters, and end markers used in the program structure; Numerical symbols and numerical symbols that represent semantics; Secondly, the frequency of symbol fragments in all problem sequences and program sequences in the training dataset is statistically analyzed, and a word merging operation based on frequency statistics is performed on the entire training corpus, using character-level or smallest symbol units as the initial vocabulary. The word merging operation follows the BPE encoding strategy, that is, in each iteration, the adjacent symbol pairs with the highest frequency in the current corpus are selected and merged into new word units. This process is repeated until the preset vocabulary size limit is reached, forming a unified word vocabulary that simultaneously covers natural language problem and program symbols. Finally, the vocabulary is solidified into a unified symbol-encoded dictionary, and a unique integer index is assigned to each sub-word unit.

7. The method for procedural parsing of natural language problems based on neural symbol learning according to any one of claims 1 to 6, characterized in that, The bidirectional modeling framework combines mapping relationships: Problem → Program Program → Problem and constraints An approximation identity mapping is used to achieve a semantic consistency closed-loop constraint, which is incorporated into the model training objective function system.

8. The method for procedural parsing of natural language problems based on neural symbol learning according to claim 7, characterized in that, When the training samples simultaneously contain natural language questions and its corresponding annotation program At the same time, the following two types of training objectives are constructed: First, the forward program parsing supervision target is used to constrain the program parsing module to satisfy: This objective is achieved by minimizing the difference between the model's predicted program sequence and the actual program sequence, and the corresponding loss function is expressed as: Secondly, the reverse consistency reconstruction objective is used to constrain: This objective is achieved by minimizing the difference between the problem generated by the model based on the real program and the original problem. The corresponding loss function is expressed as: Introduce a semantic consistency closed-loop constraint based on the model prediction procedure, which requires: The loss function corresponding to the semantic consistency closed-loop constraint is expressed as follows: By combining the above three types of objective functions, the following joint optimization objective function is constructed for scenarios with labeled data: in, and These are hyperparameters used to balance the weights of different training objectives.

9. The method for procedural parsing of natural language problems based on neural symbol learning according to claim 7, characterized in that, When the training samples only provide natural language questions Without providing a corresponding annotation program, the prediction program is first generated using the program parsing module under the current parameter state: Reuse the problem refactoring module to generate refactoring problems: And by minimizing and The model parameters are updated based on the differences between them, and the corresponding training objective function is: 。 10. The procedural parsing method for natural language problems based on neural symbol learning according to claim 9, characterized in that, In a semi-supervised training mode: For data that contains only questions but no program annotations, a pseudo-program is first generated using a pre-trained program parsing module; then, the pseudo-program is reconstructed and trained using a question reconstruction module, thereby improving model performance without relying on manual program annotations. In a semi-supervised training mode: Data samples containing annotation procedures and data samples containing only question text are included in the same training framework, and different methods are used for each. and By jointly optimizing the model parameters, unified modeling and collaborative utilization of multi-source heterogeneous data can be achieved.

Images