Germinating type pre-training method and system for logic structured data modeling

Abstract

The invention discloses a logic structured data modeling-oriented germination type pre-training method and system, and belongs to the technical field of information. The method comprises the following steps: performing data logic analysis on original data to obtain a structured representation of the original data; performing data germination based on the structured representation to obtain logic structured data x, and obtaining a plurality of data sample pairs {xi-a, xi-b} according to a growth path of the logic structured data x in the data germination process; and completing training of a target model for {xi-a, xi-b} based on the data sample to obtain a germination type pre-training model. According to the method, a logic structure possibly contained in each piece of data is fully mined, and a proper germination mode is applied under the condition that different contexts are given to accurately generate target data contents, so that downstream tasks can be processed more effectively.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to an germination pre-training method and system for logically structured data modeling. Background Art

[0002] Logically structured data refers to data forms with clear logical rules or semantic structures, such as imperative programming language code, compiler intermediate representations, and finite state machine models. Due to its high accuracy and ease of parsing, this type of data plays an important role in scientific computing, knowledge representation, automated design, and data management. Pretraining is a technique that learns general data representations by performing preliminary modeling on large-scale data. Pretraining modeling not only extracts essential grammatical and semantic information from text data but also provides shared knowledge for multiple downstream tasks, thereby improving model generalization and data utilization efficiency. However, existing pretraining techniques are primarily based on non-logically structured natural language data, focusing on capturing basic syntax, semantics, and contextual relationships while ignoring the characteristics of strongly logically structured data. For example, logically structured code data not only has grammatical and semantic rules but also includes strict logical control structures, conditional judgments, and program flow, requiring high-level and accurate accuracy. This makes it difficult for traditional pretraining methods to fully capture the inherent, strongly correlated logical structure of this data. Therefore, designing a pretraining method and system for logically structured data is a crucial issue.

[0003] Existing pre-training techniques primarily model logically structured data through two approaches: 1) sensing the data's components; and 2) predicting the data's structured representation. The first approach begins by selecting a specific component within the logically structured data, such as an identifier in code. The selected content is then masked with a certain probability. The masked content can be several tokens or all content belonging to that component. Finally, a deep learning model is used to predict the masked content or categories, thereby sensing the composition and specific content of the logically structured data and modeling the pre-trained model's overall representation of the data. The second approach first converts the logically structured data into a special structured sequence, such as the grammatical sequence of code data. Pre-training techniques similar to those used in natural language processing are then applied to this structured sequence, such as masking or adding noise to the original input and recovering the resulting form. This is then used for prediction by a deep learning model. Finally, the resulting complete structured sequence representation is reverse-converted back into its original token form, such as code text, to obtain the final output.

[0004] A major drawback of existing technologies is that they haven't fully explored methods for generating logically structured data. Currently, both the components of perception data and the structured representation of prediction data are generated using a one-time, front-to-back approach, as in traditional autoregressive methods. However, for logically structured data, the preceding content may depend on the following content. For example, adding functionality to the end of a codebase requires additional processing modules. In real-world development scenarios, programmers often struggle to write all utility functions in one go before writing other processing logic, and code is often supplemented and refined incrementally. When generating data from a front-to-back approach alone, existing pre-training methods are unable to observe the logical representation of the same data from different sequences and perspectives. Therefore, they struggle to effectively learn the underlying logical patterns within the data, requiring a large amount of data to learn and acquire information about the internal logical structure. Furthermore, in training models generated from a front-to-back approach alone, errors in the model's initial input can lead to irreversible errors in subsequent results, resulting in model hallucinations and unreliable outputs.

[0005] In summary, existing methods for pre-training logically structured data primarily guide the pre-training process based on two specific approaches: perceiving the composition of the data or predicting the structured representation of the data. However, these two approaches still face the challenge of not fully exploring the generation methods of logically structured data. This challenge means that current pre-training of logically structured data requires a large amount of data to learn and statistically analyze common logical patterns and representations, and that errors in the initial setup can lead to irreversible errors later on. Therefore, in the era of the booming development of large-scale pre-training models, designing efficient pre-training methods for logically structured data to improve the performance of the final pre-trained model has become a crucial issue. Summary of the Invention

[0006] To address the above problems, the present invention provides an emergent pre-training method and system for logically structured data modeling, which fully explores the logical structure that each piece of data may contain, and applies appropriate emergent methods to accurately generate target data content given different contexts, thereby more effectively processing downstream tasks.

[0007] To achieve the above objectives, the technical solution of the present invention includes the following contents.

[0008] A germinal pre-training method for logically structured data modeling, the method comprising:

[0009] Performing data logic analysis on the original data to obtain a structured representation of the original data;

[0010] Based on the structured representation, data germination is performed to obtain logical structured data x, and according to the growth path of the structured logical data x in the data germination process, several data sample pairs {x i-a,x i-b}; wherein the logical structured data x is data with clear logical rules or semantic structure, x i-a represents the data x before the germination in the i-th germination process, x i-b represents the data x after germination in the i-th germination process;

[0011] Based on the data sample pair {x i-a ,x i-b}Complete the training of the target model and obtain the germination pre-training model.

[0012] Furthermore, the data structure of the logically structured data x includes: natural language description text or code.

[0013] Furthermore, in the case where the data structure of the logical structured data x is a code, based on the data sample {x i-a ,x i-b Complete the pre-training of the target model and obtain the pre-trained target model, including:

[0014] Pre-training the target model on DeepSeekCoder-Base; wherein the target model is built based on the Decoder framework;

[0015] The data sample x i-a Input to the pre-trained target model and combine it with the data sample x i-b , obtain the loss of the pre-training task based on the germination strategy and the loss for the next token prediction task The pre-training task based on the germination strategy refers to taking the logical structure representation of the data as the object and starting to grow at any position in the structural data until a data form with a complete logical structure is germinated. The germination strategy includes: randomly selecting a node as the node to be grown, gradually generating from left to right, and selecting the node to be grown based on probability;

[0016] Combined with the loss and losses Get the final loss;

[0017] Backpropagation is performed based on the final loss to update the parameters of the pre-trained target model.

[0018] Furthermore, the loss Among them, n represents the number of germinations of logical structured data x, z represents the content of z tokens that need to be predicted during the i-th germination process, P (θ) represents the probability distribution fitted by the target model, θ represents the parameters of the target model, and y tRepresents the data sample x i-b The content of the tth node in x pred Represents the data sample x i-a The unknown data that needs to be predicted.

[0019] Furthermore, the loss Where k represents the data sample x i-a The number of tokens in P (θ) Represents the probability distribution fitted by the target model, x t Represents the data sample x i-a The tth token in x 1:t-1 Represents the data sample x i-a The first token to the t-1 token in .

[0020] Furthermore, the data sample pair {x i-a ,x i-b After completing the training of the target model and obtaining the germinal pre-training model, it also includes:

[0021] Setting a sprouting strategy, wherein the sprouting strategy includes: randomly selecting a node as a node to be grown, gradually generating from left to right, and selecting a node to be grown according to probability;

[0022] Based on the germination strategy, the data to be grown x is selected i-a The node to be grown;

[0023] Based on the selected nodes to be grown, the germination pre-training model is called to obtain the data to be grown x i-b , and based on the data to be grown x i-b , re-execute the method of selecting the data to be grown x based on the germination strategy i-a The nodes to be grown are continued until the set number of germinations is reached.

[0024] A germinal pre-training system for logically structured data modeling, the system comprising:

[0025] A data conversion module is used to perform data logic analysis on the original data and obtain a structured representation of the original data;

[0026] The data germination module is used to perform data germination based on the structured representation to obtain logical structured data x, and obtain several data sample pairs {x i-a ,x i-b}; wherein the logical structured data x is data with clear logical rules or semantic structure, x i-a represents the data x before the germination in the i-th germination process, xi-b represents the data x after germination in the i-th germination process;

[0027] Model training module, for training {x i-a ,x i-b}Complete the training of the target model and obtain the germination pre-training model.

[0028] An electronic device comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements any of the above-mentioned germination pre-training methods for logically structured data modeling.

[0029] A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement any of the above-mentioned germinal pre-training methods for logically structured data modeling.

[0030] A computer program product, characterized in that when the computer program product is run on a computer device, the computer device is caused to execute any of the above-mentioned germination pre-training methods for logically structured data modeling.

[0031] Compared with the prior art, the present invention has at least the following beneficial effects.

[0032] During the design phase, this invention comprehensively rethought the issues related to pre-training modeling of logically structured data, and achieved a new perspective on the germination-based pre-training of logically structured data from scratch. Leveraging this new perspective, the invention enhances the exploration of the generation methods of individual logically structured data and breaks the generation model that can only be done from the front to the back. It explores the growth methods of each germination point of each logically structured data as much as possible, thereby fully exploring the logical structure that each data may contain. Given different contexts, the invention applies appropriate germination methods to accurately generate target data content, thereby more effectively processing downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is the basic process of the germination pre-training method and system for logically structured data modeling. DETAILED DESCRIPTION

[0034] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific implementation cases and drawings.

[0035] This invention implements a germination-based pre-training method and system for logically structured data. This involves: on the input side, the user designs a data germination method based on serialized logically structured data; on the output side, the model designs pre-training tasks based on the germinated and grown data to guide the model pre-training process. The pre-trained model receives input logically structured data or related requirement descriptions and automatically synthesizes the required target data to meet the user's requirements or address the user's modification intent.

[0036] Specifically, based on the importance of fully utilizing the inherent logical association information of logically structured data and pre-training models, this paper designs a system to automatically solve the problems related to logically structured data pre-training and modeling. The input and output of this system are described as follows:

[0037] On the input side, during the logical structured data germination phase, users input raw data. The system serializes the data based on its internal logical characteristics and then performs subsequent structured germination. On the output side, after model pre-training and entering the application phase, users provide a description of the required logical structured data or the data that needs to be edited. When a description of the logical structured data requirement is entered, the model automatically generates data that meets the requirement description, such as code, through germination. When inputting logical structured data that requires editing, the model automatically edits the existing data to meet the new requirement or improve data quality.

[0038] At the output end, the system of the present invention trains a targeted model based on the large-scale logical structured data generated by germination, and automatically generates target data that meets the requirements through germination based on the knowledge learned by the model.

[0039] To improve existing pre-training technology, this paper: First, designs a germinal strategy for logically structured data to handle the growth of pre-training data; Second, uses the germinal pre-training technology to train deep learning models related to logically structured data; Third, finally, evaluates the generation results of the germinal pre-training system for logically structured data. The following is a detailed introduction:

[0040] 1. Emergent Strategies for Logically Structured Data

[0041] In order to better model the logical structured data, the present invention adopts Figure 1 The basic process shown in applies a sprouting growth strategy to logically structured data.

[0042] The core idea of the germination strategy for logically structured data is to allow logically structured data to germinate at any position without affecting the integrity and correctness of the overall logic, and to fully learn and perceive the logic and structural information within the data by utilizing the different contexts encountered when a single data sample germinates at different positions. In the present invention, logically structured data is defined as a data form with clear logical rules or semantic structures, such as imperative programming language code, compiler intermediate representation, and finite state machine model data. The germination of data is defined as a process in which data grows from zero and gradually grows into final data with a complete logical structure, wherein each germination can start from any position where growth is still required. For example, in code data with strong logical structured characteristics, the content of the code can start to grow from the root node of the abstract syntax tree until it grows into complete code content, wherein any syntax tree node can be selected for germination, and the grown result can restore the complete code content under the constraints of the code logical structure.

[0043] according to Figure 1 As shown, data germination involves the analysis of the logical structure of data. The germination strategy takes the representation of the logical structure as the object, and its process can germinate from any logical structure position and grow into complete data. The germination of the logical structure can be divided into the processing of three types of nodes: generated nodes, unknown nodes, and nodes to be predicted. Among them, the generated nodes represent the nodes with determined content in the logical structure, the unknown nodes represent the logical structure under the node is still unclear, and the nodes to be predicted are a special type of unknown nodes and their content will be determined during this germination process. Taking the code as an example, Figure 1 The first germination of the data growth process indicates that four unknown abstract syntax tree nodes are expanded from the root node, indicating that at least four syntax tree subtrees are yet to be generated (germinated). In the second germination, the third unknown node is selected as the node to be predicted for germination. After growth, the logical structure of the data is expanded to include one generated node and one unknown node awaiting the next growth. In the third germination process, the second node of the second layer of the entire abstract syntax tree is selected as the node to be predicted for germination. After growth, the overall logical structure is expanded to include three confirmed generated nodes and one unknown node awaiting the next growth. Similarly, through continuous germination, a complete logical structure representation will eventually be grown, that is, a completed abstract syntax tree structure, thereby generating complete code. Similar germination methods can also be applied to other logically structured data, and each step of the growth process can determine the content of one or more nodes in the logical structure.

[0044] For each input logical structure data x, a series of specific growth data x1, x2, x3, ..., x3 involved in its germination process can be constructed. n. Among them, x1 represents the first germination process of the data, x i represents the i-th germination process of the data, and n represents the total number of n germinations required to obtain data x with a complete logical structure. i For example, it can be split into the data before germination x i-a And the data x that completes this germination i-b The strategy of germination is i-a Select a node as the node to be predicted and generate x after the prediction is completed. i-b Finally, an original logical structured data will be decomposed into many data sample pairs related to its growth process. For example, the i-th data sample pair can be represented as {x i-a ,x i-b The data sample pairs of these growth processes can be used as the input and output of the model, allowing the model to learn the logical structure information within the data and accurately generate complete content including logical constraints in the inference stage.

[0045] This invention breaks down a single data point into a large amount of specific germination data based on the growth path of logically structured data during the germination process. The germination strategy closely adheres to the characteristics of logically structured data because, under strong constraints, the model can adaptively choose an appropriate growth method based on the problem requirements without affecting the correctness of the final model generation results. Furthermore, because the growth process can begin at any point, this strategy allows the model to explore the growth of data under various contextual constraints in a variety of ways and paths, thereby fully perceiving the constraints of the data's internal logic and structure.

[0046] 2. Germination Pre-training Model

[0047] The germinal pre-training model in the present invention takes code data with strong logical structure constraints as an example, and applies the above-mentioned germinal strategy for logically structured data for training and development. The specific development of the germinal pre-training model can be regarded as a combination of the following three parts: model structure, input and output representation, and pre-training tasks. It is worth noting that the final germinal pre-training model is an implementation method corresponding to one of the logically structured data forms, namely, code, of the germinal strategy for logically structured data proposed in the present invention.

[0048] 1) Germinal pre-training model structure

[0049] The germination strategy for logically structured data described in this paper is a general approach. The design of the pre-trained model for code data uses a classic decoder-based framework. On this basis, the present invention continues pre-training on the 1.3 billion parameter-scale model weights of DeepSeekCoder-Base, allowing the model to benefit from the knowledge and representations learned by DeepSeekCoder-Base, which is particularly beneficial for code-related tasks.

[0050] 2) Input and output representation

[0051] The formal description of the input accepted by the germination pre-training model proposed in this invention is as follows, where w i Represents the i-th token of the original data x'. [BOS] and [EOS] are special tags in the input, indicating the start and end of the sentence.

[0052] x'=[BOS],w1,w2,…,w m , [EOS]

[0053] According to the logical structure representation in the data, the logical structure contained therein can be serialized as follows:

[0054] x=[BOS],g1,g2,…,g k , [EOS]

[0055] Among them, g i A token representing the serialization of a logical structure. For example, in code, this could be a specific grammatical rule. The model generates structured code by inputting and outputting different grammatical rules. If the logically structured data itself already represents a highly logical sequence, then x and x' can have the same form. For ease of presentation, x will be used as the model input data format.

[0056] Accordingly, the output can be formally described as follows, where y i The token representing the i-th output, [BOS] and [EOS] represent the start and end tokens of the output, respectively.

[0057] y=[BOS],y1,y2,…,y z , [EOS]

[0058] Finally, after completing all the necessary growth steps, the model converts the output back into its original data form. Similarly, if the output data is already highly structured, then germination can be performed directly on the original data structure without the need for structured representation. Therefore, no conversion is required during the input process or after the output.

[0059] It is worth noting that, depending on the different requirements of different tasks, the input x can be a natural language description text (such as the content contained in the input describing the logically structured code in the code generation task) or it can be code that needs to be modified (such as the input of old logically structured code data in the code enhancement task).

[0060] 3) Pre-training tasks

[0061] The pre-training tasks involved in this invention are designed to effectively combine pairs of process data generated from the same data point at different locations. Furthermore, to preserve the capabilities and adaptability of the pre-trained model for traditional tasks, the proposed germination strategy can also be combined with traditional pre-training tasks.

[0062] a) Pre-training tasks based on germination strategies

[0063] This pre-training task aims to fully explore the logic and structure within the data using a germination strategy. Unlike traditional pre-training tasks that add random masks or noise to the input data and then predict sequentially from front to back, this task targets the logical structure of the data, starting growth at any location within the structured data until a data form with a complete logical structure emerges. It is worth noting that the germination and growth designed in this invention can start from any location, and the steps of each growth can be adjusted according to different tasks.

[0064] Formally, the loss for this task can be described as follows:

[0065]

[0066] Among them, t represents the current generated step, y t Indicates the content of the tth node in the generated logical structure that needs to be predicted, and z indicates that the content of z tokens needs to be predicted during this growth process. pred Indicates the unknown data that needs to be predicted in the data that has been marked in the logical structure. i represents the data generated during the i-th growth process. A total of n growths can generate complete logical structured data. (θ) Represents the probability distribution fitted by the code pre-training model.

[0067] b) Tasks based on traditional pre-training

[0068] In order to make the final model adaptable to the traditional model usage, the second pre-training of the present invention is based on the next token prediction task of traditional pre-training, specifically gradually generating the content in the token from the front to the back.

[0069] Formally, the loss for this task can be described as follows:

[0070]

[0071] Among them, x contains a total of k tokens, and the pre-training task predicts the content of the t-th token by inputting all tokens before the t-th token until all tokens are generated.

[0072] c) Combination of pre-training objectives

[0073] Each pre-training task can be combined according to its different importance in different tasks, and α and β represent different weights. In the germinal pre-training strategy, α is a number greater than 0 and less than or equal to 1, and β represents a number greater than or equal to 0 and less than or equal to 1. For a clearer demonstration, the present invention treats each pre-training task equally and adds the loss value of each task to obtain the overall loss, that is, the values of α and β are both 0.5. The final loss function is described as follows:

[0074]

[0075] It is worth noting that the present invention retains the germinal training strategy proposed by the present invention under different task conditions, and any training strategy added on the basis of this germinal strategy is within the scope of the present invention.

[0076] Application phase: Input the required description of logically structured data (such as code) or the logically structured data that needs to be edited into the trained germination pre-training model to generate the required target logically structured data. Specifically, the strategy for selecting nodes to be grown is first set, such as randomly selecting nodes to be grown, gradually generating from left to right, and selecting nodes with the highest model prediction probability. Then, based on the selected nodes to be grown, the model is called and the germination process is executed. Finally, by repeating the previous step, the logical structure of the target data is gradually improved until data with a complete logical structure is generated.

[0077] III. Evaluation of the Technical Effects of the Invention

[0078] Finally, in order to evaluate the functionality of the present invention, the present invention selected code data generation with logical structured characteristics to evaluate the capabilities of the germination pre-training model, and the specific task was the code structure filling task.

[0079] Code Structure Filling Task: This task involves inputting several code snippets with missing structures, and then having the pre-trained model automatically fill in the required structure at a specific location. This task requires the model to understand the missing structure and automatically generate content that reflects the logical dependencies between the previous and next steps. This task assesses the system's ability to understand the logical structure within the data and fill in the correct structure accordingly.

[0080] Evaluation results show that in the code structure completion task, the proposed method achieves a full multi-token prediction accuracy exceeding 20% on a 500-item test dataset. Notably, even if a single incorrectly generated token is considered an error, the result is considered incorrect. Models with the same parameter size often have an accuracy rate below 5% because they struggle to fill in structural information that complies with preceding and following logical constraints.

[0081] Practical application scenarios of the present invention:

[0082] Logically structured data plays an important role in scientific computing, knowledge representation, automated design, and data management due to its high precision and ease of parsing. Automatically generating this data can effectively reduce the cost of manually constructing logically structured data. For example, automatically generating logically structured code data can effectively improve software development efficiency and reduce labor costs as modern software complexity increases. Pre-trained models have great potential in achieving automatic generation of logically structured data. Utilizing the germination-type pre-training strategy of the present invention, pre-trained models can further understand and utilize the logical and structural information within such data, thereby further improving efficiency.

[0083] The specific embodiments of the present invention disclosed above are intended to facilitate understanding and implementation of the present invention. Those skilled in the art will appreciate that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the embodiments disclosed in this specification; the scope of protection of the present invention shall be determined by the scope defined in the claims.

Claims

1. A germinal pre-training method for logically structured data modeling, characterized in that: The method comprises: Performing data logic analysis on the original data to obtain a structured representation of the original data; Based on the structured representation, data germination is performed to obtain logical structured data x, and according to the growth path of the structured logical data x in the data germination process, several data sample pairs {x i-a ,x i-b }; wherein the logical structured data x is data with clear logical rules or semantic structure, x i-a represents the data x before the germination in the i-th germination process, x i-b represents the data x after germination in the i-th germination process; Based on the data sample pair {x i-a ,x i-b }Complete the training of the target model and obtain the germination pre-training model.

2. The method according to claim 1, characterized in that The data structure of the logically structured data x includes: natural language description text or code.

3. The method according to claim 2, characterized in that In the case where the data structure of the logical structured data x is a code, based on the data sample {x i-a ,x i-b Complete the pre-training of the target model and obtain the pre-trained target model, including: Pre-training the target model on DeepSeekCoder-Base; wherein the target model is built based on the Decoder framework; The data sample x i-a Input to the pre-trained target model and combine it with the data sample x i-b , obtain the loss of the pre-training task based on the germination strategy and the loss for the next token prediction task The pre-training task based on the germination strategy refers to taking the logical structure representation of the data as the object and starting to grow at any position in the structural data until a data form with a complete logical structure is germinated. The germination strategy includes: randomly selecting a node as the node to be grown, gradually generating from left to right, and selecting the node to be grown based on probability; Combined with the loss and losses Get the final loss; Backpropagation is performed based on the final loss to update the parameters of the pre-trained target model.

4. The method according to claim 3, characterized in that The loss Among them, n represents the number of germinations of logical structured data x, z represents the content of z tokens that need to be predicted during the i-th germination process, P (θ) represents the probability distribution fitted by the target model, θ represents the parameters of the target model, and y t Represents the data sample x i-b The content of the tth node in x pred Represents the data sample x i-a The unknown data that needs to be predicted.

5. The method according to claim 3, characterized in that The loss Where k represents the data sample x i-a The number of tokens in P (θ) Represents the probability distribution fitted by the target model, x t Represents the data sample x i-a The tth token in X 1:t-1 Represents the data sample x i-a The first token to the t-1 token in .

6. The method according to claim 1, characterized in that The method based on the data sample {x i-a ,x i-b After completing the training of the target model and obtaining the germinal pre-training model, it also includes: Setting a sprouting strategy, wherein the sprouting strategy includes: randomly selecting a node as a node to be grown, gradually generating from left to right, and selecting a node to be grown according to probability; Based on the germination strategy, the data to be grown x is selected i-a The node to be grown; Based on the selected nodes to be grown, the germination pre-training model is called to obtain the data to be grown x i-b , and based on the data to be grown x i-b , re-execute the method of selecting the data to be grown x based on the germination strategy i-a The nodes to be grown are continued until the set number of germinations is reached.

7. A germinal pre-training system for logically structured data modeling, characterized in that: The system comprises: A data conversion module is used to perform data logic analysis on the original data and obtain a structured representation of the original data; The data germination module is used to perform data germination based on the structured representation to obtain logical structured data x, and obtain several data sample pairs {x i-a ,x i-b }; wherein the logical structured data x is data with clear logical rules or semantic structure, x i-a Indicates that the data x is i Data before germination during the germination process, x i-b represents the data x after germination in the i-th germination process; Model training module, for training {x i-a ,x i-b }Complete the training of the target model and obtain the germination pre-training model.

8. An electronic device, characterized in that: The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the germination pre-training method for logical structured data modeling as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the germination pre-training method for logically structured data modeling as described in any one of claims 1 to 6.

10. A computer program product, characterized in that When the computer program product is run on a computer device, the computer device is enabled to execute the germinal pre-training method for logically structured data modeling according to any one of claims 1 to 6.

Images