Text analysis method and device based on double engines and storage medium

Abstract

The invention discloses a double-engine-based text analysis method and device and a storage medium, and relates to the technical field of text analysis, and the method comprises the following steps: in response to an input text, determining a text type and a text length of the input text according to a character interval and a binary coding value of the input text; processing the input text based on the first model to obtain a first analysis result, and processing the input text based on the second model to obtain a second analysis result; fusing the first analysis result and the second analysis result based on the text type and the text length, and generating a stable field tag and a local feature corresponding to the input text; and generating a text evaluation value of the input text according to the stable field label and the local feature. The technical effects that corpus suitability preliminary screening, corpus quality and potential defect two-dimensional evaluation and screening decision making are completed before training, corpus mismatched with a training target and having defects are removed in advance, and invalid training computing resource consumption is reduced are achieved.

Description

Technical Field

[0001] This application relates to the field of text parsing technology, and in particular to a text parsing method, device and storage medium based on a dual-engine architecture. Background Technology

[0002] The development of artificial intelligence, especially large language models, has enabled them to excel in numerous tasks. The performance of these models is fundamentally based on the large-scale text corpora used for their training. The quality, diversity, and cleanliness of the corpus directly determine the strength of the model's knowledge base and its capability boundaries, and are key prerequisites for the model's success or failure.

[0003] Currently, the industry practice typically relies on the authority of data sources, supplemented by basic automated cleaning, such as text deduplication, format standardization, and sensitive information filtering. The processed corpus is then directly fed into the model for training, resulting in the inability to effectively perceive its true effectiveness and potential defects before training. Problems with the corpus only become apparent after training is complete due to poor performance on downstream tasks, leading to unnecessary consumption of computational costs for the project. Summary of the Invention

[0004] The main purpose of this application is to provide a text parsing method, device and storage medium based on dual engines, which aims to solve the technical problem that the quality of training corpora can only be evaluated after the model training is completed, resulting in a waste of computing resources.

[0005] To achieve the above objectives, this application provides a dual-engine-based text parsing method, which includes: In response to the input text, the text type and text length of the input text are determined based on the character spacing and binary encoding value of the input text; The input text is processed based on a first model to obtain a first parsing result, and the input text is processed based on a second model to obtain a second parsing result. The first model includes a reward module, a penalty module, and a compensation module, and the second model includes a preset number of expression factor dimensions. Based on the text type and the text length, the first parsing result and the second parsing result are fused to generate a stable field label and local features corresponding to the input text; The text evaluation value of the input text is generated based on the stable field label and local features.

[0006] In one embodiment, the step of processing the input text based on the first model to obtain a first parsing result includes: The reward module, penalty module, and compensation module included in the first model are invoked to parse the input text and obtain reward features, penalty features, and compensation features, respectively. The first parsing result of the input text is generated by calculating the comprehensive score of reward features, penalty features and compensation features by combining the nonlinear weighted scoring function preset by the first model with the dynamic function graph. The reward features, penalty features and compensation features constitute the three-fold five-dimensional basic structure of the first model.

[0007] In one embodiment, the step of processing the input text based on the second model to obtain a second parsing result includes: The second model is invoked to extract features from the input text using a preset number of expression factor dimensions, resulting in a feature vector. The high-dimensional representation of the text is generated by using the three-level vector field mapping structure of sentence, paragraph, and text built into the second model; By combining the high-dimensional representation of the text with the dimensional features of each expression factor through a weighted ensemble algorithm, a second analytical result is output, which includes a dimensional distribution heatmap and an expression stability spectrum diagram.

[0008] In one embodiment, the step of fusing the first parsing result and the second parsing result based on the text type and the text length to generate a stable field label and local features corresponding to the input text includes: Based on the text type and the text length, the first parsing result and the second parsing result are fused to obtain the stable field label corresponding to the input text; Based on the first and second parsing results, extract the global feature package corresponding to the input text; The local features of the input text are determined based on the global feature package and the stable field label.

[0009] In one embodiment, the step of fusing the first parsing result and the second parsing result to obtain the stable field label corresponding to the input text includes: Based on the text type and the text length, a preset mapping table is obtained to determine the first model weight and the second model weight. The mapping table represents the mapping relationship between text type and text length to the first model weight and the second model weight. The stable field label corresponding to the input text is obtained by fusing the first analysis result and the second analysis result with the first model weight and the second model weight.

[0010] In one embodiment, the step of extracting the global feature package corresponding to the input text based on the first parsing result and the second parsing result includes: The core features of the thought dimension are extracted from the first analysis result to obtain a subset of thought features. The first analysis result includes the global first analysis result score of the input text and the sub-values ​​of each thought dimension. The subset of thought features includes a set of core concepts, a theoretical framework integrity marker, and a global abstraction degree statistical value. Extract the core features of the structural dimension from the second parsing result to obtain a subset of structural features. The second parsing result includes the global second parsing result score of the input text and the sub-values ​​of each structural dimension. The subset of structural features includes the average symbol density, the average logical depth, the baseline of syntactic complexity, and the semantic vector stability parameter. Frequency statistics and weight ranking are performed on the core concept set in the subset of thought features. A preset number of highly related concepts are selected to form a core concept table. The ranking weight is determined by weighting the contribution value of each concept in each thought dimension. Based on the global abstraction statistics of the thought-type feature subset and the average symbol density of the structure-type feature subset, a statistical baseline is calculated, which includes the average concept density, the standard deviation of abstraction, and the symbol density fluctuation threshold. Theoretical framework markers are extracted from the first parsing result, and full-text semantic vectors are extracted from the second parsing result. The full-text semantic vectors are generated by a three-level vector field mapping structure of sentence-paragraph-article.

[0011] In one embodiment, the step of determining the local features of the input text based on the global feature package and the stable field label includes: Based on the global feature package, the first local features of each paragraph in the input text and the second local features of each paragraph compared to the entire input text are determined. The third local features of each paragraph are determined by combining the first local features and the second local features; Based on the stable field label modulation of the third local feature, the local features of each paragraph in the input text are obtained.

[0012] In one embodiment, the step of generating a text evaluation value for the input text based on the stable field label and local features includes: Obtain the association information of the input text, which includes the application identifier of the input text and the fuzzy semantics of the text. The fuzzy semantics of the text are obtained by comparing the full-text semantic vector output by the second model with the preset semantic clustering center. The application identifier is determined by parsing the upload interface or file format of the input text. By combining the text type of the input text, the application identifier, and the fuzzy semantics of the text, a preset scene tag library is matched to determine the scene tag of the input text; Extract the global confidence quantification value corresponding to the stable field label, as well as the paragraph Q value sequence, entropy peak ratio, and average Q value contained in the local features, wherein the entropy peak ratio is the ratio of the number of paragraphs with Q values ​​greater than a preset threshold to the total number of paragraphs; The corresponding text evaluation value mapping rule is invoked according to the scene label. The mapping rule is preset with weighting coefficients for global confidence quantification, entropy peak ratio, and average Q value under different scenes. Based on the weighting coefficients of the mapping rules, the weighted sum of the global confidence quantification, entropy peak proportion, and average Q value is calculated to obtain the original score; The original score is converted into a text evaluation value of the input text within a preset range using a linear mapping formula. Furthermore, to achieve the above objective, this application also provides a dual-engine-based text parsing device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the dual-engine-based text parsing method described above.

[0013] In addition, to achieve the above objectives, this application also provides a storage medium, which is a computer-readable storage medium, on which a program implementing a dual-engine-based text parsing method is stored. The program implementing the dual-engine-based text parsing method is executed by a processor to implement the steps of the dual-engine-based text parsing method as described above.

[0014] This application provides a dual-engine-based text parsing method. Responding to input text, the method determines the text type of the input text based on the character spacing and binary encoding value; processes the input text based on a first model to obtain a first parsing result, and processes the input text based on a second model to obtain a second parsing result. The first model includes a reward module, a penalty module, and a compensation module, and the second model includes a preset number of expression factor dimensions. Based on the model weights determined by the text type, the first parsing result and the second parsing result are fused to generate a text evaluation value for the input text.

[0015] In summary, this application addresses the technical problem of wasted computational resources caused by requiring training corpora to be evaluated only after model training is completed. It achieves a two-dimensional evaluation and selection decision-making process: initial screening of corpus suitability, corpus quality and potential defects are performed before training; corpora that are mismatched with the training objective or have defects are eliminated in advance; and the consumption of computational resources for ineffective training is reduced. This approach involves responding to input text, first determining the text type based on character spacing and binary encoding values, then processing the input text using a first model containing reward, penalty, and compensation modules to obtain a first parsing result, and finally processing the input text using a second model containing a preset quantity expression factor dimension to obtain a second parsing result. Finally, it fuses the first and second parsing results based on model weights determined by the text type to generate a text evaluation value for the input text. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating an embodiment of the dual-engine-based text parsing method of this application. Figure 2 This is a flowchart illustrating Embodiment 5 of the text parsing method based on a dual-engine in this application; Figure 3 This is a schematic diagram of the hardware structure involved in the embodiment of the dual-engine text parsing device of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] Currently, the industry practice typically relies on the authority of data sources, supplemented by basic automated cleaning, such as text deduplication, format standardization, and sensitive information filtering. The processed corpus is then directly fed into the model for training, resulting in the inability to effectively perceive its true effectiveness and potential defects before training. Problems with the corpus only become apparent after training is complete due to poor performance on downstream tasks, leading to unnecessary consumption of computational costs for the project.

[0023] This application responds to input text by determining the text type and text length of the input text based on the character spacing and binary encoding value of the input text; processing the input text based on a first model to obtain a first parsing result, and processing the input text based on a second model to obtain a second parsing result, wherein the first model includes a reward module, a penalty module, and a compensation module, and the second model includes a preset number of expression factor dimensions; fusing the first parsing result and the second parsing result based on the text type and the text length to generate a stable field label and local features corresponding to the input text; and generating a text evaluation value of the input text based on the stable field label and local features.

[0024] In summary, this application addresses the technical problem of wasted computational resources caused by requiring training corpora to be evaluated only after model training is completed. It achieves a two-dimensional evaluation and selection decision-making process: initial screening of corpus suitability, corpus quality and potential defects are performed before training; corpus mismatched with training objectives and corpus with defects are eliminated in advance; and the consumption of computational resources for ineffective training is reduced. This approach involves responding to input text, first determining the text type based on character spacing and binary encoding values, then processing the input text using a first model containing reward, penalty, and compensation modules to obtain a first parsing result, and then processing the input text using a second model containing a preset quantity expression factor dimension to obtain a second parsing result. Finally, it merges the first and second parsing results based on text type and text length to generate a text evaluation value for the input text.

[0025] It should be noted that the execution entity in this embodiment can be a dual-engine text parsing system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a dual-engine text parsing device capable of performing the above functions. This embodiment does not specifically limit the specific implementation. The following description uses a dual-engine text parsing device as the execution entity to illustrate this embodiment and the subsequent embodiments.

[0026] Based on this, Embodiment 1 of this application proposes a text parsing method based on a dual-engine approach. Please refer to... Figure 1 The dual-engine-based text parsing method includes steps S10 to S40: Step S10: In response to the input text, determine the text type and text length of the input text based on the character spacing and binary encoding value of the input text.

[0027] In this embodiment, the input text is the text content submitted by the user for cognitive value analysis, which can be various written texts such as academic papers, popular science articles, and business copy. Character spacing is the distance between two adjacent characters in the input text, including visible spaces and invisible encoded spaces. Binary encoded values ​​are the binary data representations corresponding to each character in the input text, obtained by converting the character's ASCII or Unicode code. Text type is a category categorized according to the text's purpose, style, and structure, such as academic papers, popular science articles, business copy, and literary works. Text length is the total number of characters contained in the input text, counted in units of individual characters, reflecting the overall length and scale of the text.

[0028] As an optional implementation, the system first performs character-level traversal of the input text, records the interval values ​​between each adjacent character, and statistically analyzes the interval distribution characteristics; then, it converts each character of the input text into its corresponding binary code value, analyzes the distribution pattern and characteristics of the code value; combined with a preset text type feature library, it determines the text type through feature matching; at the same time, it counts the total number of characters recorded during the traversal process and directly uses it as the text length.

[0029] As another optional implementation, regular expressions are used to extract character spacing features from the input text to distinguish different types of spacing patterns, such as Chinese and English spacing and punctuation spacing; text characters are converted into binary encoded values ​​in batches using an encoding conversion tool to generate an encoding sequence; the spacing pattern and encoding sequence are input into a pre-trained text type recognition model to output the text type; and the total number of characters in the input text is quickly counted using a character counting function to obtain the text length.

[0030] Step S20: Process the input text based on the first model to obtain a first parsing result, and process the input text based on the second model to obtain a second parsing result.

[0031] In this embodiment, the first model is used to analyze the value of the thought dimension of the input text, focusing on the degree of freedom of thought in the conceptual semantic space. The reward module is a functional unit in the first model used to extract the positive value features of the text's thought, corresponding to the advantage of the possibility space of thought expression. The penalty module is a functional unit in the first model used to identify the closed features of text cognition, corresponding to the limitations of thought expression. The compensation module is a functional unit in the first model used to strengthen the text's logical verification and the integrity of thought, supplementing the scientific support of thought expression. The first analysis result is the quantitative result reflecting the value of the text's thought dimension output by the first model, including the global first analysis result score and the sub-values ​​of each thought dimension. The second model is used to analyze the complexity of the input text's structural dimension, focusing on the organizational difficulty of expression in the structural form space. The expression factor dimension is a specific dimension in the second model used to characterize the text's structural features, such as grammatical complexity and symbol density, used to quantify the complexity of structural organization. The second analysis result is the quantitative result reflecting the complexity of the text's structural dimension output by the second model, including the global second analysis result score and the sub-values ​​of each structural dimension. The first model includes a reward module, a penalty module, and a compensation module, while the second model includes a preset number of expression factor dimensions.

[0032] It should be noted that the first and second models can be implemented using deep learning models or statistical learning models, including but not limited to Transformer, RNN, CNN, GNN, and feature engineering models. The following implementation uses Transformer as an example only and is not the only limitation.

[0033] As an optional implementation, the reward module of the first model is invoked to extract positive thought features from the text from five dimensions: language density, structural complexity, depth of thought, system penetration, and model nesting. The penalty module is invoked to identify negative features in the text, such as inconsistencies, conceptual confusion, and cognitive closure. The compensation module is invoked to verify the logical consistency and scientific validity of the text's ideas based on a knowledge base, generating compensation adjustment values. A non-linear weighted scoring function, combined with a dynamic function graph, is used to calculate the comprehensive score of positive features, negative features, and compensation adjustment values, yielding the first analytical result. Simultaneously, the second model is invoked to activate the preset forty-five expression factor dimensions, extracting features from the input text to obtain the original feature values ​​for each dimension. A three-level vector field mapping structure (sentence, paragraph, and text) is used to transform the original feature values ​​into a high-dimensional representation of the text. A weighted ensemble algorithm is used to fuse the high-dimensional representation and the features of each expression factor dimension, outputting a second analytical result containing a dimension distribution heatmap and an expression stability spectrum diagram.

[0034] As an alternative implementation, the first model's reward, penalty, and compensation modules process the input text in parallel: the reward module constructs a concept graph to uncover the value of thought associations, the penalty module detects cognitive deficiencies through semantic conflict detection, and the compensation module supplements logical support through theoretical framework matching. The outputs of the three modules are input into a dynamic weight allocation model, which adaptively adjusts the weights of each module based on text features to calculate the first analytical result. The second model selects seven core expression factor dimensions, including syntactic complexity, symbol density, vector stability, computational logic depth, structural hierarchy clarity, concept dimension resolution, and cross-semantic domain transfer capability. It calculates the scores of each dimension through a feature extraction network to construct a structural feature matrix. Based on a matrix factorization algorithm, it generates a high-dimensional representation of the text structure and combines visualization tools to generate a dimension distribution heatmap and an expression stability spectrum diagram, forming the second analytical result.

[0035] Step S30: Based on the text type and the text length, fuse the first parsing result and the second parsing result to generate a stable field label and local features corresponding to the input text.

[0036] In this embodiment, the stability field label is a global confidence quantification label reflecting the overall ideological integrity and structural stability of the input text. It corresponds to the fusion result of the global first parsing result and the global second parsing result, providing a constraint boundary for the calculation of local features. Local features are a set of features reflecting the ideological strength at the paragraph level of the input text, including the ideological value and structural complexity quantification indicators of each paragraph, corresponding to the expressive energy at the paragraph level.

[0037] As an optional implementation, a preset weight mapping table is queried based on the text type and text length to determine the fusion weight of the first and second parsing results; a weighted summation algorithm is used to fuse the global first parsing result score of the first parsing result and the global second parsing result score of the second parsing result to generate a stable field label; based on the sub-items of each idea dimension of the first parsing result and the sub-items of each structure dimension of the second parsing result, features such as the core concept set, theoretical framework marker, average symbol density, and average logical depth are extracted to form a global feature package; the local features of each paragraph are compared with the global feature package, and combined with the constraints of the stable field label, the local features of each paragraph are determined.

[0038] As another optional implementation, a fusion model is constructed, with text type and text length as input parameters, and the fusion ratio of the first and second parsing results is dynamically adjusted; a multi-dimensional fusion algorithm is used to integrate the value of the thought dimension and the complexity of the structural dimension to output a stable field label; a clustering algorithm is used to extract key features from the first and second parsing results to generate a global feature package; a paragraph feature comparison standard is established based on the global feature package, and the local features of each paragraph are calculated by combining the modulation coefficient of the stable field label.

[0039] Step S40: Generate a text evaluation value for the input text based on the stable field label and local features.

[0040] In this embodiment, the text evaluation value is a quantitative label that reflects the overall cognitive value of the input text. It is mapped to a preset range, making it easier to intuitively understand the cognitive value level of the text.

[0041] As an optional implementation, the associated information of the input text is obtained, including the application identifier and fuzzy semantics of the text. The application identifier is determined by parsing the upload interface or file format of the input text, and the fuzzy semantics of the text is obtained by comparing the full-text semantic vector in the second parsing result with the preset semantic clustering center. Combining the text type, application identifier, and fuzzy semantics of the text, a preset scene label library is matched to determine the scene label of the input text. The global confidence quantification value corresponding to the stable field label is extracted, as well as the paragraph Q-value sequence, entropy peak proportion, and average Q-value contained in the local features, where the entropy peak proportion is the ratio of the number of paragraphs with Q-values ​​greater than a preset threshold to the total number of paragraphs. The corresponding text evaluation value mapping rule is called according to the scene label. This mapping rule has preset weighting coefficients for the global confidence quantification value, entropy peak proportion, and average Q-value under different scenes. The weighted sum of the three is calculated based on the weighting coefficients to obtain the original score. The original score is converted into an integer in the range of 1 to 200 through a linear mapping formula to generate the text evaluation value.

[0042] As another optional implementation, application association information of the input text is obtained through the system interface, and fuzzy semantics of the text are extracted from the second parsing result using a semantic understanding model; the text type, application identifier, and semantic topic are combined into scene features, and fuzzy matching is performed with the scene label library to determine the scene label; the paragraph Q-value sequence in the local features is statistically analyzed, the entropy peak ratio and average Q-value are calculated, and the global confidence quantification value of the stable field label is extracted; a differentiated weighted coefficient matrix is ​​preset for different scene labels, and the weighted sum of the global confidence quantification value, entropy peak ratio, and average Q-value is calculated based on the coefficients in the matrix to obtain the original score; a nonlinear mapping function is used to map the original score to the interval of 1 to 200 to generate the text evaluation value.

[0043] This embodiment uses a dual-model collaborative approach to analyze the ideological value and structural complexity of a text. By dynamically adjusting the fusion weights based on text type and length, the generated stable field labels and local features can comprehensively characterize the global confidence level and paragraph ideological strength of the text. The scenario-adaptive text evaluation value generation method makes the quantification results more in line with practical application needs. The entire process achieves objective and accurate quantification of the cognitive value of the text, avoids the bias of subjective evaluation, and provides a scientific basis for text optimization and quality assessment.

[0044] Based on any of the above embodiments, in Embodiment 2 of this application, step S10 includes: extracting the binary data stream of the input text and mapping it to a preset encoding table; determining the basic language type according to the encoding range of ASCII, GBK, or UTF-8; determining the character recognition result according to preset character rules and the character intervals of the input text, wherein the character rules are that consecutive 1-byte characters combined with spaces are marked as English text, consecutive multi-byte characters combined with no spaces are marked as Chinese text, and special intervals are marked as classical text; calling an AI language classifier, combining the basic language type and the character recognition result, matching a preset type library, and outputting the text type, the AI ​​language classifier being trained through vocabulary distribution and syntactic pattern data. Further, the number of characters in the input text is used as the text length.

[0045] In this embodiment, the binary data stream is a continuous data sequence formed by converting each character of the input text into binary form, containing the underlying encoding information of the text characters. The preset encoding table is a database that pre-stores the encoding value ranges of mainstream encoding standards such as ASCII, GBK, and UTF-8 and their corresponding language character mappings. The ASCII encoding range refers to the 0-127 decimal encoding range used to represent uppercase and lowercase English letters, numbers, and common symbols; the GBK encoding range refers to the 8140-FFFF hexadecimal encoding range used to represent simplified and traditional Chinese characters and some symbols; the UTF-8 encoding range refers to the 1-4 byte encoding range used to represent multilingual characters, with different byte lengths and encoding value ranges corresponding to different language characters. The basic language type is the language category of the input text initially determined based on the encoding range, such as English, Chinese, or a mix of both. The preset character rules are a pre-defined set of rules that determine the text category based on character byte length and spacing characteristics. Character spacing refers to the spacing state between adjacent characters in the input text, including spaces, no spaces, and special ancient Chinese punctuation spacing (such as ".", "、", ";", etc.). Character recognition results, derived from combining character rules and character spacing, represent a preliminary judgment of the specific category of the input text (e.g., English, Chinese, classical texts). The AI ​​stylistic classifier is an artificial intelligence model trained on the lexical distribution features of large-scale texts (e.g., high-frequency technical terms in academic texts, high-frequency simple words in children's texts) and syntactic pattern features (e.g., inverted sentences in classical texts, elliptical sentences in spoken texts), possessing the ability to accurately identify text categories. The preset type library is a pre-built database storing the correspondence between features and identifiers of various text types (e.g., academic papers, children's books, classical texts, online novels, etc.). The text type is the final, determined specific category of the input text after basic language type determination, character recognition result verification, and matching by the AI ​​stylistic classifier.

[0046] As an alternative implementation, first perform a full - character scan on the input text, convert each character into corresponding binary data to form a complete binary data stream; map each binary data segment in the binary data stream to a preset coding table one by one, and compare whether the byte length and coding value of this data segment fall within the corresponding coding range of ASCII, GBK or UTF - 8. If more than 80% of the data segments fall within the ASCII coding range, the basic language type is initially determined to be English. If more than 80% of the data segments fall within the GBK coding range, it is initially determined to be Chinese. If the proportion of data segments in both coding ranges exceeds 30%, it is initially determined to be a Chinese - English mixture; then, analyze the input text segment by segment according to the preset character rules, and count the proportion of text segments with consecutive single - byte characters and spaces between adjacent characters. If the proportion exceeds 60%, the character recognition result is marked as an English - type text; count the proportion of text segments with consecutive multi - byte characters and no spaces between adjacent characters. If the proportion exceeds 60%, the character recognition result is marked as a Chinese - type text; if ancient Chinese words such as "之乎者也" appear in the text and are accompanied by special intervals (such as "。", ";"), the character recognition result is marked as a classical text; finally, input the basic language type and the character recognition result into the AI style classifier. The AI style classifier calls the vocabulary distribution and syntactic pattern model trained by itself to perform feature matching on the input information, and at the same time retrieves the text type identifier with the highest feature matching degree from the preset type library to output the final text type.

[0047] In this embodiment, through a hierarchical processing method of first determining the basic language type based on the coding range, then determining the character recognition result in combination with character rules and intervals, and finally accurately matching the text type through the AI style classifier, the efficient and accurate determination of the input text type is achieved; compared with the determination method that solely relies on coding or intervals, this technical solution integrates coding features, character interval features and the vocabulary and syntactic features of AI deep learning, greatly reducing the misjudgment rate of the text type, providing an accurate text type basis for the weight allocation and feature extraction during the subsequent dual - model parsing, further ensuring the accuracy of the pre - training corpus evaluation, and reducing the corpus screening deviation and subsequent waste of computing resources caused by the misjudgment of the text type.

[0048] Based on any of the above embodiments, in the third embodiment of the present application, the step of processing the input text based on the first model to obtain the first parsing result includes: Step S21, call the reward module, penalty module and compensation module included in the first model, and respectively parse the input text to obtain reward features, penalty features and compensation features, where the reward features, penalty features and compensation features constitute the triple five - dimensional basic structure of the first model.

[0049] In this embodiment, the reward module is a functional unit within the first model used to identify high-quality cognitive features in the input text. These high-quality cognitive features include five positive dimensions: language density, structural complexity, depth of thought, system penetration, and model nesting, corresponding to the "positive five-dimensional layer" in the triple five-dimensional basic structure. The penalty module is a functional unit within the first model used to identify cognitive defect features in the input text. These cognitive defect features include five negative dimensions: paradigm closure, expressive transfer vulnerability, cultural context bias, closed-mindedness, and cognitive inducement, corresponding to the "penalty five-dimensional layer" in the triple five-dimensional basic structure. The compensation module is a functional unit within the first model used to correct and supplement the cognitive value of specific text types that may be underestimated due to their inherent expressive characteristics. The basis for correction and supplementation includes five scientific enhancement dimensions: logical rigor, empirical tension, computational complexity, cross-interpretive power, and evolvability, corresponding to the "scientific enhancement five-dimensional layer" in the triple five-dimensional basic structure. The reward feature is the feature set containing quantified data of the five positive dimensions output by the reward module after parsing the input text. The penalty feature is the feature set containing quantified data of the five negative dimensions output by the penalty module after parsing the input text. The compensation features are the feature set output by the compensation module after parsing the input text, containing quantified data across five scientific enhancement dimensions. The triple five-dimensional basic structure is the core architecture of the first model, consisting of a positive five-dimensional layer, a penalized five-dimensional layer, and a scientific enhancement five-dimensional layer. Each of the three layers contains five dimensions, forming a comprehensive coverage of the cognitive features of the text.

[0050] As an optional implementation, the input text is simultaneously transmitted to the reward module, penalty module, and compensation module of the first model, and the three modules start the parsing process in parallel. The reward module uses algorithms such as keyword extraction and semantic association analysis to identify positive dimension features in the input text, such as language density (the amount of effective information contained in a unit character) and structural complexity (the number of logical nesting levels in text paragraphs). Each dimension feature is quantified into a value of 0-100 points using a preset scoring standard to form reward features. The penalty module uses algorithms such as paradigm matching and cross-cultural context verification to identify paradigm closure (the degree of dependence on a single theoretical paradigm) and cognitive induction in the input text. The negative dimension features, such as the degree to which the reader develops a biased understanding, are also quantified from 0 to 100 points (the higher the score, the more obvious the defect), forming a penalty feature. The compensation module uses algorithms such as logical chain verification and empirical data correlation analysis to identify scientific enhancement dimension features in the input text, such as logical rigor (the degree to which the argument is without loopholes) and evolvability (the degree to which the text's viewpoint can be expanded and extended), quantified from 0 to 100 points (the higher the score, the more reasonable the need for supplementation and correction), forming a compensation feature. After the three modules are analyzed, the reward feature, penalty feature, and compensation feature are output simultaneously, which together constitute the feature data of the triple five-dimensional basic structure.

[0051] Step S22: The first parsing result of the input text is generated by calculating the comprehensive score of reward features, penalty features and compensation features by using the nonlinear weighted scoring function preset by the first model and combining the dynamic function graph.

[0052] In this embodiment, the nonlinear weighted scoring function is a pre-defined mathematical function within the first model, used to fuse quantitative data from reward, punishment, and compensation features. Its characteristic is that the weights of different features are not fixed linearly related, but dynamically adjust their influence based on the feature values. The dynamic function graph is a graph stored within the first model that shows the function parameter adjustments corresponding to text type and feature value ranges, containing the weight coefficient mapping relationships for each feature dimension under different scenarios (e.g., the weight of the science enhancement dimension is higher in academic texts than in children's texts). The comprehensive score is a quantitative result reflecting the cognitive value level of the input text, calculated by substituting the quantitative data of the three features into the nonlinear weighted scoring function; the score range is typically 0-100. The first analytical result is a text cognitive value analysis report containing the comprehensive score and core evaluation conclusions for each feature dimension (e.g., "Excellent depth of thought, no obvious cognitive deficiencies").

[0053] As an optional implementation, a preset nonlinear weighted scoring function is first retrieved from the parameter library of the first model, and the function expression is set as follows:

[0054] Where S is the comprehensive score, R is the total score of the reward feature, P is the total score of the penalty feature, C is the total score of the compensation feature, f, g, and h are nonlinear transformation functions, and α, β, and γ are initial weight coefficients. Based on the text type of the input text, the corresponding weight adjustment coefficients are retrieved from the dynamic function graph, and the initial weights α, β, and γ are corrected to values ​​suitable for academic texts, such as increasing the γ value to enhance the influence of the scientific enhancement dimension in the compensation feature. The total score of the reward feature R, the total score of the penalty feature P, and the total score of the compensation feature C are calculated, and substituted into the corresponding nonlinear transformation functions to obtain the transformed values. The transformed values ​​and the corrected weight coefficients are substituted into the function to calculate the comprehensive score S. Combining the highest-scoring positive dimension in the reward feature, the lowest-scoring negative dimension in the penalty feature (i.e., the dimension with the slightest defect), and the highest-scoring scientific enhancement dimension in the compensation feature, a first analytical result containing the comprehensive score and core evaluation conclusions is generated.

[0055] As an alternative implementation method, the numerical values ​​of each dimension of the reward feature, punishment feature, and compensation feature are first standardized, mapping all dimension values ​​to the 0-1 range. Based on the dynamic function graph, the weight of each dimension in the nonlinear weighted scoring function is determined according to the text type (e.g., the weight of the "language density" dimension is reduced in children's texts, and the weight of the "model nesting" dimension is reduced). The weights of each dimension are allocated to the corresponding features (reward feature, punishment feature, compensation feature) using the analytic hierarchy process (AHP), obtaining the dynamic weights at the feature level. The standardized values ​​of each dimension are multiplied by their corresponding weights and summed to obtain the weighted score of each feature. The weighted scores of the reward feature, punishment feature (taking negative values), and compensation feature are substituted into the nonlinear fusion formula to calculate the comprehensive score. Based on the standardized values ​​and weights of each dimension, a "contribution analysis of each dimension" is generated, and combined with the comprehensive score to form the first analytical result. This method can intuitively demonstrate the degree of influence of each dimension on cognitive value assessment.

[0056] This embodiment first accurately obtains the reward, penalty, and compensation features of the text based on a triple five-dimensional basic structure, and then calculates the comprehensive score by combining a dynamic function graph and a nonlinear weighted scoring function, thus realizing a refined evaluation of the cognitive value of the text. Compared with single-dimensional or linearly weighted evaluation methods, this technical solution can more accurately reflect the true cognitive level of the text, avoid evaluation bias caused by fixed feature weights, provide a reliable cognitive dimension basis for subsequent fusion with the analysis results of the second model, further ensure the accuracy of the corpus quality evaluation before training, and reduce ineffective training and waste of computational resources caused by misjudgment of cognitive value.

[0057] Based on any of the above embodiments, in Embodiment 4 of this application, the step of processing the input text based on the second model to obtain the second parsing result includes: Step S23: Call the second model to extract features from the input text using a preset number of expression factor dimensions to obtain a feature vector; Step S24: Use the three-level vector field mapping structure of sentence, paragraph, and text built into the second model to generate a high-dimensional representation of the text.

[0058] In this embodiment, the preset number of expression factor dimensions is a set of specific evaluation indicators pre-configured in the second model to characterize the mathematical structure features of the text. The number is set according to the needs of the text parsing scenario and covers dimensions such as grammatical complexity, symbol density, and computational logic depth. Feature vectors are numerical vectors formed by extracting features from the input text through the expression factor dimensions, with each dimension corresponding to a feature quantification value. The sentence-paragraph-article three-level vector field mapping structure is a built-in mapping system in the second model used to achieve layer-by-layer aggregation of text features from basic units to the overall structure, corresponding to the feature representations at the word, sentence, and article levels, respectively. The high-dimensional text representation is a high-dimensional numerical representation formed by fusing feature vectors from each level through the three-level vector field mapping structure, comprehensively reflecting the mathematical structure features of the text. The dimensions are typically 768 to 1024.

[0059] As an optional implementation, a second model based on the Transformer architecture is first initialized, loading a preset number of expression factor dimension parameters adapted to the input text type. The input text is then segmented into words, and the segmentation results are input into the Transformer encoder of the second model. The encoder captures the semantic relationships between words through a self-attention mechanism, while activating the feature extraction function of each expression factor dimension. For each word, its quantization value in dimensions such as syntactic complexity and symbol density is calculated to form a word-level feature vector. All word-level feature vectors within the same sentence are weighted and aggregated according to semantic weights to generate sentence-level feature vectors, while preserving the logical order and dependencies of words within the sentence. Then, all sentence-level feature vectors in the entire text are hierarchically aggregated according to the discourse structure logic (such as causal and progressive relationships) to generate discourse-level feature vectors. Finally, the word-level, sentence-level, and discourse-level feature vectors are input into a three-level vector field mapping structure, and the deep fusion of the three levels of features is achieved through a vector space mapping algorithm to output a high-dimensional representation of the text.

[0060] Step S25: By combining the high-dimensional representation of the text and the dimensional features of each expression factor through a weighted integration algorithm, a second analytical result including a dimensional distribution heatmap and an expression stability spectrum is output.

[0061] In this embodiment, the weighted ensemble algorithm is used to collaboratively fuse the high-dimensional representation of the text with the dimensional features of each expression factor. It dynamically allocates weight coefficients based on the importance of different features in assessing the text's mathematical structure. The dimensional distribution heatmap is a visual chart that intuitively displays the distribution of quantitative values ​​of each expression factor's dimensional features in the form of a heatmap. Different color depths correspond to different dimensional scores, facilitating quick identification of the text's strengths and weaknesses in each mathematical dimension. The expression stability spectrum is a line graph or curve graph used to display the changing trends of the text's expression stability at the sentence, paragraph, and chapter levels. Stability is calculated through the fluctuation amplitude of feature vectors, and the spectrum reflects the consistency of the text's overall mathematical structure. The second analytical result is an analytical report containing a comprehensive score of the text's mathematical structure, the dimensional distribution heatmap, the expression stability spectrum, and the assessment conclusions of the core mathematical features, reflecting the level of the input text's mathematical structure.

[0062] It should be noted that the weight sequences corresponding to the high-dimensional representation of the text and the dimensional features of each expression factor in the weighted ensemble algorithm are associated with the text type.

[0063] As an optional implementation, the initial weight coefficients of the weighted fusion algorithm are first determined by retrieving the weight allocation rules of each expression factor dimension feature and the high-dimensional text representation from the parameter library of the second model according to the type of input text. The high-dimensional text representation is then converted into a vector form that can be directly computed in conjunction with the expression factor dimension features, and input into the weighted fusion algorithm along with the quantized values ​​of each expression factor dimension feature. The algorithm achieves collaborative fusion between features through matrix operations, adjusting the weight coefficients in real time during the calculation process. Weights are appropriately reduced for expression factor dimension features with large score fluctuations, and weights are increased for features with high stability in the high-dimensional text representation. The weights of each dimension are appropriately increased; after the fusion calculation is completed, a comprehensive score of the text's mathematical structure is generated; based on the quantified values ​​of each expression factor dimension, a visualization tool is used to generate a dimension distribution heatmap, where red areas represent high-scoring dimensions and blue areas represent low-scoring dimensions; based on the fluctuation amplitude of the feature vectors at the sentence, paragraph, and text levels, an expression stability spectrum is generated, with the horizontal axis representing the text paragraph number and the vertical axis representing the stability score; the comprehensive score, dimension distribution heatmap, expression stability spectrum, and the evaluation conclusion of "core advantage dimension: computational logic depth; dimension to be optimized: cross-semantic domain transfer capability" are integrated to form and output the second analysis result.

[0064] This embodiment generates high-dimensional representations through feature extraction based on the Transformer architecture and three-level vector field mapping. It then combines a weighted ensemble algorithm to achieve feature collaboration and visualization output, thus constructing a refined evaluation path for the mathematical structure of text. Compared with traditional single-dimensional evaluation, this scheme can capture the deep mathematical features of text through high-dimensional representations and present the evaluation results intuitively through visualization charts. This significantly improves the accuracy and readability of the mathematical dimension evaluation of text, provides accurate and easy-to-understand mathematical dimension basis for the subsequent fusion of dual-model analysis results, further reduces the corpus selection bias caused by misjudgment of mathematical features, and reduces the computational resources consumed by ineffective training.

[0065] Furthermore, the weight sequences corresponding to the high-dimensional representation of the text and the dimensional features of each expression factor in the weighted ensemble algorithm are associated with the text type.

[0066] In this embodiment, the weight sequence is a set of coefficients used in the weighted ensemble algorithm to perform weighted calculations on the high-dimensional representation of the text and the features of each expression factor dimension. Each coefficient corresponds to the weight value of a feature (the high-dimensional representation of the text or a feature of a certain expression factor dimension). The sequence as a whole reflects the relative importance of each feature in the fusion calculation. The association between text type and weight sequence means that different text types correspond to different weight sequence configurations. That is, according to the type of input text (such as academic papers, children's stories, classical texts, etc.), the weight sequence of that type is dynamically matched and adapted to ensure that the feature fusion calculation conforms to the inherent attributes of the text and the evaluation requirements.

[0067] As an optional implementation, a text type-weight sequence mapping database is pre-constructed. The database stores two sets of weight sub-sequences for each text type: one set corresponds to the weight coefficients of the high-dimensional representation of the text, and the other set corresponds to the weight coefficients of each expression factor dimension feature. For example, in the weight sequence configured for the "academic paper" type, the weight coefficients of expression factor dimension features such as "computational logic depth" and "symbol density" are relatively high (0.8-0.9), while the weight coefficient of the "rhetorical richness" dimension is relatively low (0.2-0.3), and the weight coefficient of the high-dimensional representation of the text is set to 0.5 (balancing overall and local features). In the weight sequence configured for the "children's story" type, the weight coefficients of expression factor dimension features such as "syntactic conciseness" and "emotional mapping intensity" are relatively high (0.7-0.8), while the weight coefficient of the "computational logic depth" dimension is relatively low (0.1-0.2), and the weight coefficient of the high-dimensional representation of the text is set to 0.6 (emphasizing overall expressive fluency). When the weighted integration algorithm is started, the type of the input text is first obtained, and the corresponding weight sequence is retrieved from the mapping database and directly used for the fusion calculation of the high-dimensional representation of the text and each expression factor dimension feature.

[0068] This embodiment associates the weight sequence in the weighted ensemble algorithm with the text type, enabling feature fusion calculation to adapt to the inherent attributes and evaluation needs of different text types. Compared with the fusion method using a fixed weight sequence, this scheme can avoid evaluation bias caused by the mismatch between weight configuration and text type (e.g., using the weight of academic text to evaluate children's text will underestimate its emotional value), significantly improve the pertinence and accuracy of text mathematical structure evaluation, provide a mathematical dimension basis that is more in line with the actual characteristics of the text for the final fusion of the dual-model analysis results, further ensure the accuracy of corpus selection before training, and reduce the waste of computing resources caused by evaluation bias.

[0069] Based on any of the above embodiments, in Embodiment 5 of this application, referring to Figure 2 , Figure 2 Steps S31-S33 are shown. Step S30 includes: Step S31: Based on the text type and the text length, fuse the first parsing result and the second parsing result to obtain the stable field label corresponding to the input text.

[0070] In this embodiment, the generation of stable field labels is based on text type and text length. By dynamically adjusting the fusion weight of the first and second parsing results, the overall ideological integrity and structural stability of the text can be accurately quantified.

[0071] As an optional implementation, a text type-text length-weight mapping table is pre-constructed. This mapping table stores the first model weights and second model weights corresponding to different combinations of text types and lengths. Short text types correspond to a weight configuration where the first parsing result accounts for 40% and the second parsing result accounts for 60%; standard article types correspond to a weight configuration where the first parsing result accounts for 55% and the second parsing result accounts for 45%; and long works correspond to a weight configuration where the first parsing result accounts for 65% and the second parsing result accounts for 35%. Based on the identified text type and text length, the system matches the corresponding weight parameters from the mapping table, performs a weighted summation of the global first parsing result score in the first parsing result and the global second parsing result score in the second parsing result, and the resulting quantized value is the stable field label, which ranges from 0 to 1.

[0072] Step S32: Based on the first parsing result and the second parsing result, extract the global feature package corresponding to the input text.

[0073] In this embodiment, the global feature package is a collection that integrates the core features of the text's thought dimension and the key features of its structural dimension, providing a global reference benchmark for the subsequent calculation of local features.

[0074] As an optional implementation, core features of thought, such as the core concept set, theoretical framework integrity markers, and global abstraction statistics, are extracted from the first analysis result; core features of structure, such as average symbol density, average logical depth, syntactic complexity baseline, and semantic vector stability parameters, are extracted from the second analysis result. Frequency statistics and weight ranking are performed on the core concept set, and fifty highly relevant concepts are selected to form a core concept table. Based on the global abstraction statistics and average symbol density, a statistical baseline is calculated, consisting of average concept density, abstraction standard deviation, and symbol density fluctuation threshold. Simultaneously, theoretical framework markers from the first analysis result and full-text semantic vectors generated from the sentence-paragraph-article three-level vector field mapping structure from the second analysis result are extracted. The core concept table, statistical baseline, theoretical framework markers, full-text semantic vectors, and terminology dictionary are integrated to form a global feature package.

[0075] Step S33: Determine the local features of the input text based on the global feature package and the stable field label.

[0076] In this embodiment, the determination of local features relies on the reference standard provided by the global feature package, and is also constrained and modulated by the stable field label to ensure the rationality of the paragraph-level thought intensity assessment.

[0077] As an optional implementation, the input text is split into paragraphs, and local idea features and local structural features are extracted for each paragraph. Local idea features are obtained by finding the intersection of the paragraph's concept set and the core concept table in the global feature package to obtain the core concept alignment. The ratio of the paragraph's concept density to the average concept density in the statistical baseline is calculated to obtain the relative surge, determining whether the paragraph's concepts participate in the systematic system corresponding to the theoretical framework label. Local structural features are obtained by analyzing indicators such as the paragraph's syntactic complexity, symbol density, and logical depth. The local idea features and local structural features are fused with a weight of 7:3 to obtain a local energy value. This value is then modulated based on the quantization value of the stable field label. If the stable field label value is higher than 0.8, the local energy value remains unchanged; if it is lower than 0.5, the local energy value is multiplied by a modulation coefficient of 0.7, ultimately yielding the local features for each paragraph.

[0078] This embodiment first obtains stable field labels based on dynamic text-scale fusion, establishing constraint boundaries for the overall evaluation; then, it extracts global feature packages, building a bridge between the local and the global; finally, it combines global reference and constraint modulation to determine local features, strictly adhering to the logical necessity of "global first, local later". This process achieves accurate characterization of the overall text confidence and paragraph thought intensity, ensuring the objectivity and completeness of the analysis results, and providing solid support for the subsequent generation of accurate text evaluation values.

[0079] Based on any of the above embodiments, in Embodiment Six of this application, step S31 includes: Step S311: Based on the text type and the text length, obtain a preset mapping table and determine the first model weight and the second model weight, wherein the mapping table is the mapping relationship between text type and text length to the first model weight and the second model weight.

[0080] In this embodiment, the mapping table is a pre-built structured data table that stores the correspondence between text type, text length, and the weights of the first and second models, providing a direct basis for weight determination. The first model weight is the proportion of the first parsing result in the fusion calculation, and the second model weight is the proportion of the second parsing result in the fusion calculation; the sum of the two is 1.

[0081] As an optional implementation, the text types are first categorized and labeled, including academic papers, popular science articles, business copywriting, and literary works. Text lengths are also divided into ranges, including short texts (less than 1,000 words), standard articles (1,000 to 10,000 words), and long works (more than 10,000 words). Based on the combination of text type and length, corresponding weight parameters are set. For example, in the short text scenario, academic papers correspond to a first model weight of 40% and a second model weight of 60%; in the standard article scenario, popular science articles correspond to a first model weight of 55% and a second model weight of 45%; and in the long work scenario, literary works correspond to a first model weight of 65% and a second model weight of 35%. These correspondences are organized into a mapping table and stored in the system database. When it is necessary to determine the weights, the system retrieves the mapping table in the database based on the identified text type and length, and matches the corresponding first model weight and second model weight.

[0082] Step S312: Based on the first model weights and the second model weights, fuse the first parsing result and the second parsing result to obtain the stable field label corresponding to the input text.

[0083] In this embodiment, the fusion calculation is the process of combining the first and second parsing results according to their corresponding weights. The purpose is to integrate the value of the thought dimension and the complexity of the structural dimension to obtain a quantitative result that reflects the overall confidence of the text.

[0084] As an optional implementation, the global first analysis result score from the first analysis result and the global second analysis result score from the second analysis result are extracted, ensuring that both scores are normalized to the range of 0 to 1. A weighted summation formula is used for fusion calculation: Stable field label = (First analysis result × First model weight) + (Second analysis result × Second model weight). For example, if the global first analysis result score of the first analysis result is 0.9 and the first model weight is 55%, and the global second analysis result score of the second analysis result is 0.85 and the second model weight is 45%, then the stable field label = 0.9 × 0.55 + 0.85 × 0.45 = 0.495 + 0.3825 = 0.8775, which is the stable field label corresponding to the input text.

[0085] This embodiment clarifies the weight configuration under different scenarios through a preset mapping table, ensuring the standardization and consistency of weight determination; it adopts a weighted summation fusion method to achieve a scientific integration of the value of the ideological dimension and the complexity of the structural dimension. The generated stable field label can accurately reflect the overall ideological integrity and structural stability of the text, providing a reliable global constraint benchmark for the subsequent modulation of local features.

[0086] Based on any of the above embodiments, in Embodiment Seven of this application, step S32 includes: Step S321: Extract the core features of the thought dimension from the first parsing result to obtain a subset of thought features. The first parsing result includes the global first parsing result score of the input text and the sub-values ​​of each thought dimension. The subset of thought features includes a set of core concepts, a theoretical framework integrity marker, and a global abstraction degree statistical value.

[0087] In this embodiment, the core features of the thought dimension are key features reflecting the ideological value of the input text, focusing on the degree of freedom of thought in the conceptual semantic space. The global first parsing result score is the overall quantitative result of the first model on the value of the thought dimension of the input text. The sub-values ​​of each thought dimension are individual scores of the five positive dimensions (language density, structural complexity, thought depth, system penetration, and model nesting) in the first model. The thought-class feature subset is a set that integrates core thought features, providing thought dimension support for the construction of the global feature package. The core concept set is the set of key terms in the input text that carry core thoughts. The theoretical framework integrity marker is an identifier representing whether the input text establishes a systematic thought framework. The global abstraction statistics are the quantitative statistical results of the overall abstraction level of the input text.

[0088] As an optional implementation, the global first analysis result score and the sub-values ​​of each thought dimension in the first analysis result are analyzed. The keyword extraction algorithm is used to filter out terms that are highly related to each thought dimension from the input text to form a core concept set. Based on the consistency and relevance of the sub-values ​​of each thought dimension, it is determined whether the text forms a self-consistent theoretical system and a theoretical framework integrity marker (such as "complete", "partially complete", "incomplete") is generated. The mean and distribution characteristics of the abstract level related indicators in the sub-values ​​of each thought dimension are calculated to obtain the global abstraction statistics. The core concept set, the theoretical framework integrity marker, and the global abstraction statistics are integrated to form a subset of thought-type features.

[0089] Step S322: Extract the core features of the structural dimension from the second parsing result to obtain a subset of structural features. The second parsing result includes the global second parsing result score of the input text and the sub-values ​​of each structural dimension. The subset of structural features includes the average symbol density, the average logical depth, the baseline of syntactic complexity, and the semantic vector stability parameter.

[0090] In this embodiment, the core features of the structural dimension are key features reflecting the structural complexity of the input text, focusing on the difficulty of organizing expressions in the structural form space. The global second parsing result score is the overall quantitative result of the second model on the structural dimension complexity of the input text. The sub-values ​​of each structural dimension are individual scores for each expression factor dimension in the second model. The structural class feature subset is a set that integrates core structural features, providing structural dimension support for the construction of the global feature package. The average symbol density is the average quantitative value of information carried per unit length of the input text. The average logical depth is the average statistical result of the number of reasoning steps in the input text. The syntactic complexity baseline is a benchmark value characterizing the syntactic structural complexity of the input text. The semantic vector stability parameter is a parameter reflecting the degree of anchoring of the input text in the semantic space.

[0091] As an optional implementation, the global second parsing result score and the sub-items of each structural dimension in the second parsing result are analyzed. All sub-items of the symbol density dimension are extracted, and their average value is calculated to obtain the average symbol density. All sub-items of the logical depth dimension are summarized and their average value is taken to obtain the logical depth mean. The lowest effective score of the syntactic complexity dimension is used as the benchmark, and the syntactic complexity baseline is obtained by combining it with text length correction. The fluctuation data of the semantic vector stability dimension are analyzed, and key parameters such as variance and standard deviation are extracted to form the semantic vector stability parameter. The average symbol density, the logical depth mean, the syntactic complexity baseline, and the semantic vector stability parameter are integrated to form a structural feature subset.

[0092] Step S323: Perform frequency statistics and weight ranking on the core concept set in the subset of thought features, and select a preset number of highly related concepts to form a core concept table. The ranking weight is determined by weighting the contribution value of each concept in each thought dimension.

[0093] In this embodiment, frequency statistics are the process of counting the number of times each concept appears in the core concept set. Weighted sorting is the operation of prioritizing core concepts according to preset weights. The preset number is the number of concepts included in the core concept table pre-defined by the system (usually fifty). Highly relevant concepts are those that are highly related to the core ideas of the input text. The core concept table is a structured table that integrates high-priority core concepts. The contribution value is a quantitative value of the degree to which each concept supports the value of each dimension of the idea.

[0094] As an optional implementation, the core concept set in the subset of thought features is traversed, and the frequency of each concept in the input text is counted. Based on the weight of each thought dimension (each accounting for 20% in the initial stage), the contribution value of each concept in five thought dimensions such as language density and structural complexity is calculated, and the total contribution value of each concept is obtained by weighted summation. The concepts are sorted in descending order according to the comprehensive score of "frequency of occurrence × total contribution value". The top 50 highly relevant concepts are selected and organized into a core concept table according to the score from high to low.

[0095] Step S324: Based on the global abstraction statistics in the thought-type feature subset and the average symbol density in the structure-type feature subset, a statistical baseline is calculated. The statistical baseline includes the average concept density, the standard deviation of abstraction, and the symbol density fluctuation threshold.

[0096] In this embodiment, the statistical baseline is a set of benchmark data used for subsequent local feature comparison, providing a reference standard for paragraph-level feature evaluation. The average concept density is the average distribution density of core concepts in the input text. The abstraction standard deviation is a quantification of the dispersion of the global abstraction statistics. The symbol density fluctuation threshold is the maximum range boundary value that allows fluctuations in the average symbol density.

[0097] As an optional implementation, the total number of paragraphs in the input text is counted, and the average concept density is obtained by dividing the total number of concepts in the core concept set by the total number of paragraphs. The deviation of each data point in the global abstraction statistic from the mean is calculated, and the standard deviation of abstraction is obtained by using the standard deviation formula. Based on the average symbol density, combined with the fluctuation coefficient corresponding to the text type (0.15 for academic papers, 0.2 for popular science, etc.), the symbol density fluctuation threshold (average symbol density × (1 ± fluctuation coefficient)) is calculated. The average concept density, the standard deviation of abstraction, and the symbol density fluctuation threshold are integrated to form a statistical baseline.

[0098] Step S325: Extract theoretical framework markers from the first parsing result and extract full-text semantic vectors from the second parsing result. The full-text semantic vectors are generated by a sentence-paragraph-article three-level vector field mapping structure.

[0099] In this embodiment, the theoretical framework marker is a specific identifier representing the systematic nature of the input text's thought framework, and it is more detailed than the theoretical framework integrity marker. The full-text semantic vector is a vectorized representation of the overall semantics of the input text, reflecting the core semantic direction of the text. The sentence-paragraph-article three-level vector field mapping structure is the hierarchical structure used in the second model to generate semantic vectors, gradually constructing semantic representations from the sentence, paragraph, and article levels.

[0100] As an optional implementation, theoretical framework markers are extracted from the first analysis result. These markers contain detailed information such as the core logic of the thought framework, the way concepts are related, and the closed loop of the argument. The full-text semantic vector generated by the sentence-paragraph-article three-level vector field mapping structure is called from the second analysis result. This vector first generates semantic vectors for individual words through a word vector model, then integrates word vectors into sentence vectors through a sentence vector fusion algorithm, then aggregates sentence vectors into paragraph semantic vectors through a paragraph vector aggregation algorithm, and finally aggregates paragraph vectors into full-text semantic vectors through an article vector aggregation algorithm, thus fully representing the semantic features of the input text.

[0101] For example, a user submits a 30,000-word long text, and the system obtains a first analysis result and a second analysis result after the preceding steps. From the first analysis result, the system extracts the global first analysis result score (0.91) and the sub-values ​​of each ideological dimension, selecting core concepts such as "cognitive revolution" and "cultural evolution" to form a core concept set. The system determines that the text forms a complete theoretical framework, generating a theoretical framework integrity marker of "complete," and calculates a global abstraction statistical value of 0.85, integrating these to form an ideological feature subset. From the second analysis result, the system extracts the global second analysis result score (0.88) and the sub-values ​​of each structural dimension, calculating the average symbol density (0.76), the average logical depth (0.82), and the baseline syntactic complexity (0.65). Semantic vector stability parameters (variance 0.08, standard deviation 0.28) are extracted to form a structural feature subset. The system performs frequency statistics and contribution value calculations on the core concept set, sorting and selecting fifty highly relevant concepts according to their comprehensive scores to form a core concept table. The average concept density was calculated to be 3.2, the standard deviation of abstraction was 0.12, and the symbol density fluctuation threshold was (0.656-0.864), forming a statistical baseline. Theoretical framework markers containing core logic and argumentation loop information were extracted from the first analysis result, and full-text semantic vectors generated by the sentence-paragraph-article three-level vector field mapping structure were extracted from the second analysis result.

[0102] This embodiment extracts core features from the ideological and structural dimensions step by step, and systematically constructs a global feature package basic component that includes a core concept table, statistical baseline, theoretical framework markers, and full-text semantic vectors. Each step is progressive, ensuring that the global feature package can comprehensively cover the core ideas and key structural information of the text. This provides an accurate and reliable global reference benchmark for the subsequent calculation of local features, ensuring the effective implementation of the "global first, local later" calculation logic and improving the objectivity and completeness of the text parsing results.

[0103] Based on any of the above embodiments, in Embodiment 8 of this application, step S33 includes: Step S331: Based on the global feature package, determine the first local features of each paragraph in the input text, and the second local features of each paragraph compared to the full text of the input text.

[0104] In this embodiment, the first local feature is the core feature of the paragraph itself in terms of its thought and structure, reflecting the paragraph's independent expressive value. The second local feature is the quantified result of the paragraph's feature differences and correlations relative to the whole text, reflecting the paragraph's position and contribution in the overall text. The global feature package provides a unified reference standard for the calculation of the two types of local features, ensuring consistency in the evaluation.

[0105] As an optional implementation, the input text is split into paragraphs, and local idea features and local structural features are extracted from each paragraph: local idea features are obtained by identifying the core concepts, abstraction levels, and participation of theoretical frameworks in the paragraph; local structural features are obtained by analyzing the syntactic complexity, symbol density, and logical depth of the paragraph. The two are integrated into the first local feature. At the same time, the intersection ratio between the paragraph concept set and the global core concept table is calculated to obtain the core concept alignment; the ratio of the paragraph concept density to the average concept density in the statistical baseline is calculated to obtain the relative surge (a ratio greater than 1.5 indicates a significant surge); the similarity between the paragraph vector and the semantic vector of the whole text is calculated to obtain the semantic relevance. The core concept alignment, relative surge, and semantic relevance are integrated into the second local feature.

[0106] Step S332: Merge the first local features and the second local features to determine the third local features of each paragraph.

[0107] In this embodiment, the third local feature is a quantitative result that integrates the intrinsic value of a paragraph with its overall relevance, representing a preliminary integrated assessment of the paragraph's expressive energy. The fusion process must balance paragraph independence with overall harmony to ensure a comprehensive assessment.

[0108] As an optional implementation, a weighted fusion algorithm is used to determine the fusion weights of the first and second local features based on the text type: for academic papers and long-form works, the first local feature accounts for 60% of the weight, and the second local feature accounts for 40%; for popular science and commercial copywriting texts, the weights are each 50%. The quantized values ​​of the first and second local features are weighted and summed according to their corresponding weights to obtain the third local feature, which has a value range of 0 to 1.

[0109] Step S333: Modulate the third local feature based on the stable field label to obtain the local features of each paragraph in the input text.

[0110] In this embodiment, modulation refers to the process of rationality correction of the third local feature using the stable field label as a constraint benchmark. The local feature is a quantitative indicator that ultimately reflects the intensity of the paragraph's ideas after being constrained by global confidence, corresponding to the expressive energy at the paragraph level.

[0111] As an optional implementation, the quantized value of the stable field label is extracted, and modulation rules are set: if the stable field label value is ≥0.8, it indicates that the overall confidence of the text is high, and the third local feature remains unchanged; if 0.5≤stable field label value<0.8, the third local feature is multiplied by a modulation coefficient of 0.9; if the stable field label value<0.5, it indicates that the overall confidence of the text is low, and the third local feature is multiplied by a modulation coefficient of 0.7. The result after correction by this rule is the final local feature of each paragraph.

[0112] For example, a user submits a 15,000-word academic paper. The system generates a global feature package and a stable field label (quantization value 0.86) through previous steps. In step S331, the text is divided into 92 paragraphs. Three core concepts are extracted from paragraph 35, which has a high level of abstraction and moderate syntactic complexity, forming the first local feature (quantization value 0.82). The alignment of the core concepts in this paragraph is calculated to be 0.75, the relative spurious increase is 1.8, and the semantic relevance is 0.91, which are then integrated into the second local feature (quantization value 0.83). In step S332, according to the weighting of academic papers, the third local feature is calculated as 0.82 × 0.6 + 0.83 × 0.4 = 0.824. In step S333, because the stable field label 0.86 ≥ 0.8, the third local feature remains unchanged, and the final local feature value for this paragraph is 0.824.

[0113] This embodiment first calculates the paragraph's own features and relative features, then performs weighted fusion, and finally modulates with a global stable field label, strictly adhering to the core logic of "local dependence on global". The generated local features not only reflect the independent value of the paragraph but also take into account the constraints of the overall text, accurately depicting the intensity of thought at the paragraph level. This provides reliable basic data for the subsequent generation of text evaluation values ​​and the construction of entropy landscape, ensuring the objectivity and hierarchy of text parsing.

[0114] Based on any of the above embodiments, in Embodiment Nine of this application, step S40 includes: Step S401: Obtain the association information of the input text. The association information includes the application identifier of the input text and the fuzzy semantics of the text. The fuzzy semantics of the text are obtained by comparing the full-text semantic vector output by the second model with the preset semantic cluster center. The application identifier is determined by parsing the upload interface or file format of the input text.

[0115] In this embodiment, the association information is supplementary information that helps determine the application scenario of the input text, providing multi-dimensional basis for scenario label matching. The application identifier is a unique identifier of the application to which the input text belongs, used to distinguish the source scenario of the text. The textual fuzzy semantics is a general description of the core theme and expressive intent of the input text, reflecting the content direction of the text. The full-text semantic vector is generated by the second model through a sentence-paragraph-article three-level vector field mapping structure, and is vector data that can completely represent the semantic features of the text. The preset semantic clustering centers are semantic benchmark vectors pre-trained by the system, covering various common topics. The upload interface is the system interface used when the user submits the input text, and the file format is the storage format type of the input text.

[0116] As an optional implementation, the system identifies the application identifier by detecting the submission channel of the input text, identifying the corresponding upload interface, or parsing information such as the file extension and encoding format of the text; it calls the full-text semantic vector output by the second model, calculates the cosine similarity between the vector and the preset semantic clustering center, selects the topic corresponding to the clustering center with the highest similarity as the text fuzzy semantics; and integrates the application identifier and the text fuzzy semantics to form the association information of the input text.

[0117] Step S402: Combining the text type of the input text, the application identifier, and the fuzzy semantics of the text, match the preset scene tag library to determine the scene tag of the input text.

[0118] In this embodiment, the preset scene tag library is a structured database pre-built by the system, storing the correspondence between text type, application identifier, semantic topic, and scene tag. Scene tags are precise annotations of the actual application scenario of the input text, used to adapt to the corresponding text evaluation value mapping rules.

[0119] As an optional implementation, the preset scenario tag library includes various tags such as academic paper publication scenarios, popular science text dissemination scenarios, commercial copywriting optimization scenarios, and literary creation evaluation scenarios. Each tag corresponds to a combination of conditions for text type, application identifier, and semantic theme. The system uses the text type, application identifier, and fuzzy semantics of the input text as query conditions and performs precise matching in the preset scenario tag library. If all the combination conditions of a certain scenario tag are met, then that tag is determined as the scenario tag of the input text; if multiple partially matching tags exist, the tag with the most matching dimensions is selected as the scenario tag.

[0120] Step S403: Extract the global confidence quantification value corresponding to the stable field label, as well as the paragraph Q-value sequence, entropy peak ratio, and average Q-value contained in the local features, wherein the entropy peak ratio is the ratio of the number of paragraphs with Q-values ​​greater than a preset threshold to the total number of paragraphs.

[0121] In this embodiment, the global confidence quantification is the specific numerical representation of the stable field label, ranging from 0 to 1, reflecting the overall ideological integrity and structural stability of the text. The paragraph Q-value sequence is a sequence of local feature values ​​from all paragraphs in the input text, arranged in paragraph order, recording the ideological intensity of each paragraph. The entropy peak proportion is an indicator measuring the distribution of paragraphs with peak ideological content, reflecting the concentration of the text's core value. The average Q-value is the arithmetic mean of the paragraph Q-value sequences, reflecting the overall average ideological intensity level of the text. The preset threshold is a critical value pre-set by the system for judging entropy peaks, typically set to 0.85.

[0122] As an optional implementation, the corresponding global confidence quantification is directly extracted from the stable field label; the Q value of each paragraph is extracted from the local features and arranged by paragraph number to form a paragraph Q value sequence; the number of paragraphs in the paragraph Q value sequence that are greater than a preset threshold of 0.85 is counted, and the entropy peak ratio is obtained by dividing this number by the total number of paragraphs in the input text; the average Q value is obtained by calculating the sum of all values ​​in the paragraph Q value sequence and dividing it by the total number of paragraphs.

[0123] Step S404: Call the corresponding text evaluation value mapping rule according to the scene label. The mapping rule is preset with weighting coefficients for global confidence quantification, entropy peak ratio and average Q value under different scenes.

[0124] In this embodiment, the text evaluation value mapping rule is a set of rules preset by the system for calculating text evaluation values, and different rule configurations correspond to different scene labels. The weighting coefficient is a proportional parameter set in the mapping rule to adjust the importance of the global confidence quantification value, entropy peak ratio, and average Q value in the calculation, and the sum of the weighting coefficients of each parameter is 1.

[0125] As an optional implementation, the system pre-configures corresponding weighting coefficients for different scenario labels: in the academic paper publication scenario, the global confidence quantification value has a weight of 40%, the entropy peak percentage has a weight of 35%, and the average Q value has a weight of 25%; in the popular science text dissemination scenario, the global confidence quantification value has a weight of 30%, the entropy peak percentage has a weight of 25%, and the average Q value has a weight of 45%; in the business copywriting optimization scenario, the global confidence quantification value has a weight of 35%, the entropy peak percentage has a weight of 30%, and the average Q value has a weight of 35%; and in the literary creation evaluation scenario, the global confidence quantification value has a weight of 25%, the entropy peak percentage has a weight of 40%, and the average Q value has a weight of 35%. Based on the determined scenario label, the corresponding weighting coefficient configuration is invoked as the text evaluation value mapping rule.

[0126] Step S405: Based on the weighting coefficients of the mapping rule, calculate the weighted sum of the global confidence quantification, entropy peak ratio, and average Q value to obtain the original score.

[0127] In this embodiment, the raw score is an intermediate quantitative result that combines global confidence and local thought strength, with a value ranging from 0 to 1, and serves as the basic data for generating text evaluation values. The weighted sum calculation is the process of multiplying each indicator value by its corresponding weighting coefficient and then summing the results, used to integrate multi-dimensional evaluation indicators.

[0128] As an optional implementation, a weighted summation formula is used to calculate the original score: Original Score = (Global Confidence Quantification × Global Confidence Weighting Coefficient) + (Entropy Peak Proportion × Entropy Peak Proportion Weighting Coefficient) + (Average Q Value × Average Q Value Weighting Coefficient). For example, if the scenario label is academic paper publication scenario, the global confidence quantification is 0.9, the entropy peak proportion is 0.3, and the average Q value is 0.8, then the original score = 0.9 × 0.4 + 0.3 × 0.35 + 0.8 × 0.25 = 0.36 + 0.105 + 0.2 = 0.665.

[0129] Step S406: The original score is converted into a text evaluation value of the input text within a preset range using a linear mapping formula.

[0130] In this embodiment, the linear mapping formula is a mathematical formula that transforms the original score from the 0-1 range to a preset evaluation range, ensuring that the text evaluation value meets the requirements of intuitive evaluation. The preset range is the range of text evaluation values ​​set by the system, usually an integer range from 1 to 200, with higher values ​​representing higher cognitive value of the text. The text evaluation value is the final quantitative result reflecting the overall cognitive value of the input text, making it easy for users to quickly understand the text quality level.

[0131] As an optional implementation, the preset range is set to 1 to 200, and a linear mapping formula is used: Text evaluation value = round(original score × 199 + 1), where round() is a rounding function. The original score obtained in step S405 is substituted into the formula to calculate the corresponding integer result, which is the text evaluation value of the input text. For example, if the original score is 0.665, then the text evaluation value = round(0.665 × 199 + 1) = round(132.335) = 132.

[0132] This embodiment accurately matches scene tags with multi-dimensional related information, integrates global and local evaluation indicators based on scene-based mapping rules, and finally generates a quantified text evaluation value. The entire process achieves scene adaptability and objectivity in text cognitive value assessment, avoids the limitations of a single standard evaluation, and the text evaluation value can intuitively reflect the core value level of the text in different scenarios, providing users with targeted quality reference.

[0133] This application provides a dual-engine-based text parsing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to execute the dual-engine-based text parsing method in Embodiment 1 above.

[0134] The following is for reference. Figure 3 This document illustrates a structural diagram of a dual-engine-based text parsing device suitable for implementing embodiments of this application. The dual-engine-based text parsing device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablets, and in-vehicle terminals, as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The dual-engine-based text parsing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0135] like Figure 3As shown, the dual-engine-based text parsing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1002 or programs loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the dual-engine-based text parsing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the dual-engine-based text parsing device to communicate wirelessly or wiredly with other devices to exchange data. Although a dual-engine-based text parsing device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0136] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0137] The dual-engine text parsing device provided in this application, employing the dual-engine text parsing method described in the above embodiments, solves the technical problem of wasted computational resources caused by the fact that training corpora can only be evaluated for quality after model training is completed. Compared with the prior art, the beneficial effects of the dual-engine text parsing device provided in this application are the same as those of the dual-engine text parsing device provided in the above embodiments, and other technical features of this dual-engine text parsing device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0138] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0139] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0140] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the dual-engine-based text parsing method in the above embodiments.

[0141] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), or any suitable combination thereof.

[0142] The aforementioned computer-readable storage medium may be included in a dual-engine text parsing device; or it may exist independently and not assembled into a dual-engine text parsing device.

[0143] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a dual-engine-based text parsing device, cause the dual-engine-based text parsing device to: determine the text type of the input text based on the character spacing and binary encoding value of the input text in response to input text; process the input text based on a first model to obtain a first parsing result, and process the input text based on a second model to obtain a second parsing result, wherein the first model includes a reward module, a penalty module, and a compensation module, and the second model includes a preset number of expression factor dimensions; and fuse the first parsing result and the second parsing result according to the model weights determined by the text type to generate a text evaluation value for the input text.

[0144] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0146] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0147] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the aforementioned dual-engine-based text parsing method. This solves the technical problem that the quality of training corpora can only be evaluated after model training is complete, leading to a waste of computational resources. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the dual-engine-based text parsing method provided in the above embodiments, and will not be repeated here.

[0148] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the dual-engine-based text parsing method described above.

[0149] The computer program product provided in this application can solve the technical problem that the quality of training corpora can only be evaluated after model training is completed, resulting in a waste of computing resources. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the dual-engine-based text parsing method provided in the above embodiments, and will not be repeated here.

[0150] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A text parsing method based on a dual-engine architecture, characterized in that, The dual-engine-based text parsing method includes: In response to the input text, the text type and text length of the input text are determined based on the character spacing and binary encoding value of the input text; The input text is processed based on a first model to obtain a first parsing result, and the input text is processed based on a second model to obtain a second parsing result. The first model includes a reward module, a penalty module, and a compensation module, and the second model includes a preset number of expression factor dimensions. Based on the text type and the text length, the first parsing result and the second parsing result are fused to generate a stable field label and local features corresponding to the input text; The text evaluation value of the input text is generated based on the stable field label and local features.

2. The text parsing method based on a dual-engine as described in claim 1, characterized in that, The step of processing the input text based on the first model to obtain the first parsing result includes: The reward module, penalty module, and compensation module included in the first model are invoked to parse the input text and obtain reward features, penalty features, and compensation features, respectively. The first parsing result of the input text is generated by calculating the comprehensive score of reward features, penalty features and compensation features by combining the nonlinear weighted scoring function preset by the first model with the dynamic function graph. The reward features, penalty features and compensation features constitute the three-fold five-dimensional basic structure of the first model.

3. The text parsing method based on a dual-engine as described in claim 1, characterized in that, The step of processing the input text based on the second model to obtain the second parsing result includes: The second model is invoked to extract features from the input text using a preset number of expression factor dimensions, resulting in a feature vector. The high-dimensional representation of the text is generated by using the three-level vector field mapping structure of sentence, paragraph, and text built into the second model; By combining the high-dimensional representation of the text with the dimensional features of each expression factor through a weighted ensemble algorithm, a second analytical result is output, which includes a dimensional distribution heatmap and an expression stability spectrum diagram.

4. The text parsing method based on a dual-engine as described in claim 1, characterized in that, The step of fusing the first parsing result and the second parsing result based on the text type and the text length to generate a stable field label and local features corresponding to the input text includes: Based on the text type and the text length, the first parsing result and the second parsing result are fused to obtain the stable field label corresponding to the input text; Based on the first and second parsing results, extract the global feature package corresponding to the input text; The local features of the input text are determined based on the global feature package and the stable field label.

5. The text parsing method based on a dual-engine as described in claim 4, characterized in that, The step of fusing the first parsing result and the second parsing result according to the text type and the text length to obtain the stable field label corresponding to the input text includes: Based on the text type and the text length, a preset mapping table is obtained to determine the first model weight and the second model weight. The mapping table represents the mapping relationship between text type and text length to the first model weight and the second model weight. The stable field label corresponding to the input text is obtained by fusing the first analysis result and the second analysis result with the first model weight and the second model weight.

6. The text parsing method based on a dual-engine as described in claim 4, characterized in that, The step of extracting the global feature package corresponding to the input text based on the first parsing result and the second parsing result includes: The core features of the thought dimension are extracted from the first analysis result to obtain a subset of thought features. The first analysis result includes the global first analysis result score of the input text and the sub-values ​​of each thought dimension. The subset of thought features includes a set of core concepts, a theoretical framework integrity marker, and a global abstraction degree statistical value. Extract the core features of the structural dimension from the second parsing result to obtain a subset of structural features. The second parsing result includes the global second parsing result score of the input text and the sub-values ​​of each structural dimension. The subset of structural features includes the average symbol density, the average logical depth, the baseline of syntactic complexity, and the semantic vector stability parameter. Frequency statistics and weight ranking are performed on the core concept set in the subset of thought features. A preset number of highly related concepts are selected to form a core concept table. The ranking weight is determined by weighting the contribution value of each concept in each thought dimension. Based on the global abstraction statistics of the thought-type feature subset and the average symbol density of the structure-type feature subset, a statistical baseline is calculated, which includes the average concept density, the standard deviation of abstraction, and the symbol density fluctuation threshold. Theoretical framework markers are extracted from the first parsing result, and full-text semantic vectors are extracted from the second parsing result. The full-text semantic vectors are generated by a three-level vector field mapping structure of sentence-paragraph-article.

7. The text parsing method based on a dual-engine as described in claim 4, characterized in that, The step of determining the local features of the input text based on the global feature package and the stable field label includes: Based on the global feature package, the first local features of each paragraph in the input text and the second local features of each paragraph compared to the entire input text are determined. The third local features of each paragraph are determined by combining the first local features and the second local features; Based on the stable field label modulation of the third local feature, the local features of each paragraph in the input text are obtained.

8. The text parsing method based on a dual-engine as described in claim 1, characterized in that, The step of generating a text evaluation value for the input text based on the stable field label and local features includes: Obtain the association information of the input text, which includes the application identifier of the input text and the fuzzy semantics of the text. The fuzzy semantics of the text are obtained by comparing the full-text semantic vector output by the second model with the preset semantic clustering center. The application identifier is determined by parsing the upload interface or file format of the input text. By combining the text type of the input text, the application identifier, and the fuzzy semantics of the text, a preset scene tag library is matched to determine the scene tag of the input text; Extract the global confidence quantification value corresponding to the stable field label, as well as the paragraph Q value sequence, entropy peak ratio, and average Q value contained in the local features, wherein the entropy peak ratio is the ratio of the number of paragraphs with Q values ​​greater than a preset threshold to the total number of paragraphs; The corresponding text evaluation value mapping rule is invoked according to the scene label. The mapping rule is preset with weighting coefficients for global confidence quantification, entropy peak ratio, and average Q value under different scenes. Based on the weighting coefficients of the mapping rules, the weighted sum of the global confidence quantification, entropy peak proportion, and average Q value is calculated to obtain the original score; The original score is converted into a text evaluation value of the input text within a preset range using a linear mapping formula.

9. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the dual-engine-based text parsing method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the dual-engine-based text parsing method as described in any one of claims 1 to 8.

Images