Digital semantic understanding and conversion system and method based on multi-modal feature fusion

Abstract

The invention discloses a digital semantic understanding and conversion system and method based on multi-modal feature fusion, the contribution degrees of three types of features including semantics, mathematical and context in different digital type recognition are displayed through a multi-modal feature importance thermodynamic diagram, a macroscopic-microscopic double-layer classification structure is adopted, error propagation is effectively blocked, and the recognition efficiency is improved. Through three-way cooperation of a rule engine, a statistical model and a semantic engine, high-robustness decision making is realized, and an intelligent conversion routing and grammar adaptation mechanism is developed for a digital semantic conversion link. The method can effectively solve the technical bottlenecks of insufficient accuracy, poor expandability, strong training data dependence and the like of a traditional method in complex number type recognition, and is suitable for multiple application scenes such as TTS voice generation, natural language generation, financial document processing, education science and technology and the like; accurate conversion from digital character strings to semantic expressions conforming to human language habits can be achieved.

Description

Technical Field

[0001] The present invention belongs to the cross - technical field of natural language processing and machine learning, and particularly relates to a digital semantic understanding and conversion system and method based on multi - modal feature fusion. Background Art

[0002] Currently, digital semantic understanding technologies mainly use rule - based regular matching or single machine learning models for processing. The regular matching method identifies specific digital formats (such as phone numbers, currency amounts, etc.) through predefined patterns, but it cannot adapt to undefined new patterns and is difficult to distinguish different types of numbers with similar semantics (e.g., "2023" may be a year or an ordinary number). The single machine learning model method (such as a BERT - based classifier) has certain generalization ability, but when facing more than 20 types of numbers, there are obvious error propagation problems, the fine - grain classification accuracy drops significantly, and in some individual scenarios, due to context relationships, 1 may be read as "yao" or "one", and 2 may also be read as "liang" or "two".

[0003] The existing technologies have the following core defects: Multi - classification confusion problem: When the number of digital types exceeds 20, the decision boundaries of traditional multi - classification models become highly complex, and similar types (such as "temperature" and "angle", "phone" and "code") are extremely likely to be confused.

[0004] Training data bottleneck: The labeled data for specific digital types (such as coordinates, version numbers, events) is scarce, and it is difficult for the model to cover long - tail scenarios.

[0005] Decision non - interpretability: The end - to - end model lacks a transparent decision path and cannot trace the error cause and make timely supplements.

[0006] Insufficient context understanding: Existing methods are difficult to dynamically adapt to the semantic changes of numbers in different contexts (e.g., "1:30" may be time or ratio).

[0007] For example, in the financial text "Payment amount: ￥1,000.00", existing methods may misidentify the comma separator as an IP address separator; in the technical text "Version number: 3.2.1", the dot may be misjudged as a decimal point; in multiple event texts containing numbers, where the reading should be for events such as "jiusan" but may be misidentified as "jiushisan", etc. These errors all stem from the inherent defects of the technical architecture. Summary of the Invention

[0008] The purpose of this invention is to provide a digital semantic understanding and conversion system and method based on multimodal feature fusion, which can effectively solve the technical bottlenecks of traditional methods such as insufficient accuracy, poor scalability and strong dependence on training data in the recognition of complex number types. It is applicable to multiple application scenarios such as TTS speech generation, natural language generation, financial document processing, and educational technology, and can achieve accurate conversion from digital strings to semantic expressions that conform to human language habits.

[0009] To achieve the above objectives, this invention provides a digital semantic understanding and conversion method based on multimodal feature fusion, comprising the following steps: S10: Input the raw text and perform encoding standardization and noise filtering; S20: Employs a multi-pattern regular expression engine to identify all numbers and related patterns in the text, and applies a non-overlapping matching optimization algorithm to resolve multi-pattern conflicts; S30: Convert the numeric string identified in S20 into a structured object; S40: Collaboratively extract semantic features, mathematical features, and contextual features from digital instances, and generate a unified fusion feature vector using a dynamic weighted fusion algorithm; S50: Uses a macro-micro hierarchical classification statistical model engine to infer the type of numbers, including using a macro classifier to determine which major category a number belongs to, such as mathematical, time, quantitative, identifier, financial, or special categories, and then using the corresponding micro classifier for fine-grained classification. It also applies a hierarchical joint probability model to calculate the final probability, introduces a confidence propagation mechanism, and integrates macro and micro confidence to produce reliable decision-making basis. S60: High-confidence digital patterns are directly routed to the corresponding converter via the rules engine; S70: Construct semantic field vectors using contextual keywords to enhance the confidence of type judgment; S80: The final decision is made by combining the outputs of the rule engine, statistical model and semantic engine, using a multi-engine dynamic weight allocation algorithm. S90: Output the final type label based on the final decision, and select the corresponding digital converter based on the final type label; S100: Calls the conversion method of the selected converter to generate a number reading method that conforms to Chinese reading habits; S110: Seamlessly integrate the conversion result back into the original text position, preserving the original grammatical structure; S120: Output the final transformed text and interpretable decision log.

[0010] As a further aspect of the present invention: the specific process of S40 includes: Semantic feature extraction: The BERT model is used to extract semantic embedding vectors of the context surrounding the digit, and a dynamic window adjustment algorithm is applied to determine the optimal context range. The formula for the dynamic window adjustment algorithm is as follows: ; in, This represents the optimal dynamic window size for extracting the semantic embedding vector of the context surrounding the digit. Indicates the base window size. Represents the numerical complexity score. Indicates the degree of contextual ambiguity, where α and β are adjustment parameters; Extracting mathematical features: including format features, numerical features, mathematical attributes, and special pattern features. Format features include symbols, separators, and number length distribution. Numerical features include logarithmic transformation, numerical range classification, and special value detection. Mathematical attributes include the approximation degree of mathematical constants and power features. Special patterns include repetition patterns, sequence patterns, and symmetry detection. Extracting contextual features: including extracting prefix and suffix keywords and analyzing sentence structure; Feature fusion: A dynamic weighted fusion algorithm is used, expressed by the formula: ; Weight The calculation formula is: ; in, This represents the comprehensive feature vector obtained by dynamic weighted fusion after standardization of multiple feature classes. Represents the feature vector of the i-th class. Represents the feature standardization function, H represents the feature quality score. i α represents the historical accuracy of features. i and β i These are learnable parameters, which are optimized together with the classifier parameters using the gradient descent algorithm during the model training phase.

[0011] As a further aspect of this invention: the S50 hierarchical classification statistical model engine employs a zero-shot training mechanism for digit type inference, generating training data through the Jinja2 template engine, specifically including: Template definition phase: Design multiple text templates for each number type, and define the formatting rules and context variation rules for number variables; Variable instantiation: Generate a large number of number instances according to formatting rules; Contextual variation: Applying contextual variation rules to generate diverse text environments; Quality control: Use rules to check the reasonableness and diversity of the generated data; Automatic annotation: Automatically generates annotation information based on template metadata; The formula for calculating the hierarchical joint probability model is: ; in, This represents the comprehensive features of the input digit to be classified and its context, i.e., the comprehensive feature vector in S40. , Macro-level classification labels representing numbers. Micro-level classification labels representing numbers, Indicates based on input features Inferring that the number belongs to a certain macro category The posterior probability, Indicates a known macro-classification and input features Under the given conditions, infer that the number belongs to a specific micro-category within this macro-category. The posterior probability, This indicates that the final deduction leads to the conclusion that the number belongs to a certain micro-category. The joint posterior probability, which is the core probability output of this type of inference; The confidence propagation mechanism formula is expressed as: ; in, This represents the final comprehensive classification confidence score output by the hierarchical classification statistical model engine, which integrates the confidence scores of macro-classification and micro-classification. Indicates the confidence level for macro-classification. Indicates the confidence level of micro-classification. For the weight parameters, satisfying .

[0012] As a further aspect of the present invention: the rule engine in S60 includes a rule priority management algorithm and a rule conflict resolution mechanism, wherein the formula for the rule priority management algorithm is: ; in, This represents the historical accuracy of the rule, Coverage(R) represents the range of numeric types covered by the rule, and Conflict(R) represents the number of rule conflicts. These are weight parameters; The rule conflict resolution mechanism includes the following decision-making process: 1) Priority comparison: Select the rule with higher priority; 2) Specificity comparison: Selecting more specific rules for the pattern; 3) Context adaptability: Select the rule that is more suitable for the current context.

[0013] As a further aspect of the present invention: the formula for the multi-engine dynamic weight allocation algorithm in S80 is expressed as follows: ; in, This represents the dynamically allocated weights of the i-th engine (rule engine, statistical model engine, semantic engine, corresponding to j=1,2,3 in the formula). A i This indicates the engine's historical accuracy. C i This indicates the current confidence level of the corresponding engine. S i This represents the scene adaptability score, where α, β, and γ are adjustment parameters. The final decision is a weighted vote, expressed by the formula: ; in, This represents the final decision result of the number type obtained after dynamic weighted voting by multiple engines, i.e., the final determined number type label. This represents the set of all candidate numeric type labels, where i represents the engine index. This represents the dynamic weight allocation for the i-th engine. This represents the comprehensive features of the input digit to be processed and its context, i.e., the comprehensive feature vector. , This indicates that the i-th engine is based on input features. Infer the posterior probability that the number belongs to type y. This indicates that the type y that maximizes the subsequent summation result is selected, meaning that the type with the highest joint support from the three engines is chosen as the final decision through a weighted voting method.

[0014] As a further aspect of the present invention: the interpretability decision log in S120 includes the following: The raw string of numbers and its position in the text; The structured parsing results include the extracted numeric values, units, and context window; Detailed results of multimodal feature extraction; Probability distributions and confidence levels of macro- and micro-classifications; Matching results from the rules engine; Semantic field vectors and their contribution to decision-making; Weight allocation for each decision engine and the rationale for the final decision; Final type label and conversion result; Process timestamps and system resource usage.

[0015] To achieve the above objectives, the present invention also provides a digital semantic understanding and conversion system based on multimodal feature fusion, used to implement the aforementioned digital semantic understanding and conversion method based on multimodal feature fusion, comprising: The text input and preprocessing module is used to receive raw text input and perform encoding standardization and noise filtering. The digit instance detection and localization module is used to identify all digits and related patterns in text; The number instance structure parsing module is used to convert number strings into structured objects; The multimodal feature extraction module is used to collaboratively extract semantic features, mathematical features, and contextual features from digital instances; The hierarchical classification model inference module is used for macro-micro hierarchical classification. The rule engine's priority routing module is used to directly route high-confidence patterns to the corresponding converters; The semantic field construction and enhancement module is used to enhance the confidence of type judgment by utilizing contextual keywords; The hybrid decision engine integration module is used to integrate results from multiple sources to make a final decision; Type-converter dynamic routing module, used to select the corresponding converter based on the final type; The number-to-Chinese pronunciation generation module is used to generate number pronunciations that conform to Chinese reading habits; The context adaptation and text replacement module is used to seamlessly integrate the transformation results back into the original text; The output generation and decision traceability module is used to output the final results and interpretable decision logs.

[0016] As a further aspect of the present invention: the multimodal feature extraction module includes: The semantic feature extraction submodule uses the BERT model to extract contextual semantic embeddings and applies a dynamic window adjustment algorithm; The mathematical feature extraction submodule extracts format features, numerical features, mathematical attributes, and special pattern features; The feature fusion submodule uses a dynamic weighted fusion algorithm to generate a unified fusion feature vector.

[0017] As a further aspect of the present invention: the hierarchical classification model inference module includes: Macro classifiers are used to determine which major category a number belongs to: mathematical, time-based, quantitative, identifier, financial, or special categories. A cluster of micro-classifiers, comprising a dedicated micro-classifier corresponding to each macro-class; The hierarchical joint probability calculation submodule is used to calculate the final probability; The confidence propagation and fusion submodule is used to fuse macro and micro confidence levels.

[0018] As a further aspect of the present invention: the system also includes a zero-shot training module, used to generate training data through the Jinja2 template engine, automatically generating training data through a template method, including: Template definition component, used to define templates for the context in which numbers appear; The variable instantiation component is used to generate number instances according to formatting rules; Context mutation component for generating diverse text environments; Quality control components are used to check the reasonableness and diversity of the generated data; The automatic annotation component is used to automatically generate annotation information based on template metadata.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By integrating multimodal feature fusion and a hierarchical decision-making architecture, the problem of multi-class confusion is completely solved; 2. Zero-shot training mechanism overcomes data bottlenecks and enables rapid deployment of new data types; 3. The hybrid decision engine significantly improves the robustness of handling complex scenarios; 4. The context-aware conversion system ensures natural and accurate semantic expression. Attached Figure Description

[0020] Figure 1 This is a flowchart of the digital semantic understanding and conversion method based on multimodal feature fusion of the present invention.

[0021] Figure 2 This is a flowchart of the digital instance structured parsing process of the present invention, which shows the processing from raw text to structured digital objects.

[0022] Figure 3 This invention presents a heatmap of the importance of multimodal features, demonstrating the contribution of semantic, mathematical, and contextual features to the recognition of different digit types.

[0023] Figure 4 This is a schematic diagram of the multimodal feature extraction and fusion of the present invention, demonstrating the extraction and dynamic weighted fusion process of semantic, mathematical, and contextual features.

[0024] Figure 5 This is a schematic diagram of the hierarchical classification decision tree structure of the present invention, illustrating the type inference process from macro to micro.

[0025] Figure 6 This is a schematic diagram of the multi-engine hybrid decision collaboration of the present invention, demonstrating the collaborative mechanism of the rule engine, statistical model and semantic engine.

[0026] Figure 7 This is a schematic diagram of the rule engine priority management and conflict resolution mechanism of the present invention.

[0027] Figure 8 This is a flowchart of the zero-sample training data generation process of the present invention, which shows the automated data generation and annotation process based on Jinja2 templates.

[0028] Figure 9 This is a schematic diagram of the routing and syntax adaptation of the digital converter provided in an embodiment of the present invention, demonstrating the process of generating Chinese pronunciations for different types of numbers.

[0029] Figure 10 This is a comparison chart of the effects of digital conversion and text replacement provided in the embodiments of the present invention, which visually displays the macro and micro categories, decision engines, and some mathematical features. Detailed Implementation

[0030] The invention will now be further described with reference to the accompanying drawings.

[0031] like Figure 1 and Figure 2 As shown, the digital semantic understanding and conversion method based on multimodal feature fusion includes the following steps: S10: Input the original text and perform encoding standardization and noise filtering. Specifically, the input text is uniformly converted to UTF-8 encoding, and non-text symbols (such as control characters, garbled characters, etc.) and extra spaces are filtered out. For example, the text "Price is ¥1,000.00 (Special Price)!" becomes "Price is ¥1000.00 (Special Price)!" after preprocessing.

[0032] S20: Employs a multi-pattern regular expression engine to identify all numbers and related patterns in the text. The regular expression engine integrates various number patterns, including integers, decimals, percentages, fractions, times, ranges, IP addresses, version numbers, etc. A non-overlapping matching optimization algorithm is applied to resolve multi-pattern conflicts; the specific steps are as follows: 1) Generate all possible digit pattern matching results; 2) Grouping based on the starting position of the match (matching results within the same group have the same starting position); 3) Apply the longest match priority principle within the same starting position group; if the length difference between the longest match and the second longest match is ≤2, further filter based on context keywords (prioritize retaining matching results that are consistent with the context semantics). 4) Remove short matches that are completely contained within longer matches; 5) Output the final non-overlapping matching results.

[0033] For example, in the text “2023 revenue growth of 15.5%”, the regular expression engine will recognize the two numerical instances “2023” and “15.5%” and avoid recognizing “15.5” and “%” separately.

[0034] S30: Convert the numeric string identified in S20 into a structured object, including parsing the numeric format, extracting units, obtaining the context window, and performing numeric conversion. Specifically, each numeric instance is encapsulated as a NumberInstance object, containing attributes such as the original string, start position, end position, cleaned string, format tags, unit, numeric result, and context window. For example, the numeric string "15.5%" is parsed as: - raw_str: "15.5%"; - cleaned: "15.5"; - format_tokens: ['decimal', 'percentage']; - unit: "percentage"; - numeric_value: 15.5; - surrounding_chars: "Length 15.5%".

[0035] S40: Collaboratively extract semantic features, mathematical features, and contextual features from digital instances, and generate a unified fused feature vector using a dynamic weighted fusion algorithm. For example... Figure 4 As shown, the process includes: 1. Semantic Feature Extraction: A pre-trained BERT model is used to extract semantic embedding vectors of the context surrounding the digit. A dynamic window adjustment algorithm is applied to determine the optimal context range. The formula for the dynamic window adjustment algorithm is: ; in, This represents the optimal dynamic window size for extracting the semantic embedding vector of the context surrounding the digit. Indicates the base window size (flexibly adjustable from 3 to 10 characters for text types), the default is 5 characters. The numerical complexity score is calculated based on the length of the number and the number of special characters. The degree of contextual ambiguity is represented by the keyword conflict detection in the context. α and β are adjustment parameters that can be dynamically adjusted according to the text domain. For financial and technology texts, α is set to 0.6-0.7 (emphasizing numerical complexity), while for everyday texts, β is set to 0.4-0.5 (emphasizing contextual ambiguity). The initial values ​​are determined by grid search (α∈[0.3,0.8], β∈[0.2,0.6]).

[0036] 2. Mathematical Feature Extraction: This includes format features (such as symbols, separators, and number length distribution), numerical features (such as logarithmic transformations and numerical range classifications), mathematical attributes (such as the approximation degree of mathematical constants and power features), and special pattern features (such as repetition patterns, sequence patterns, and symmetry). For example, the mathematical features of the number "15.5%" include has_decimal_point=1, has_percent=1, is_percentage_range=1, and value_magnitude=1.

[0037] 3. Contextual Feature Extraction: This includes prefix and suffix keyword extraction and sentence structure analysis. Semantic field vectors are constructed using the SemanticFieldBuilderV2 module, and soft classification signals for numeric types are generated based on a predefined keyword weight mapping table (e.g., "growth" → percentage weight 0.8, "income" → currency weight 0.7).

[0038] 4. Feature Fusion: A dynamic weighted fusion algorithm is adopted, expressed by the formula as follows: ; Weight The calculation formula is: ; in, This represents the comprehensive feature vector obtained by dynamic weighted fusion after standardization of multiple feature classes; Represents the i-th class of feature vectors (semantic / mathematical / contextual features); The feature standardization function is represented by min-max standardization. The feature quality score is calculated based on the mutual information between the feature and the type label; H i α represents the historical accuracy of features. i and β i These are learnable parameters, which are optimized together with the classifier parameters using the gradient descent algorithm during the model training phase. The initial values ​​are set to 0.6 and 0.4.

[0039] S50: Uses a macro-micro hierarchical classification statistical model engine for number type inference (referred to as the "statistical model engine"). For example... Figures 5 to 8 As shown, the macro-classifier categorizes numbers into six major categories: mathematical, temporal, quantitative, identifier, financial, and special. The micro-classifier performs fine-grained classification within each macro-category, such as further subdividing the time category into time, duration, and year.

[0040] The hierarchical classification statistical model engine employs a zero-shot training mechanism for numeric type inference, generating training data through the Jinja2 template engine, specifically including: Template definition phase: Design multiple text templates for each number type, and define the formatting rules and context variation rules for number variables; Variable instantiation: Generate a large number of number instances according to formatting rules; Contextual variation: Applying contextual variation rules to generate diverse text environments; Quality control: Use rules to check the reasonableness and diversity of the generated data; Automatic annotation: Automatically generates annotation information based on template metadata; The hierarchical joint probability model calculates the final probability using the following formula: ; in, This represents the comprehensive features of the input digit to be classified and its context, i.e., the comprehensive feature vector in S40. , Macro-level classification labels for numbers (any of the following categories: mathematics, time, quantitative, identifier, finance, and special). Micro-level classification labels for numbers (such as subcategories like "moment", "duration", and "year" under the macro-level "time" category). Indicates based on input features Inferring that the number belongs to a certain macro category The posterior probability, Indicates a known macro-classification and input features Under the given conditions, infer that the number belongs to a specific micro-category within this macro-category. The posterior probability, This indicates that the final deduction leads to the conclusion that the number belongs to a certain micro-category. The joint posterior probability is the core probability output of this type of numerical reasoning.

[0041] The confidence propagation mechanism formula is expressed as: ; in, This represents the final comprehensive classification confidence score output by the hierarchical classification statistical model engine, which integrates the confidence scores of macro-classification and micro-classification. Indicates the confidence level for macro-classification. Indicates the confidence level of micro-classification. For the weight parameters, satisfying experience points .

[0042] Both macro and micro classifiers employ the LightGBM model, with feature input consisting of fused feature vectors generated by S40. Training data is automatically generated using the Jinja2 template engine, supporting zero-shot learning. For example, the Jinja2 template for the currency class is: { "template": "{{ number}} yuan", "variables": { "number": { "type": "currency", "formats": ["###.##", "#,###.##", "###"] } }, "context_variations": [ "The price is {{ number}}", "Price: {{ number}} yuan" "Only {{ number}} yuan" ] }

[0043] S60: For high-confidence numerical patterns, the rule engine directly routes to the corresponding converter. The rule engine has multiple built-in high-confidence rules. For example, if the number contains "%" or the context contains keywords such as "percentage" or "ratio", it routes to the percentage converter; if the number format is "X:Y" and the context contains time keywords such as "point", "minute", or "second", it routes to the time converter.

[0044] The rule engine includes a rule priority management algorithm and a rule conflict resolution mechanism.

[0045] The rule priority management algorithm formula is expressed as follows: ; in, To represent the overall priority score of the R-th high-confidence rule in the rule engine, Here, α represents the historical accuracy of the rule, Coverage(R) represents the rule coverage area, Conflict(R) represents the number of rule conflicts, and the weight parameters α, β, and γ are set to 0.5, 0.3, and 0.2, respectively.

[0046] The rule conflict resolution mechanism includes the following decision-making process: 1) Priority comparison: Select the rule with higher priority; 2) Specificity comparison: Selecting more specific rules for the pattern; 3) Context adaptability: Select rules that are more suitable for the current context. Context adaptability is determined by both keyword matching degree (threshold ≥ 0.7) and semantic field vector similarity (threshold ≥ 0.6). Rules that meet both indicators or have a higher overall score are given priority.

[0047] S70: Utilize a semantic engine to extract contextual keywords and construct a semantic field vector to enhance type judgment confidence. The SemanticFieldBuilderV2 module extracts keywords from the context surrounding the number and generates a semantic field vector based on a predefined weight mapping table. For example, the context of the number "15.5%" is "income growth of 15.5%", with the keyword "growth" having a weight of 0.8 (percentage) and "income" having a weight of 0.7 (currency). Therefore, the semantic field vector is {percentage: 0.8, currency: 0.7}, which, after normalization, becomes {percentage: 0.53, currency: 0.47}.

[0048] S80: The outputs of the rule engine, statistical model engine and semantic engine are combined, and a multi-engine dynamic weight allocation algorithm is used for the final decision.

[0049] The multi-engine dynamic weight allocation algorithm formula is expressed as follows: ; in, This represents the dynamically allocated weights of the i-th engine (rule engine, statistical model engine, semantic engine, corresponding to j=1,2,3 in the formula). This represents the engine's historical accuracy, calculated based on the past 1000 decisions. This indicates the current confidence level of the corresponding engine. The scene adaptability score is represented by α, β, and γ, which are adjustment parameters (example values ​​are 0.5, 0.3, and 0.2, respectively), and can be optimized through experiments. The final decision is a weighted voting result, expressed by the formula: ; in, This represents the final decision result of the number type obtained after dynamic weighted voting by multiple engines, i.e., the final determined number type label, such as "RMB amount", "Gregorian calendar date", "ratio", etc., which will be directly used by S90 to select the corresponding converter. It represents the set of all candidate number type labels, where i represents the engine index. This represents the dynamic weight allocation for the i-th engine. This represents the combined features of the input number to be processed and its context (i.e., (Comprehensive feature vector) This indicates that the i-th engine is based on input features. , infer the posterior probability that the number belongs to type y, It means to select the number type y that maximizes the subsequent summation result. That is, through the weighted voting method, select the type with the highest combined support of the three engines as the final decision.

[0050] S90: Output the final type label according to the final decision, and select the corresponding digital converter according to the final type label. The system has multiple built-in converters, such as telephone converters, currency converters, time converters, fraction converters, etc., and supports dynamic expansion. For example, when the number type is percentage, it is routed to the PercentageConverter.

[0051] S100: Call the conversion method of the selected converter to generate a digital reading that conforms to Chinese habits. For example: "15.5%" is converted to "fifteen point five percent"; "1,000.00 yuan" is converted to "one thousand yuan"; "3 / 4" is converted to "three fourths"; "192.168.1.1" is converted to "one nine two point one six eight point one point one".

[0052] S110: Seamlessly integrate the conversion result back to the original text position, maintaining the original grammar structure. Replace the converted digital reading back to the original text through the position mapping algorithm, and adjust the grammar form of adjacent words to ensure the smoothness of the statement. For example, "increase by 15.5%" is replaced by "increase by fifteen point five percent".

[0053] S120: Output the final converted text and the interpretable decision log.

[0054] The interpretable decision log includes the following content: The original digital string and its position in the text; The structured parsing result, including the extracted numerical value, unit, and context window; The detailed result of multimodal feature extraction; The probability distribution and confidence of macro classification and micro classification; The matching situation of the rule engine (if the match is successful, record the matching rule); The semantic field vector and its contribution degree to the decision; The weight allocation of each decision engine and the final decision reason; The final type label and the conversion result; The processing timestamp and the system resource usage. As Figure 10 shown, the interpretable decision log is output in a structured JSON format, supporting decision traceability and system tuning.

[0055] A digital semantic understanding and conversion system based on multimodal feature fusion, used in the aforementioned digital semantic understanding and conversion method based on multimodal feature fusion, includes: The text input and preprocessing module is used to receive raw text input and perform encoding standardization and noise filtering. The digit instance detection and localization module is used to identify all digits and related patterns in text; The number instance structure parsing module is used to convert number strings into structured objects; The multimodal feature extraction module is used to collaboratively extract semantic features, mathematical features, and contextual features from digital instances; Specifically, the multimodal feature extraction module includes: The semantic feature extraction submodule uses the BERT model to extract contextual semantic embeddings and applies a dynamic window adjustment algorithm; The mathematical feature extraction submodule extracts format features, numerical features, mathematical attributes, and special pattern features; The feature fusion submodule uses a dynamic weighted fusion algorithm to generate a unified fusion feature vector; The hierarchical classification model inference module is used for macro-micro hierarchical classification. Specifically, the hierarchical classification model inference module includes: Macro classifiers are used to determine which major category a number belongs to: mathematical, time-based, quantitative, identifier, financial, or special categories. A cluster of micro-classifiers, comprising a dedicated micro-classifier corresponding to each macro-class; The hierarchical joint probability calculation submodule is used to calculate the final probability; The confidence propagation and fusion submodule is used to fuse macro and micro confidence levels. The rule engine's priority routing module is used to directly route high-confidence patterns to the corresponding converters; The semantic field construction and enhancement module is used to enhance the confidence of type judgment by utilizing contextual keywords; The hybrid decision engine integration module is used to integrate results from multiple sources to make a final decision; Type-converter dynamic routing module, used to select the corresponding converter based on the final type; The number-to-Chinese pronunciation generation module is used to generate number pronunciations that conform to Chinese reading habits; The context adaptation and text replacement module is used to seamlessly integrate the transformation results back into the original text; The output generation and decision traceability module is used to output the final results and interpretable decision logs.

[0056] Furthermore, the system also includes a zero-shot training module, used to generate training data through the Jinja2 template engine. This automatic generation of training data via templated methods overcomes the bottleneck of scarce labeled data, enabling rapid expansion and support for new number types, including: Template definition component, used to define the context template for the appearance of numbers (including placeholders for number variables and formatting rules). The variable instantiation component is used to generate a large number of numeric instances that conform to type characteristics according to format rules. Context mutation component, used to generate diverse text environments based on predefined rules (≥5 mutation scenarios for each number type); The quality control component checks the reasonableness and diversity of the generated data, specifically through two rules: 1) Filter instances where the number format does not conform to the template definition; 2) Ensure that the contextual variations for each number type cover different sentence structures (declarative sentences, imperative sentences, etc.); An automatic labeling component is used to automatically generate labeling information (including macro-level classification labels, micro-level classification labels, and initial confidence values) based on template metadata.

[0057] Principles and effects of this invention: I. Multimodal Feature Dynamic Fusion Framework like Figure 3 As shown, this invention demonstrates the contribution of semantic, mathematical, and contextual features to the recognition of different digit types through a multimodal feature importance heatmap, which is significantly better than single feature methods. Figure 4 It further demonstrates the specific process of multimodal feature extraction and dynamic weighted fusion, including BERT semantic embedding, mathematical feature parsing, context keyword extraction and dynamic weight calculation.

[0058] To address the limitations of traditional methods in feature extraction due to their singularity and one-sidedness, this invention employs a multimodal feature dynamic fusion framework to achieve the collaborative extraction and optimized fusion of semantic features, mathematical features, and contextual features. In the feature extraction stage, the system utilizes a three-dimensional parallel processing mechanism: 1. Semantic feature extraction uses a pre-trained BERT model to generate context embedding vectors, and introduces a dynamic window adjustment algorithm. It automatically optimizes the analysis range based on numerical complexity and contextual ambiguity, thus solving the feature truncation problem caused by a fixed window.

[0059] 2. Mathematical Feature Extraction: A dedicated engine analyzes the format features (symbols, separators, length distribution), numerical features (logarithmic transformation, range classification), mathematical attributes (constant approximation, power features), and special patterns (repetition, sequence, symmetry) of numbers, comprehensively capturing the inherent mathematical properties of numbers.

[0060] 3. The feature fusion mechanism employs a dynamic weighting algorithm. Based on feature quality scoring (Mutual information with type labels) and historical accuracy Adaptively assign weights to generate the optimal fused feature vector. .

[0061] This technology enables multi-dimensional feature collaborative optimization in digital semantic understanding, such as... Figure 3 As shown, the contribution of each dimension of features to the decision is intuitively displayed through the feature importance heatmap, which solves the problem of misjudgment caused by incomplete feature extraction in traditional methods.

[0062] II. Hierarchical Decision Tree Classification Architecture like Figure 5 As shown, this invention employs a macro-micro dual-layer classification structure, effectively blocking error propagation. Figure 8 This invention demonstrates a zero-shot training data generation process based on Jinja2 templates, including template definition, variable instantiation, context mutation, quality control, and automatic annotation. Addressing the core technical challenge of multi-class confusion, this invention innovatively designs a hierarchical decision tree classification architecture, effectively blocking error propagation paths through a macro-micro dual-layer decision-making mechanism. 1. The macro-classification layer categorizes over 30 numeric types into 6 semantic categories (mathematics, time, quantification, identifiers, finance, and special), employing a lightweight LightGBM model for coarse-grained classification. This layer features a specially designed category boundary optimization algorithm that significantly improves macro-classification accuracy by enhancing inter-category distinguishing features (such as time separators for time categories and currency symbols for finance categories).

[0063] 2. The micro-classification layer equips each macro-category with a dedicated classifier (such as subdividing the time category into subcategories like time, duration, and year), employing a hierarchical joint probability model. Calculate the final probability. An innovative confidence propagation mechanism is introduced. It generates reliable decision-making basis by integrating macro and micro confidence levels.

[0064] 3. Zero-shot training mechanism overcomes data bottlenecks; Jinja2 template engine is used to achieve automatic data generation: • Define a context template that includes numeric variable placeholders (e.g., "{{number}} yuan"); • Configure number format rules (e.g., currency types support formats such as ###.##, #,###.##, etc.); • Design at least 3 basic templates for each number type. The templates should cover different sentence structures (declarative sentences, imperative sentences, interrogative sentences) and contextual distance (1-5 characters before and after the number). • Generate contextual variation combinations (e.g., "price {{number}}", "selling price {{number}} yuan"); • Filter out abnormal samples through the quality control module; This architecture is the first to achieve error propagation blocking in digital semantic understanding, such as Figure 5 The decision tree structure shown illustrates the information transmission path at the micro level, solving the confusion problem in fine-grained recognition of traditional multi-classification models.

[0065] III. Multi-engine Hybrid Decision System like Figure 6 As shown, this invention achieves highly robust decision-making through the collaborative efforts of a rule engine, a statistical model, and a semantic engine. Figure 7 It further demonstrates the priority management and conflict resolution mechanisms of the rules engine, ensuring the consistency and efficiency of rule application.

[0066] To improve decision robustness in complex scenarios, this invention creates a hybrid decision system with multi-engine collaboration: 1. The rule engine has a built-in high-confidence matching rule library and uses a priority dynamic management algorithm. Optimize the order of rule application and design a three-level conflict resolution mechanism (priority comparison → specificity comparison → context adaptability analysis) to ensure rule consistency.

[0067] 2. The statistical engine outputs probability distributions based on a hierarchical classification model and innovatively incorporates a scenario adaptability assessment module to analyze the semantic compatibility of numbers with context.

[0068] 3. The semantic engine constructs a semantic field vector by mapping keyword weights (such as "telephone" → "telephone"), providing supplementary judgment basis for low-confidence cases.

[0069] 4. The dynamic fusion module adopts a weighted voting mechanism. Based on engine historical accuracy Current confidence level and scene adaptability Automatically assign weights to generate optimal decisions .

[0070] like Figure 6The diagram shown illustrates the integration process of three decision signals, addressing the failure of traditional single decision-making mechanisms in edge cases.

[0071] IV. Context-Aware Transition System For the digital semantic conversion stage, this invention develops an intelligent conversion routing and syntax adaptation mechanism, such as... Figure 9 As shown, this invention achieves intelligent generation of Chinese reading methods by using dynamic routing of type-converter and multi-mode conversion algorithm, supporting accurate conversion of complex formats such as IP address, version number, and currency amount. 1. Type-converter dynamic routing establishes fine-grained mapping relationships (such as telephone→phone converter), and supports custom extension rules to adapt to new digital formats.

[0072] 2. The multi-mode conversion algorithm designs dedicated conversion logic for different number types: • Handling complex formats: IP address (192.168.1.1 → 192.168.1.1), version number (3.2.1 → 3.2.1) • Unit synthesis mechanism: Intelligent integration of quantifiers such as "yuan", "du", and "fen" (50.99 yuan → fifty point ninety-nine yuan) • Grammar adaptation engine: Analyzes the grammatical role of numbers in a sentence and adjusts the tone and style of the converted result. 3. Structure-preserving replacement technology achieves seamless text replacement through position mapping algorithms, preserving the original punctuation, formatting, and paragraph structure.

[0073] The specific implementation method is as follows: For example, consider this company's 2023 financial report: "The company's annual revenue increased by 18.5% year-on-year, reaching 1,500,000,000 yuan. The contact number has changed to 010-66885678; please confirm before 15:30 the following day. Furthermore, the budget for Project 93 accounts for approximately 3 / 4 of the total budget, and the core equipment supports the 802.11ac protocol." 1. First, perform text input and preprocessing. The system receives the raw text encoded in UTF-8 and performs noise filtering.

[0074] 2. Perform digital instance detection and localization. The multi-pattern regular expression engine scans the entire text to identify and locate all numeric patterns. In this example, the detected numeric instances are as follows: “2023” (Starting position index: 2, Ending position: 6); "18.5%" (Start position index: 18, End position: 22); “1,500,000,000” (Start position index: 27, End position: 41); “010-66885678” (Starting position index: 52, Ending position: 65); “15:30” (Starting position index: 72, Ending position: 76); “93” (Starting position index: 85, Ending position: 87); “3 / 4” (Start position index: 95, End position: 98); “802.11ac” (Start position index: 109, End position: 117); The non-overlapping matching algorithm ensures that “1,500,000,000” is recognized as a whole rather than split into multiple digits, and ensures that “802.11” is not incorrectly split into “802” and “11”.

[0075] 3. Structured parsing of digital instances Each detected numeric string is instantiated as a NumberInstance object and subjected to deep parsing: "2023": The format is marked as ['integer'], the value is 2023.0, and the context window is "Company 2023 Financial Statements".

[0076] "18.5%": The format is marked as ['decimal', 'percentage'], and after cleaning it is "18.5", the value is 18.5, the unit is extracted as "percentage", and the context window is "18.5% long, up to".

[0077] "1,500,000,000": The format is marked as ['comma_number', 'integer'], and after cleaning it is "1500000000", the value is 1500000000.0, the unit is extracted as "yuan", and the context window is "to 1,500,000,000 yuan."

[0078] “010-66885678”: The format is marked as ['range'] (because it contains a hyphen), but through unit extraction and context analysis, its unit is initially determined to be “telephone”.

[0079] "15:30": The format is marked as ['time'], the value cannot be directly parsed, and the context window is "15:30 before".

[0080] "93": The format is marked as ['integer'], the value is 93.0, and the context window is "External, Item 93".

[0081] "3 / 4": The format is marked as ['fraction'], the value is calculated as 0.75, and the context window is "3 / 4, core".

[0082] “802.11”: The format is marked as ['decimal'], but the “ac” in the context suggests that it may be a special code.

[0083] 4. Multimodal feature extraction and fusion For each instance, the system extracts three types of features in parallel and dynamically fuses them: Semantic Features: The BERT model processes the context of each number (e.g., "Annual revenue increased by 18.5% year-on-year") to generate a high-dimensional semantic vector. A dynamic window adjustment algorithm expands the window to "1,500,000,000" to capture key information such as "revenue" and "yuan".

[0084] Mathematical features: Extract the mathematical attributes of each number. For example, "1,500,000,000" has integer_length=10 and is_very_large=1; "3 / 4" has is_fraction=1, value=0.75, and is_probability_range=1.

[0085] Contextual features: The semantic field builder analyzes contextual keywords. For example: The terms "financial report," "revenue," and "yuan" strongly suggest a currency (financial-monetary) type.

[0086] "Year-on-year growth" and "%" strongly indicate the percentage (mathematical - percentage) type.

[0087] "Contact number" and "Change" strongly point to the telephone (identifier - telephone) type.

[0088] "Next day", "Pre-confirmation" and ":" strongly indicate the time (time-moment) type.

[0089] The terms "project," "budget," "total," and fraction format strongly suggest a fraction (mathematics-fraction) type.

[0090] The special formats for "protocol" and "802.11" point to the code (identifier-code) type.

[0091] Feature fusion: The dynamic weighting algorithm generates a unified fusion feature vector for each digit instance, which is used by the classifier. For "93", there is a conflict between its mathematical features (ordinary integer) and semantic features ("number item"), and the fusion weights tend to favor the semantic features.

[0092] 5. Hierarchical classification and hybrid decision-making Each digital instance enters a hierarchical classification and hybrid decision-making process: "18.5%": The rule engine directly matches the high-confidence rule (containing "%)", skips model inference, and is directly classified as mathematical -> percentage with a confidence level of 0.92.

[0093] “1,500,000,000 yuan”: The rule engine matches the unit rule for “yuan”, which is classified as financial -> currency, with a confidence level of 0.93.

[0094] “010-66885678”: The rule engine matches the “contact phone number” context rule, classifying it as identifier -> telephone, with a confidence level of 0.95.

[0095] “15:30”: The rule engine matches the “:” format and the “preconfirmation” context, classifying it as temporal ->time, with a confidence level of 0.92.

[0096] “3 / 4”: The rule engine matches the “ / ” format, classifies it as mathematical -> fraction, with a confidence level of 0.90.

[0097] "2023": The macro-classifier classifies it as temporal based on its numerical range (1900-2100) and the context of "year". The micro-classifier further subdivides it into temporal -> year based on the strong semantic signal "year", with a combined confidence level of 0.88.

[0098] "Item 93": The macro-classifier categorizes it as quantitative. The micro-classifier receives conflicting signals: the mathematical features resemble regular_number, but in the semantic field, "item number" points more to an identifier (such as an item number). Due to the higher weight of the semantic field, it is ultimately classified as identifier -> code, with a confidence level of 0.75.

[0099] "802.11ac": The macro-classifier classifies it as special. The micro-classifier combines its format (similar to a version number) and the context "protocol" and classifies it as special -> version with a confidence of 0.80.

[0100] 6. Digital Conversion and Text Replacement The type-converter dynamic routing module calls the corresponding converter according to the final type: The year converter: "2023" -> "two thousand and twenty-three"; The percentage converter: "18.5%" -> "eighteen point five percent"; The currency converter: "1,500,000,000 yuan" -> "fifteen hundred million yuan"; The telephone converter: "010-66885678" -> "zero one zero dash six six eight eight five six seven eight"; The time converter: "15:30" -> "fifteen thirty"; The code converter: "93" -> "ninety-three"; The fraction converter: "3 / 4" -> "three fourths"; The version converter: "802.11" -> "eight zero two point one one"; The context adaptation module seamlessly replaces the conversion result back to the original position in the original text to generate the final output: "The company's financial report for the year 2023 shows that its annual revenue increased by 18.5% year-on-year to reach 1500000000 yuan. The contact phone number has been changed to 010-66885678. Please confirm before 15:30 the next day. In addition, the budget for Project 93 accounts for approximately three fourths of the total, and the core equipment supports the 802.11ac protocol."

Claims

1. A digital semantic understanding and conversion method based on multimodal feature fusion, characterized in that, Includes the following steps: S10: Input the raw text and perform encoding standardization and noise filtering; S20: Employs a multi-pattern regular expression engine to identify all numbers and related patterns in the text, and applies a non-overlapping matching optimization algorithm to resolve multi-pattern conflicts; S30: Convert the numeric string identified in S20 into a structured object; S40: Collaboratively extract semantic features, mathematical features, and contextual features from digital instances, and generate a unified fusion feature vector using a dynamic weighted fusion algorithm; S50: Uses a macro-micro hierarchical classification statistical model engine to infer the type of numbers, including using a macro classifier to determine which major category a number belongs to, such as mathematical, time, quantitative, identifier, financial, or special categories, and then using the corresponding micro classifier for fine-grained classification. It also applies a hierarchical joint probability model to calculate the final probability, introduces a confidence propagation mechanism, and integrates macro and micro confidence to produce reliable decision-making basis. S60: High-confidence digital patterns are directly routed to the corresponding converter via the rules engine; S70: Construct semantic field vectors using contextual keywords to enhance the confidence of type judgment; S80: The final decision is made by combining the outputs of the rule engine, statistical model and semantic engine, using a multi-engine dynamic weight allocation algorithm. S90: Output the final type label based on the final decision, and select the corresponding digital converter based on the final type label; S100: Calls the conversion method of the selected converter to generate a number reading method that conforms to Chinese reading habits; S110: Seamlessly integrate the conversion result back into the original text position, preserving the original grammatical structure; S120: Output the final transformed text and interpretable decision log.

2. The digital semantic understanding and conversion method based on multimodal feature fusion according to claim 1, characterized in that, The S40 process includes: Semantic feature extraction: The BERT model is used to extract semantic embedding vectors of the context surrounding the digit, and a dynamic window adjustment algorithm is applied to determine the optimal context range. The formula for the dynamic window adjustment algorithm is as follows: ; in, This represents the optimal dynamic window size for extracting the semantic embedding vector of the context surrounding the digit. Indicates the base window size. Represents the numerical complexity score. Indicates the degree of contextual ambiguity, where α and β are adjustment parameters; Extracting mathematical features: including format features, numerical features, mathematical attributes, and special pattern features. Format features include symbols, separators, and number length distribution. Numerical features include logarithmic transformation, numerical range classification, and special value detection. Mathematical attributes include the approximation degree of mathematical constants and power features. Special patterns include repetition patterns, sequence patterns, and symmetry detection. Extracting contextual features: including extracting prefix and suffix keywords and analyzing sentence structure; Feature fusion: A dynamic weighted fusion algorithm is used, expressed by the formula: ; Weight The calculation formula is: ; in, This represents the comprehensive feature vector obtained by dynamic weighted fusion after standardization of multiple feature classes. Represents the feature vector of the i-th class. Represents the feature standardization function, H represents the feature quality score. i α represents the historical accuracy of features. i and β i These are learnable parameters, which are optimized together with the classifier parameters using the gradient descent algorithm during the model training phase.

3. The digital semantic understanding and conversion method based on multimodal feature fusion according to claim 2, characterized in that, The S50 mid-level classification statistical model engine employs a zero-shot training mechanism for digit type inference, generating training data through the Jinja2 template engine, specifically including: Template definition phase: Design multiple text templates for each number type, and define the formatting rules and context variation rules for number variables; Variable instantiation: Generate a large number of number instances according to formatting rules; Contextual variation: Applying contextual variation rules to generate diverse text environments; Quality control: Use rules to check the reasonableness and diversity of the generated data; Automatic annotation: Automatically generates annotation information based on template metadata; The formula for calculating the hierarchical joint probability model is: ; in, This represents the comprehensive features of the input digit to be classified and its context, i.e., the comprehensive feature vector in S40. , Macro-level classification labels representing numbers. Micro-level classification labels representing numbers, Indicates based on input features Inferring that the number belongs to a certain macro category The posterior probability, Indicates a known macro-classification and input features Under the given conditions, infer that the number belongs to a specific micro-category within this macro-category. The posterior probability, This indicates that the final deduction leads to the conclusion that the number belongs to a certain micro-category. The joint posterior probability, which is the core probability output of this type of inference; The confidence propagation mechanism formula is expressed as: ; in, This represents the final comprehensive classification confidence score output by the hierarchical classification statistical model engine, which integrates the confidence scores of macro-classification and micro-classification. Indicates the confidence level for macro-classification. Indicates the confidence level of micro-classification. For the weight parameters, satisfying .

4. The digital semantic understanding and conversion method based on multimodal feature fusion according to claim 1, characterized in that, The S60 rule engine includes a rule priority management algorithm and a rule conflict resolution mechanism. The formula for the rule priority management algorithm is as follows: ; in, This represents the historical accuracy of the rule, Coverage(R) represents the range of numeric types covered by the rule, and Conflict(R) represents the number of rule conflicts. These are weight parameters; The rule conflict resolution mechanism includes the following decision-making process: 1) Priority comparison: Select the rule with higher priority; 2) Specificity comparison: Selecting more specific rules for the pattern; 3) Context adaptability: Select the rule that is more suitable for the current context.

5. The digital semantic understanding and conversion method based on multimodal feature fusion according to claim 1, characterized in that, The formula for the multi-engine dynamic weight allocation algorithm in S80 is expressed as follows: ; in, This represents the dynamically allocated weights of the i-th engine (rule engine, statistical model engine, semantic engine, corresponding to j=1,2,3 in the formula). A i This indicates the engine's historical accuracy. C i This indicates the current confidence level of the corresponding engine. S i This represents the scene adaptability score, where α, β, and γ are adjustment parameters. The final decision is a weighted vote, expressed by the formula: ; in, This represents the final decision result of the number type obtained after dynamic weighted voting by multiple engines, i.e., the final determined number type label. This represents the set of all candidate numeric type labels, where i represents the engine index. This represents the dynamic weight allocation for the i-th engine. This represents the comprehensive features of the input digit to be processed and its context, i.e., the comprehensive feature vector. , This indicates that the i-th engine is based on input features. Infer the posterior probability that the number belongs to type y. This indicates that the type y that maximizes the subsequent summation result is selected, meaning that the type with the highest joint support from the three engines is chosen as the final decision through a weighted voting method.

6. The digital semantic understanding and conversion method based on multimodal feature fusion according to claim 1, characterized in that, The interpretable decision log in S120 includes the following: The raw string of numbers and its position in the text; The structured parsing results include the extracted numeric values, units, and context window; Detailed results of multimodal feature extraction; Probability distributions and confidence levels of macro- and micro-classifications; Matching results from the rules engine; Semantic field vectors and their contribution to decision-making; Weight allocation for each decision engine and the rationale for the final decision; Final type label and conversion result; Process timestamps and system resource usage.

7. A digital semantic understanding and conversion system based on multimodal feature fusion, used to implement the digital semantic understanding and conversion method based on multimodal feature fusion as described in any one of claims 1-6, characterized in that, include: The text input and preprocessing module is used to receive raw text input and perform encoding standardization and noise filtering. The digit instance detection and localization module is used to identify all digits and related patterns in text; The number instance structure parsing module is used to convert number strings into structured objects; The multimodal feature extraction module is used to collaboratively extract semantic features, mathematical features, and contextual features from digital instances; The hierarchical classification model inference module is used for macro-micro hierarchical classification. The rule engine's priority routing module is used to directly route high-confidence patterns to the corresponding converters; The semantic field construction and enhancement module is used to enhance the confidence of type judgment by utilizing contextual keywords; The hybrid decision engine integration module is used to integrate results from multiple sources to make a final decision; Type-converter dynamic routing module, used to select the corresponding converter based on the final type; The number-to-Chinese pronunciation generation module is used to generate number pronunciations that conform to Chinese reading habits; The context adaptation and text replacement module is used to seamlessly integrate the transformation results back into the original text; The output generation and decision traceability module is used to output the final results and interpretable decision logs.

8. The digital semantic understanding and conversion system based on multimodal feature fusion according to claim 7, characterized in that, The multimodal feature extraction module includes: The semantic feature extraction submodule uses the BERT model to extract contextual semantic embeddings and applies a dynamic window adjustment algorithm; The mathematical feature extraction submodule extracts format features, numerical features, mathematical attributes, and special pattern features; The feature fusion submodule uses a dynamic weighted fusion algorithm to generate a unified fusion feature vector.

9. The digital semantic understanding and conversion system based on multimodal feature fusion according to claim 7, characterized in that, The hierarchical classification model inference module includes: Macro classifiers are used to determine which major category a number belongs to: mathematical, time-based, quantitative, identifier, financial, or special categories. A cluster of micro-classifiers, comprising a dedicated micro-classifier corresponding to each macro-class; The hierarchical joint probability calculation submodule is used to calculate the final probability; The confidence propagation and fusion submodule is used to fuse macro and micro confidence levels.

10. The digital semantic understanding and conversion system based on multimodal feature fusion according to claim 7, characterized in that, The system also includes a zero-shot training module, used to generate training data through the Jinja2 template engine. This module automatically generates training data using template methods, including: Template definition component, used to define templates for the context in which numbers appear; The variable instantiation component is used to generate a large number of overlapping type feature numeric instances according to format rules; Context mutation component, used to generate diverse text environments based on predefined rules; Quality control components are used to check the reasonableness and diversity of the generated data; The automatic annotation component is used to automatically generate annotation information based on template metadata.

Images