Unit normalization and dimension consistency checking method and system for numerical terms

Abstract

The invention relates to the technical field of natural language processing and dimensional reasoning, and particularly discloses a unit normalization and dimensional consistency checking method and system for numerical terms. According to the method, through text table joint extraction, dimension diagram construction and conversion chain search, intra-term and cross-source consistency check, generation stage dimension guard and re-retrieval correction, strong binding of numerical values, units and evidences is achieved, dimension correctness, cross-source consistency and auditing performance are improved, and model illusion is effectively inhibited.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and dimensional reasoning technology, specifically relating to a method and system for unit normalization and dimensional consistency verification of numerical terms. Background Technology

[0002] In fields such as power, energy, and industrial control, which involve a large number of technical standards and specifications, the accurate extraction, consistency verification, and controlled generation of numerical clauses (such as thresholds, ranges, power, voltage, frequency, etc.) in normative texts and tables are key technologies for achieving automated compliance review, intelligent Q&A, and report generation.

[0003] Currently, existing technologies are mainly developing along two routes: One type is the rule-based and table-structure-based engineering parsing method. This method mainly relies on regular expressions, unit dictionaries, table structure recovery (such as merging cell expansion and header hierarchy parsing), and templated formatted output. It has certain advantages in terms of engineering controllability and structured output (such as "parameter-threshold-unit-condition" templates). However, this type of method usually treats "units" as simple string-level replacements or mappings, failing to systematize units into dimensional vectors with algebraic operational properties. Therefore, it has inherent defects in the canonical processing of SI prefixes (such as k, M, μ), affine transformations (such as the conversion between dB and linear values), and consistent modeling of scaling benchmarks (such as the reference benchmark for pu). At the same time, this method is quite sensitive to column header inheritance, merged cells, and loss of regional units in the table, which can easily lead to "value-unit" mismatches, thus causing systematic errors in subsequent calculations.

[0004] Another category is data-driven learning and generation methods. These methods utilize supervised or weakly supervised sequence labeling, tabular question answering (such as DocVQA and Tapas-like models), retrieval-enhanced generation (RAG), and large language models (LLM), combined with strategies like index retrieval and pseudo-label self-training to improve coverage and generalization ability. When annotations are sufficient and retrievals are accurate, this method can provide answers in natural language form and exhibit high apparent accuracy. However, it often ignores the fundamental physical principle of "dimensional conservation" in the generation phase (especially the decoding phase) and lacks a strong binding mechanism with source evidence. This can lead to the model outputting incorrect values, inverted inequalities, or interval boundaries in the absence of clear units or in cross-standard contexts, or even directly generating "magic numbers" without supporting evidence. When encountering differences across standards (such as GB / T, DL, IEC, IEEE), versions, or device models, strategies based purely on distribution alignment or pseudo-label upgrades are prone to "style bias," meaning the output may appear reasonable in text format but is unusable in terms of physical magnitude or actual compliance.

[0005] The common shortcomings of the two types of methods mentioned above also include: There is insufficient special handling for dimensionless or logarithmic base parameters such as pu and dB.

[0006] The general neglect of significant figures and tolerance specifications.

[0007] Semantic regularization is not robust to scientific notation, comparison operators (≥, ≤), and interval expressions (such as "10-35").

[0008] From the perspective of systems engineering and compliance auditing, existing technologies are generally insufficient in terms of "auditability, traceability, and uncertainty estimation." Most works only output final assertions (e.g., a clause should be ≥10 kV), but lack key information such as intermediate physical quantities, conversion paths, original evidence anchors, and version effective time windows for verification. When results need to be retrospectively verified, existing systems usually only provide confidence scores or text fragments, rather than structured "number-unit-evidence-version" binding relationships and uncertainty items. Therefore, when performing multi-source fusion and cross-version comparison, existing systems often use simple global uniform alignment or statistical aggregation (e.g., mean / mode), lacking robust statistics (e.g., median combined with MAD / IQR) and version weighting mechanisms, making it difficult to reliably identify real anomalies caused by standard differences or version evolution. In addition, existing methods do not implement prior prevention of dimensional conflicts (e.g., mistakenly treating kV and A as convertible units) at the algorithm level, making conversion errors a common hidden danger.

[0009] The root cause of these shortcomings lies in the fact that existing research and engineering practices have long equated the processing of numerical terms with "string / style alignment" and "label migration" problems, failing to regard quantitative information as a first-class element that needs to meet strict algebraic and dimensional constraints. There is a lack of strong binding and guarding mechanisms between the extraction and generation modules, and the training and evaluation metrics of the models are mainly guided by apparent accuracy, neglecting key practical requirements such as numerical reversibility, dimensional consistency, and audit friendliness.

[0010] In summary, existing technologies struggle to ensure the dimensional accuracy and cross-source consistency of numerical assertions in scenarios such as regulatory interpretation, technical review, and automated compliance verification. They are prone to errors in dimensional conversion, misjudgments of thresholds due to missing units, or unsupported figures generated by the model. This significantly increases the cost of manual verification and reduces the confidence level and compliance risk management capabilities of automated systems. Summary of the Invention

[0011] The main technical problem to be solved by this invention is to overcome the defects in the processing of numerical clauses in normative texts and tables, such as mismatch of units, inconsistency across sources, and generation of magic numbers by models, in the existing technology, and to provide a systematic method that can achieve automatic unit normalization, unit consistency verification, and auditable and controlled generation.

[0012] To solve the above-mentioned technical problems, the present invention is implemented as follows: This invention provides a method for verifying unit normalization and dimensional consistency for numerical terms, comprising the following steps: By using a joint extraction unit of text and tables, numerical values, units, applicable conditions and source anchors are identified from the input document to generate a set of numerical items, where each numerical item includes a numerical value, unit, dimensional vector, applicable conditions and clause identifier. A dimensional graph is constructed based on the mapping relationship between units and dimensional vectors. This dimensional graph is represented as follows: ,in For unit sets, For the convertible relation edge, unit conversion is allowed only when the dimensions are equivalent; By minimizing the path cost function Select the optimal conversion chain to complete the unit normalization of the numerical values, where Let be the number of sides. This indicates the upper bound of the boundary-level conversion error. As weight, The unit in which the current value is located. The target unit to be converted; After normalizing the units of the numerical items under the same clause, the consistency conclusion and reason code within the clause are output based on the deviation matrix, weighted satisfaction degree, interval combinability and significant number rules. For multi-source numerical items under the same conceptual key, consistency, outlier score and representative value are calculated by weighting based on source level and effective time window, using weighted median and median absolute deviation. During the decoding process, dimensionality guards, conversion guards, and relational guards are applied to candidate numbers, and probability penalties are used to force numerical values ​​to be bound to evidence. Numerical consistency scores are calculated based on dimensional consistency rate, normalization error, and citation coverage, and then integrated with retrieval margin, evidence coverage, and semantic implication scores to form the total confidence score. When the total confidence score or numerical consistency score is below the threshold, unit consistency rewriting, targeted supplementary verification, and conversion chain reselection are triggered.

[0013] Furthermore, in the joint extraction step of text and table, the unit determination follows the rule of "column header first, row header second, and cell as a catch-all", and the automatic completion and tracing of regional unit omissions are achieved by merging cell expansion and parsing table header hierarchy.

[0014] Furthermore, the construction of the dimensional graph includes: resolving the unit into a linear combination of fundamental dimensional vectors, only when... Establish transformation edges at time, where as a unit A dimensional vector.

[0015] Furthermore, in the consistency verification step within the clauses, the weighted satisfaction degree is calculated as follows: ; in As weight; This refers to the numerical tolerance threshold within the clause; For standardized relational predicates; For numerical terms The normalized value; For indicator functions; This is the deviation matrix within the clauses.

[0016] Furthermore, in the cross-source consistency comparison step, the source weight is calculated as follows: ,in For source level, The time window constant, This is an exponential decay factor based on time difference, used to reduce the weight of sources from older time periods; the consistency degree is calculated as follows: ; in Classified as abnormal; This is the abnormal threshold.

[0017] Furthermore, in the dimension guarding step of the generation phase, decoding is performed using logarithmic probability penalty constraints: ; in This is the penalty coefficient; Tolerance scale; To assert the value in the target unit; For evidence normalization; The most relevant span.

[0018] Furthermore, the numerical consistency score is calculated as follows: ; in For a set of numerical assertions; An indication exists for the reference.

[0019] The present invention also provides a system for implementing the method, comprising: Extraction and binding unit, used to perform joint extraction of values ​​and units from text and tables; The dimension determination and conversion unit is used to construct a dimension graph and perform a conversion chain search. Consistency verification unit, used for intra-clause and cross-source consistency verification; Controlled generation and scoring fusion unit, used to implement dimensionality guarding and confidence fusion; Then, the correction unit is retrieved to perform adaptive correction and closed-loop control.

[0020] Compared with the prior art, the advantages of this invention are as follows: (1) By using the dimensional diagram prior judgment and the optimal conversion chain mechanism, illegal unit conversion and dimensional deviation are fundamentally avoided.

[0021] (2) By combining table structure analysis and table header inheritance, the problem of missing regional units is robustly solved, forming an evidence chain that can be accurately traced.

[0022] (3) Using robust statistics-based cross-source aggregation and anomaly detection, it can clearly identify and indicate differences in standards or versions.

[0023] (4) Introducing dimension guards at the generation end allows physical constraints to be applied directly to the decoding process, which significantly improves the dimensional accuracy and consistency of the numerical output.

[0024] (5) The entire process is auditable and outputs intermediate results such as dimension vectors, conversion paths, and reason codes, which greatly facilitates compliance verification and problem tracing.

[0025] This invention is applicable to fields such as power, energy, and industrial control that involve a large number of normative numerical clauses, and can provide highly reliable technical support for automatic compliance review, intelligent Q&A, and report generation. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 A flowchart of the unit normalization and dimensional consistency verification method for numerical clauses provided in Embodiment 1 of the present invention; Figure 2 This is a framework diagram of the unit normalization and dimensional consistency verification system for numerical clauses provided in Embodiment 2 of the present invention. Figure 3 This is one of the hardware structure schematic diagrams of the electronic device provided in Embodiment 3 of the present invention; Figure 4 This is the second schematic diagram of the hardware structure of the electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The terms "first," "second," etc., used in this specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, the first object can be one or more. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0029] This invention provides a method for unit normalization and dimensional consistency verification of numerical clauses. It aims to achieve strong binding between numerical values, units, and evidence, and auditable uncertainty output through techniques such as prior dimensional determination and conversion, joint table-text extraction, robust cross-source consistency comparison, and generation-end dimensional guarding. This significantly reduces the risk of misjudgment caused by dimensional mismatch and model magic numbers. The method provides a unified dimensional constraint link and re-retrieval correction mechanism in each stage of extraction, comparison, and generation, and is applicable to multi-source, multi-version, and multi-standard clause numerical verification scenarios. The method specifically includes the following steps: By using a joint extraction unit of text and tables, numerical values, units, applicable conditions and source anchors are identified from the input document to generate a set of numerical items, where each numerical item includes a numerical value, unit, dimensional vector, applicable conditions and clause identifier. A dimensional graph is constructed based on the mapping relationship between units and dimensional vectors. This dimensional graph is represented as follows: ,in For unit sets, For the convertible relation edge, unit conversion is allowed only when the dimensions are equivalent; By minimizing the path cost function Select the optimal conversion chain to complete the unit normalization of the numerical values, where Let be the number of sides. This indicates the upper bound of the boundary-level conversion error. As weight, The original unit for this conversion step. The target unit to be converted in this step; After normalizing the units of the numerical items under the same clause, the consistency conclusion and reason code within the clause are output based on the deviation matrix, weighted satisfaction degree, interval combinability and significant number rules. For multi-source numerical items under the same conceptual key, consistency, outlier score and representative value are calculated by weighting based on source level and effective time window, using weighted median and median absolute deviation. During the decoding process, dimensionality guards, conversion guards, and relational guards are applied to candidate numbers, and probability penalties are used to force numerical values ​​to be bound to evidence. Numerical consistency scores are calculated based on dimensional consistency rate, normalization error, and citation coverage, and then integrated with retrieval margin, evidence coverage, and semantic implication scores to form the total confidence score. When the total confidence score or numerical consistency score is below the threshold, unit consistency rewriting, targeted supplementary verification, and conversion chain reselection are triggered.

[0030] In the joint extraction steps of text and table, the unit determination follows the rule of "column header first, row header second, and cell as a catch-all", and achieves automatic completion and tracing of missing regional units through merging cell expansion and table header hierarchy parsing.

[0031] The construction of the dimensional graph includes: resolving the unit into a linear combination of fundamental dimensional vectors, only when... Establish transformation edges at time, where as a unit A dimensional vector.

[0032] In the consistency verification step within the clause, the weighted satisfaction degree is calculated as follows: ; in As weight; This refers to the numerical tolerance threshold within the clause; For standardized relational predicates; For numerical terms The normalized value; For indicator functions; This is the deviation matrix within the clauses.

[0033] In the cross-source consistency comparison step, the source weight is calculated as follows: ,in For source level, The time window constant, The coefficient of variation is an exponential decay factor based on time difference; the consistency degree is calculated as follows: ; in Classified as abnormal; This is the abnormal threshold.

[0034] In the dimension guarding step of the generation phase, decoding is performed using logarithmic probability penalty constraints: ; in This is the penalty coefficient; Tolerance scale; To assert the value in the target unit; For evidence normalization; The most relevant span.

[0035] The numerical consistency score is calculated as follows: ; in For a set of numerical assertions; An indication exists for the reference.

[0036] The following is in conjunction with the appendix Figure 1 The specific embodiments of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the following description.

[0037] Example 1 This embodiment 1 provides a method for unit normalization and dimensional consistency verification of numerical clauses. This method achieves strong binding between numerical values, units, and evidence through joint text and table extraction, dimensional graph construction and conversion chain search, intra-clause and cross-source consistency verification, dimensional guarding during the generation stage, and re-retrieval correction, thereby improving dimensional accuracy, cross-source consistency, and auditability. The specific steps are as follows: Step S101: Joint extraction of numerical values ​​and units from text and tables The input is an electronic document (such as PDF or Word format) containing natural language text and tables. First, the document is parsed using layout analysis techniques (such as a deep learning-based document layout analysis model) to separate text paragraphs and table areas.

[0038] For the text portion: A cascaded extraction strategy is employed, using predefined regular expressions to match "value + unit" patterns (e.g., "10kV"), combined with a unit dictionary (covering variations such as "kilovolt," "kV," and "kV") for auxiliary matching. For complex or implicit numerical relationships, a sequence labeling model (such as a BERT-based CRF model or pointer network) is used for fine-grained identification, labeling the value, comparison operators (≥, ≤, >, <), and unit. Finally, structured numerical items are generated. : ; in It is a numerical value (which can be a scalar or an interval). As a unit, Let be the dimensional vector to be determined. For applicable conditions, This serves as a clause identifier. It also records character anchors. The position of the statement is used for tracing the source.

[0039] Table section: The logical structure is recovered through a table structure recognition algorithm, including merging and expanding merged cells and parsing multi-level headers. Unit determination follows the rule of "column header first, row header second, and catch-all within the cell." ; Numerical items are bound to table numbers, row numbers, and column numbers as evidence anchors.

[0040] After extraction, the units are normalized (e.g., "KV" is normalized to "kV") and the SI prefix multiplier is parsed. (e.g., k=10) 3 M=10 6 μ=10 6 This ultimately forms a set of numerical items. Includes units, initial values ​​of dimensions, location anchor points, and conditions. and the source tag RefTag.

[0041] Step S102: Construction of dimensional diagram and search of conversion chain The input is a set of numerical items. and target unit (e.g., commonly used units in the power industry such as V, A, and Hz). Construct a dimensional diagram. : Dimensional vector calculation: Dimensional vector calculation: Dimensioning units Analysis as a fundamental dimensional basis A power combination yields a dimensional vector. For example, power units Corresponding dimensional vector [1,2, 3,… ].

[0042] Transformable edges are established only when: Establishing edges ,in This is the multiplier factor. For affine terms (in most cases) (Except for nonlinear scaling such as dB).

[0043] For those that need to be normalized to Numerical items First, determine the consistency of dimensions: if If there is a conflict in the units of measurement, then the units of measurement must be clearly distinguished; otherwise, in the case of a conflict, the units of measurement must be clearly distinguished. The optimal transformation chain is searched upwards. The path cost function is: ; in The number of edges in the path. This is the upper bound of the boundary-level conversion error. These are the weighting coefficients. The cumulative parameters of the optimal path are: ; The normalized numerical calculation is as follows: ; Simultaneously, the conversion chain path is recorded for auditing purposes.

[0044] Step S103: Verification of consistency and constraints within the terms Input is the same clause and conditions The set of numerical items Normalize all values ​​to the target unit. The comparison agreement intervals are then normalized to half-open intervals. and predicate Consistency checks include: Numerical consistency: Calculating the deviation matrix The weighted satisfaction level is: ; in As importance weight, This is the tolerance threshold. This is an indicator function.

[0045] Logical consistency: Check that the intersection of intervals is not empty. ; Significant figures and tolerance: Verify the number of significant figures And deviation .

[0046] The consistency within the terms is determined as follows: ; Output standardized result quadruple And the reason code (such as out of tolerance, direction conflict, etc.).

[0047] Step S104: Cross-source consistency comparison and anomaly detection Input is a concept key A set of multi-source numerical items. Mapped via ontology. Synonyms are grouped into the same category. Then, assign source weights to each item: ; in The weighting is based on hierarchical levels (national standards > industry standards > enterprise standards). For source level, For time decay, This is the time window constant.

[0048] Robust statistical methods are employed: Central estimate: weighted median ; Divergence estimation: ; Abnormal score: ; in It is a numerically stable term.

[0049] Cross-source consistency is: ; in, This is the abnormal threshold. This is an indicator function.

[0050] The aggregated representative value is the weighted median: ; in The candidate representative value variable used to solve for the weighted median is the median candidate value searched over all possible values.

[0051] like Output consistent results; otherwise, retain the source of discrepancies for auditing. The above outputs will serve as constraint inputs for controlled generation and dimension guards, ensuring the consistency and traceability of numerical assertions across standard / version scenarios.

[0052] Step S105: "Dimensional Guardian" in the Generation Phase During text generation (such as LLM decoding), the current decoding state is monitored. Candidate digital token Implement triple protection: Evidence binding: through similarity function Select the most relevant evidence span ,and To obtain its unit Normalized value and quote tags.

[0053] Dimensional Guardian: Check ,like Then the unit will be forcibly rewritten or generation will be prohibited.

[0054] Conversion Guards: Requirements .

[0055] Relationship guard: Ensures that the output comparison relation is consistent with the evidence.

[0056] Apply a penalty to the decoding probability: ; in The penalty coefficient is... This is the tolerance scale. A rollback or re-search is triggered upon violation.

[0057] Step S106: Fusion of numerical consistency score and confidence level For generating answers Set of numerical assertions Calculate the consistency score: ; in An indication exists for the reference.

[0058] The total confidence level is fused as follows: ; in To retrieve the margin, For evidence coverage, For semantic entailment, For time-sensitive indication, For weights. If or This triggers a fix.

[0059] Step S107: Re-retrieval and adaptive correction Enter answer Evidence set ,score With threshold The target unit is denoted as The applicable conditions are as follows: Define the trigger indicator: ; in Indicates dimensional consistency. This is already bound evidence. If... First, perform unit unification rewriting: change the output units. The standard is or , and according to ; Recalculate the normalized value and comparison relationship. Then perform targeted supplementary evidence retrieval: generate a query based on missing fields and clause identifiers. By retrieval operator New evidence obtained , merged into ,exist Reselecting the most relevant span under constraints With the optimal transformation chain. Update candidates. Calculate new and ,like and Then adopt And output the bound Otherwise, proceed to the next round until the round limit is reached. For cross-source divergences, calculate the weighted proportion of anomalies. ; when Simultaneously generate an exception message indicating "cross-standard differences exist" and its source—version anchor. If the iteration terminates and the conditions are still met... or If a conservative statement (range-based / rejection) is output and the reason code is recorded, the final result will be auditably consistent in terms of dimensions, numerical values, and evidence.

[0060] Through the above steps, this invention achieves the following: establishing a traceable "value-unit-condition" ternary binding through joint text-table extraction and unit form normalization; determining the dimension before conversion using dimensional diagrams and the shortest conversion chain to obtain a unique and auditable normalized value; resolving contradictions within clauses through intra-clause consistency and interval / direction constraints; identifying standard / version differences through cross-source robust aggregation and anomaly identification; generating constraints and preventing incorrect dimensions and magic numbers during the decoding period; quantifying the reliability of conclusions through the fusion of numerical consistency scoring and confidence levels; and forming closed-loop convergence through targeted re-retrieval and adaptive correction. Thus, with minimal human intervention, it obtains a structured output that is dimensionally correct, cross-source consistent, auditable, and traceable, meeting the needs of engineering-level compliance verification.

[0061] Example 2 Please see Figure 2 As shown, this embodiment 2 also provides a system for implementing the method, including an extraction and binding unit 1, a dimensional determination and conversion unit 2, a consistency verification unit 3, a controlled generation and scoring fusion unit 4, and a re-retrieval and correction unit 5.

[0062] The extraction and binding unit 1 is used to perform joint extraction of numerical values ​​and units from text and tables.

[0063] The dimension determination and conversion unit 2 is used to construct a dimension graph and perform a conversion chain search.

[0064] The consistency verification unit 3 is used for intra-clause and cross-source consistency verification.

[0065] The controlled generation and scoring fusion unit 4 is used to implement dimension guarding and confidence fusion.

[0066] The re-retrieval and correction unit 5 is used to perform adaptive correction and closed-loop control.

[0067] The above units can be deployed in computing devices as software modules, hardware circuits, or a combination of software and hardware.

[0068] Example 3 like Figure 3 As shown, this embodiment 3 also provides an electronic device 700, which includes a processor 710, a memory 709, and a program or instructions stored in the memory 709 and executable on the processor 710. When the program or instructions are executed by the processor 710, they implement the various processes of the above-described embodiment of the unit normalization and dimensional consistency verification method for numerical terms, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0069] It should be noted that the first electronic device in the embodiments of the present invention includes the mobile electronic device and the non-mobile electronic device described above.

[0070] Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present invention.

[0071] The electronic device 700 includes, but is not limited to, components such as: radio frequency unit 701, network module 702, audio output unit 703, input unit 704, sensor 705, display unit 706, user input unit 707, interface unit 708, memory 709, and processor 710.

[0072] Those skilled in the art will understand that the electronic device 700 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 710 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0073] It should be understood that, in this embodiment of the invention, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042. The GPU 7041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 707 includes a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here. The memory 709 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 710 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understandable that the aforementioned modem processor may not be integrated into the processor 710.

[0074] This invention also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of the unit normalization and dimensional consistency verification method for numerical terms, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0075] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0076] This invention also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described embodiment of the unit normalization and dimensional consistency verification method for numerical terms, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0077] It should be understood that the chip mentioned in the embodiments of the present invention may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0078] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0079] Furthermore, it should be noted that the scope of the methods and systems in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0080] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the scope of protection of the present invention, and all of these forms are within the protection scope of the present invention.

Claims

1. A method for verifying unit normalization and dimensional consistency for numerical terms, characterized in that, Includes the following steps: By using a joint extraction unit of text and tables, numerical values, units, applicable conditions and source anchors are identified from the input document to generate a set of numerical items, where each numerical item includes a numerical value, unit, dimensional vector, applicable conditions and clause identifier. A dimensional graph is constructed based on the mapping relationship between units and dimensional vectors. This dimensional graph is represented as follows: ,in For unit sets, For the convertible relation edge, unit conversion is allowed only when the dimensions are equivalent; By minimizing the path cost function Select the optimal conversion chain to complete the unit normalization of the numerical values, where Let be the number of sides. This indicates the upper bound of the boundary-level conversion error. As weight, The source unit for the transformation edge. The target unit for the conversion edge; After normalizing the units of the numerical items under the same clause, the consistency conclusion and reason code within the clause are output based on the deviation matrix, weighted satisfaction degree, interval combinability and significant number rules. For multi-source numerical items under the same conceptual key, consistency, outlier score and representative value are calculated by weighting based on source level and effective time window, using weighted median and median absolute deviation. During the decoding process, dimensionality guards, conversion guards, and relational guards are applied to candidate numbers, and probability penalties are used to force numerical values ​​to be bound to evidence. Numerical consistency scores are calculated based on dimensional consistency rate, normalization error, and citation coverage, and then integrated with retrieval margin, evidence coverage, and semantic implication scores to form the total confidence score. When the total confidence score or numerical consistency score is below the threshold, unit consistency rewriting, targeted supplementary verification, and conversion chain reselection are triggered.

2. The method for verifying unit normalization and dimensional consistency for numerical clauses according to claim 1, characterized in that, In the joint extraction steps of text and table, the unit determination follows the rule of "column header first, row header second, and cell as a catch-all", and the automatic completion and tracing of regional unit omissions are achieved by merging cell expansion and parsing table header hierarchy.

3. The method for verifying unit normalization and dimensional consistency for numerical clauses according to claim 1, characterized in that, The construction of the dimensional graph includes: resolving the unit into a linear combination of fundamental dimensional vectors, only when... Establish transformation edges at time, where as a unit A dimensional vector.

4. The method for verifying unit normalization and dimensional consistency for numerical clauses according to claim 1, characterized in that, In the consistency verification step within the clause, the weighted satisfaction degree is calculated as follows: ； in As weight; This refers to the numerical tolerance threshold within the clause; For standardized relational predicates; For numerical terms The normalized value; For indicator functions; This is the deviation matrix within the clauses.

5. The method for verifying unit normalization and dimensional consistency for numerical clauses according to claim 1, characterized in that, In the cross-source consistency comparison step, the source weight is calculated as follows: ,in As for the source level, The time window constant, Based on time difference An exponential time decay factor is used to reduce the impact of stale sources on cross-source consistency computation; the consistency degree is calculated as follows: ； in Classified as abnormal; This is the abnormal threshold.

6. The method for verifying unit normalization and dimensional consistency for numerical clauses according to claim 1, characterized in that, In the dimension guarding step of the generation phase, decoding is performed using logarithmic probability penalty constraints: ； in This is the penalty coefficient; Tolerance scale; To assert the value in the target unit; For evidence normalization; The most relevant span.

7. The method for verifying unit normalization and dimensional consistency for numerical clauses according to claim 1, characterized in that, The numerical consistency score is calculated as follows: ； in For a set of numerical assertions; An indication exists for the reference.

8. A unit normalization and dimensional consistency verification system for implementing the method of any one of claims 1 to 7, characterized in that, include: Extraction and binding unit, used to perform joint extraction of values ​​and units from text and tables; The dimension determination and conversion unit is used to construct a dimension graph and perform a conversion chain search. Consistency verification unit, used for intra-clause and cross-source consistency verification; Controlled generation and scoring fusion unit, used to implement dimensionality guarding and confidence fusion; Then, the correction unit is retrieved to perform adaptive correction and closed-loop control.

Images