Unstructured report content extraction method based on knowledge graph

Abstract

This application relates to the field of data extraction technology, specifically to a method for extracting content from unstructured reports based on knowledge graphs. This method includes: extracting candidate columns from a two-dimensional image by converting its layout features into a one-dimensional signal; constructing a coordinate mapping relationship that conforms to physical monotonicity constraints using an ordinal-preserving regression algorithm, and quantifying the degree of local stretching as a local coordinate scaling factor; dynamically adjusting the matching cost using the local coordinate scaling factor, and searching for the global optimal solution using a dynamic programming algorithm; updating the original standard field width ratio to obtain the final width ratio, regenerating the baseline coordinate sequence, and triggering the entire process calculation. This application aims to introduce a nonlinear mapping and deformation weighting mechanism constrained by monotonicity, solving the frequent column misalignment problem in existing linear template matching techniques when processing non-uniformly distorted reports.

Description

Technical Field

[0001] This application relates to the field of data extraction technology, specifically to a method for extracting content from unstructured reports based on knowledge graphs. Background Technology

[0002] With the advancement of industrial digital transformation, automatic content extraction for various reports (such as quality inspection reports, logistics lists, and production records) has become a crucial step. These reports typically have fixed field layout standards, but in actual workflow, limitations such as printer roller wear, scanner mechanical paper feeding deviations, or damp and curled paper often result in non-linear geometric distortions in the digitized document images. Specifically, this manifests as fan-shaped distortion (one side sparse, the other dense) caused by slight differences in the paper feed roller speed, or localized non-uniform stretching caused by fluctuations in roller speed.

[0003] Existing technologies face numerous challenges when processing industrial reports. First, many industrial reports lack complete table borders, or the borders are broken due to poor scanning quality, rendering traditional line detection-based segmentation methods ineffective. Mainstream template matching methods based on absolute coordinates rely on the assumption that the document only undergoes globally uniform scaling or translation, but for local non-uniform stretching, linear models cannot accurately predict region offsets, especially in the middle or end of the report, where dense data columns are prone to misalignment with the standard template, leading to extraction errors. Summary of the Invention

[0004] In light of the above, it is necessary to provide a method for extracting unstructured report content based on knowledge graphs to solve the aforementioned problems.

[0005] One embodiment of this application provides a method for extracting unstructured report content based on knowledge graphs, the method comprising: For each report image to be processed, optical character recognition and orientation correction are performed to obtain the effective pixel width of the image. Within the effective pixel width range, the density spectrum corresponding to the horizontal coordinate of each pixel in the image is obtained. Combined with the text block content features corresponding to the preset range of each column of pixels in the image, candidate columns of the image are obtained. A pre-set ordered sequence of standard data fields is used. Based on the standard field width ratio associated with each standard data field and the field order, regression is performed on the set of all image candidate columns and the sequence of standard data fields to obtain a coordinate mapping polyline. Based on the changing trend of the coordinate mapping polyline, the local coordinate scaling factor is determined. Based on the image candidate columns and standard data fields, the differences between the actual and predicted locations and the differences between data types are analyzed. Combined with the local coordinate scaling factor, the matching difference metric is determined, a state matrix is ​​constructed, and dynamic programming is used to search for the optimal matching path of the image candidate columns. Along the optimal matching path of each document, analyze the positional deviation between the actual position and the predicted position of each successfully matched standard data field. Combined with the local coordinate scaling factor, update the original standard field width ratio to obtain the final width ratio, regenerate the reference coordinate sequence, and trigger the full-process calculation.

[0006] Specifically, obtaining the density spectrum corresponding to the horizontal coordinate of each pixel in the image involves: The position indicator function is set to 1 when the x-coordinate of each pixel in the image falls within the horizontal projection range of each text block; otherwise, it is set to 0. The product of the position indicator function value of the x-coordinate of each pixel in the image falling on each text block, the height of each text block, and the recognition confidence obtained by OCR for each text block is calculated. The products of the x-coordinate of each pixel in the image corresponding to all text blocks are accumulated to obtain the density spectrum corresponding to the x-coordinate of each pixel in the image.

[0007] Specifically, the image candidate column is the column corresponding to the x-coordinate of the maximum point after the density spectrum has been smoothed.

[0008] Specifically, obtaining the coordinate mapping polyline involves: Add the standard field width ratios of all standard data fields preceding each standard data field, and then add the standard field width ratio corresponding to each standard data field with a preset weight to obtain the standard center position ratio of each standard data field. When the dominant data type of the image candidate column is completely consistent with the preset strong feature of the standard data field, the coordinate combination composed of the standard field width ratio and the standard center position corresponding to the image candidate column is established as a feature alignment anchor point. If the number of feature alignment anchor points is less than the preset value, a linear mapping model is used to obtain a coordinate mapping polyline; otherwise, an ordinal regression combined with piecewise linear interpolation is used to obtain a coordinate mapping polyline.

[0009] Specifically, determining the local coordinate scaling factor involves: If a slope exists on the coordinate mapping polyline for each standard center position, the slope is used as the corresponding local coordinate scaling factor; otherwise, the maximum slope of the left and right segments of each standard center position in the coordinate mapping polyline is used as the corresponding local coordinate scaling factor.

[0010] Specifically, the matching difference metric is determined by the sum of two parts: content incompatibility penalty and position deviation penalty. Among them, the first The candidate columns of the image are matched to the first... The positional deviation penalty for each standard data field is denoted as... The calculation formula is as follows: In the formula, Indicates the center coordinates of the j-th candidate column of the image; Represents the prediction center coordinates of the k-th standard data field; This represents the standard center position ratio of the k-th standard data field; Indicates the preset smoothing coefficient; This represents the local coordinate scaling factor for the k-th standard data field; This represents the preset minimum scaling threshold; max() represents the maximum value function. If the first The dominant data type of the first image candidate column is the same as that of the second. If the preset strong features of a standard data field do not match, the penalty for content incompatibility is set to [value]. Twice the theoretical maximum value; otherwise, the content incompatibility penalty is 0.

[0011] The state transition equation for dynamic programming search is as follows: In the formula, Indicates the previous Each standard data field successfully matched the previous one. The nth image candidate column, and the nth The first standard data field exactly matches the first... Minimum cumulative cost when there are 10 candidate columns of images; Indicates the previous Each standard data field successfully matched the previous one. The nth image candidate column, and the nth The first standard data field exactly matches the first... The minimum cumulative cost when considering 10 candidate image columns; min{} represents the minimum value function; Indicates the first The candidate columns of the image are matched to the first... The matching difference measure of each standard data field, where i and j represent the index of the candidate image column.

[0012] The process of obtaining the optimal matching path is as follows: select the node with the smallest matching difference metric value from the last standard data field as the endpoint, and backtrack along the predecessor node index of the record to obtain the optimal matching path; each node is specifically the element corresponding to the corresponding standard data field in the state matrix.

[0013] The specific steps for updating the original standard field width ratio to obtain the final width ratio are as follows: Along the optimal matching path, the negative correlation mapping result between the local coordinate scaling factor and the maximum value of the preset minimum scaling threshold is used as the statistical weight of the corresponding standard data field. The position deviation values ​​of all successfully matched standard data fields are weighted and summed to obtain the batch weighted deviation mean of each standard data field. The batch weighted deviation mean is converted into a logical ratio adjustment amount and superimposed on the original standard field width ratio to generate a temporary correction value. After normalization, the final width ratio of each standard data field is obtained.

[0014] Specifically, the condition for updating the original standard field width ratio is that the absolute value of the batch-weighted deviation mean of each standard data field is greater than a preset ignore threshold.

[0015] This application has at least the following beneficial effects: This application first constructs a coordinate mapping polyline through ordinal-preserving regression and interpolation, which can accurately fit the stretching or compression trend of an image as its position changes. Compared to linear models, this nonlinear fitting ensures that the predicted coordinates can follow the local deformation of the image, enabling the extraction system to maintain the alignment of data columns and fields even at the end of the document or in severely deformed areas.

[0016] Secondly, by incorporating the local coordinate scaling factor into the cost function of dynamic programming, adaptive matching with tolerance in the stretching region and strictness in the compression region is achieved. This mechanism effectively prevents large geometric deviations caused by local stretching from blocking the search for the globally optimal path, significantly improving the robustness of the matching algorithm in complex distortion scenarios.

[0017] Finally, through weighted calibration and normalization updates based on scaling factors, systematic environmental errors can be accurately extracted from batch data while suppressing the effects of stretching noise. This enables the system to automatically adapt to mechanical aging of the scanning equipment or environmental changes without human intervention, maintaining high-precision extraction capabilities over the long term. Attached Figure Description

[0018] Figure 1 A flowchart of the knowledge graph-based unstructured report content extraction method provided in this application; Figure 2 This is a schematic diagram illustrating the updating of the original standard field width ratio provided in this application. Detailed Implementation

[0019] In the description of the embodiments in this application, the words "exemplary," "or," and "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary," "or," and "for example" is intended to present the relevant concepts in a specific manner.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0021] It should also be noted that the methods disclosed in the embodiments of this application or the methods shown in the flowcharts include one or more steps for implementing the method. Without departing from the scope of protection of this application, the execution order of multiple steps can be interchanged, and some steps can also be deleted.

[0022] This application proposes a method for extracting unstructured report content based on knowledge graphs, applied in the field of data extraction technology. (See attached document.) Figure 1 The method includes the following steps: S1: Perform optical character recognition and orientation correction on each report image to be processed, obtain the effective pixel width of the image, and within the effective pixel width range, obtain the density spectrum corresponding to the horizontal coordinate of each pixel in the image. Combine the text block content features corresponding to the preset range of each column of pixels in the image to obtain the candidate columns of the image.

[0023] In the digitization process of unstructured reports, due to mechanical scanning or printing quality issues, report images often lack clear table borders, or the borders are broken due to improper binarization threshold settings, causing traditional layout analysis methods based on line detection to fail. To accurately locate data regions without borders, this application calculates the distribution density of text pixels, transforming the layout features of a two-dimensional image into a one-dimensional signal, thereby extracting discrete candidate column objects as the basis for subsequent structured matching.

[0024] The scenario assumed in this application is as follows: although the report image to be processed has geometric distortion or positional offset, its column structure is basically complete. That is, the number of data columns in the image roughly corresponds to the number of fields in the standard template. The main problem lies in the nonlinear offset and local deformation of the position, rather than a large number of missing or disorderly proliferation of columns. This assumption provides reasonable boundary conditions for the subsequent dynamic programming algorithm.

[0025] S101: Image preprocessing and coordinate system normalization.

[0026] Before performing column analysis on the report, it is necessary to eliminate geometric deviations generated during the scanning process and establish a unified coordinate baseline. Due to paper feed deviations, the text lines in the original image may be tilted overall, and direct vertical projection will result in blurred column spacing.

[0027] Specifically, first, the set of raw text blocks output by the Optical Character Recognition (OCR) engine is obtained; then, a histogram of the angle distribution of the center lines of all text blocks in the entire document is plotted, and the angle corresponding to the frequency peak is selected as the overall tilt angle of the document. Using rotation matrices to adjust the vertex coordinates of all text blocks. Perform a reverse rotation transformation to correct the document to a horizontal orientation.

[0028] Next, in order to eliminate black border interference at the edges of the scanned image, this application crops the blank or noisy areas around the image based on the corrected text block distribution boundaries, and resets the upper left corner of the cropped image to the coordinate origin. In this coordinate system, calculate the effective pixel width of the image in the horizontal direction. , specifically This is the width of the minimum bounding rectangle that includes at least 95% of the center of the OCR text block after correction. Parameter It plays a crucial role, not only defining the range of pixel coordinate values. It will also serve as the base denominator for subsequent connections between the pixel domain (physical space) and the logical domain (normalized space).

[0029] S102: Vertical alignment density spectrum generation.

[0030] To quantify the horizontal density distribution of report data, discrete text blocks need to be mapped to a continuous distribution signal. Since data in the same column are approximately aligned vertically, their horizontal projection density will inevitably exhibit local high values.

[0031] Specifically, along the horizontal axis Calculate the longitudinally aligned density spectrum function .for arrive Any x-coordinate within the range Its density spectrum Defined as the weighted sum of the valid text blocks covered at the current position. The calculation formula is as follows: In the formula, This represents a single text block in set B.

[0032] This indicates a position indicator function. Specifically, it means: when the x-coordinate Falling on text block Horizontal projection range The value is 1 if the text is inside the pixel, and 0 otherwise. This function is used to determine whether there is text covering the current pixel position.

[0033] The height (in pixels) of the text block is used to increase the weight of multi-line text (such as long description fields) in the density signal, because a higher text block usually means that the content of the column is richer, and its confidence as the column center should be higher.

[0034] This represents the recognition confidence level (value from 0 to 1) of the OCR engine output, used to suppress the influence of misidentified noise. The lower the confidence level (which may be noise or dirt), the smaller its contribution to the density spectrum.

[0035] Using the formula above, the original two-dimensional text distribution is transformed into a one-dimensional waveform signal, namely the density spectrum. The peak position corresponds to the potential data column center.

[0036] S103: Image candidate column location and type determination.

[0037] To extract specific column objects from a continuous density spectrum signal, it is necessary to eliminate minute jitter in the signal and identify significant distribution features. Due to font size differences or scan noise, the original density spectrum... It may contain high-frequency spikes, and directly taking the extreme value can easily lead to false detections.

[0038] Specifically, a one-dimensional Gaussian filter is first used to analyze the density spectrum. Smoothing is then performed. Subsequently, all local maxima of the smoothed density spectrum are extracted, and their x-coordinates are recorded as the coordinates of the image column centers. subscript Indicates the detected number Image candidate columns.

[0039] For each defined center coordinate The search proceeds left and right from this point until the density value drops to a preset percentage of the peak value (30% in this embodiment), thus defining the effective width range of the image column. Within this range, the content features of all text blocks falling into this area are statistically analyzed, and the candidate image column is determined. The dominant data type.

[0040] To ensure the accuracy of subsequent anchor point mining, this application adopts the following dominant type determination logic: First, multiple types of regular expression rules are preset, including pure numbers, date formats YYYY-MM-DD, monetary formats, and Chinese text in this embodiment; second, for each text block within the column range, attempts are made to match the above regular expression rules; finally, the number of successful matches for each rule is counted. If the matching percentage of a certain type of regular expression rule (such as date format) exceeds a preset threshold (set to 70% in this embodiment), the dominant data type of the column is marked as that specific type (such as date type); otherwise, if no rule has a significant percentage, it is marked as a mixed / unknown type.

[0041] After the above processing, the output is a set of image candidate columns sorted by coordinates from smallest to largest. This set contains the center coordinates of each column. With clearly defined dominant type attributes, this data will serve as the physical basis for subsequently establishing coordinate mapping relationships.

[0042] S2: A pre-set ordered sequence of standard data fields is used. Based on the standard field width ratio associated with each standard data field and the field order, regression is performed on the set of all image candidate columns and the sequence of standard data fields to obtain a coordinate mapping polyline. Based on the changing trend of the coordinate mapping polyline, the local coordinate scaling factor is determined.

[0043] This step in this application, based on the obtained set of candidate image columns and the standard field configuration from the structured field configuration library (knowledge graph), solves the matching misalignment problem caused by nonlinear distortion. It uses an order-preserving regression algorithm to construct a coordinate mapping relationship that conforms to the physical monotonicity constraint and quantifies the degree of local stretching as a scaling factor, thereby providing an adaptive adjustment basis for subsequent dynamic matching.

[0044] S201: Standard reference coordinate calculation.

[0045] To achieve matching between the standard template and the current image, the abstract field width parameter first needs to be transformed into measurable position coordinates. Due to the resolution differences between different images, directly using absolute pixel values ​​is not universal. Therefore, this application constructs a standard position reference within a normalized logical space.

[0046] Specifically, a pre-defined ordered sequence of standard data fields is provided. The k-th standard data field is associated with a standard field width ratio. (satisfies the normalization condition) The center position of each field is calculated iteratively based on the field order. For the ... The standard center position ratio of each standard data field The calculation formula is: In the formula: This represents the standard field width ratio associated with the m-th standard data field.

[0047] This calculation ensures the generated coordinate sequence exist The interval is strictly monotonically increasing and reflects the close arrangement of each field in the logical space.

[0048] S202: Feature Anchor Mining and Linear Backoff Strategy.

[0049] In order to establish the mapping relationship, this application needs to find several reliable corresponding points, i.e. feature alignment anchor points, between the standard logical space and the image pixel space.

[0050] Specifically, traverse the image candidate column set. With standard data field sequence Based on the dominant data type, this application applies a strong feature rule base for matching. Matching is only performed when the image candidate column... Dominant type and standard field When the preset strong features (such as a specific date format YYYY-MM-DD or header keywords) are completely consistent, the coordinate pair will be combined. Established as a feature alignment anchor point .

[0051] To ensure robustness, a linear backoff strategy is adopted: if the number of mined anchor points is less than 2 (unable to fit a nonlinear curve), it is determined that the current image distortion is small or the features are insufficient, and a linear mapping model is directly adopted (i.e., assuming that the image only undergoes global uniform scaling), and the subsequent mapping function is: And skip S203, directly proceeding to S204. Here, u represents the logical space coordinate ratio. Indicates the effective pixel width of the image. This represents the predicted pixel coordinates.

[0052] S203: Coordinate mapping based on ordinal-preserving regression and interpolation.

[0053] Due to scanner mechanical feed deviations or paper deformation, the distribution of anchor points is often non-linear. However, the order of the table columns on the image has strict monotonicity, i.e., the order of the columns is... The pixel coordinates of the column must be greater than those of the first column. To fit this trend, this application employs isotonic regression combined with piecewise linear interpolation.

[0054] The specific implementation steps are as follows: (1) Order-preserving regression processing: The collected anchor point set The input is fed into the order-preserving regression algorithm. In this embodiment, the PAVA algorithm is used. This algorithm continuously merges adjacent points that violate monotonicity (i.e., the column pixel coordinates of the later point are less than those of the previous point), and outputs a set of corrected regression points that strictly satisfy the non-decreasing constraint. This step ensures that the mapping relationship conforms to physical laws and avoids erroneous predictions due to reversed column order.

[0055] (2) Piecewise Linear Interpolation: Since the output of ordinal-preserving regression is discrete points, in order to obtain a continuous mapping function, this application performs piecewise linear interpolation between adjacent regression points. That is, for any two adjacent regression points... and Logical position between The predicted pixel coordinates are calculated using a linear equation.

[0056] After the above processing, a continuous coordinate mapping polyline is generated. The curve accepts a logical position ratio. As input, the output is the predicted image pixel coordinates. .

[0057] S204: Calculation of local coordinate scaling factor.

[0058] To quantify the degree of stretching in different regions of the image, based on the generated continuous polylines... Calculate the local coordinate scaling factor .

[0059] This sub-step provides an engineering definition for the calculation of derivatives to address the issue of non-differentiability of polylines at nodes. The specific calculation logic is as follows: (1) Calculation of the slope of the line segment: For the standard center position First, determine where it falls. On which line segment is it located, and calculate the geometric slope of that line segment as the local coordinate scaling factor for the corresponding standard center position.

[0060] (2) Normalization: Divide the geometric slope by the total width of the image. Eliminate the impact of resolution.

[0061] (3) Special handling of nodes: If It is located exactly at the connection node (inflection point) of the two line segments (i.e., the slope does not exist). Calculate the slope of the left line segment and the slope of the right line segment respectively, and take the maximum value of the two as the local coordinate scaling factor of that position.

[0062] The final local coordinate scaling factor It has a clear physical meaning: when When, it indicates that the image in that region is stretched (the pixel distribution is sparser than the standard template); when When this occurs, it indicates that the image in that region is compressed (the pixel distribution is more compact than the standard template).

[0063] This local coordinate scaling factor will be used to dynamically adjust the matching algorithm's tolerance for positional deviations.

[0064] S3: Based on the image candidate columns and standard data fields, analyze the differences between the actual and predicted positions and the differences between data types. Combined with the local coordinate scaling factor, determine the matching difference metric, construct the state matrix, and use dynamic programming to search for the optimal matching path of the image candidate columns.

[0065] This application analyzes the candidate column set of the image, as well as the coordinate mapping polyline and the local coordinate scaling factor. Its core task is to establish a one-to-one matching relationship between the image columns and the standard fields. To solve the matching failure problem caused by non-uniform stretching, this application uses the local coordinate scaling factor to dynamically adjust the matching cost and uses the dynamic programming (DP) algorithm to search for the global optimum.

[0066] S301: Construction of dynamic cost matrix based on scaling factor.

[0067] Before performing a matching search, it is necessary to quantify the matching differences between any candidate image column and any standard data field. To do this, a... The cost matrix, where the first... Line number The elements of the column represent the first Image candidate columns Match to the Standard data fields Matching difference metric .

[0068] This metric is determined by the sum of two parts: content incompatibility penalty and position deviation penalty. The formula for calculating the position deviation penalty is as follows: In the formula, Indicates the center coordinates of the j-th candidate column of the image; Represents the prediction center coordinates of the k-th standard data field; This represents the standard center position ratio of the k-th standard data field; This represents the preset smoothing coefficient, which is set to 50 pixels in this embodiment; This represents the local coordinate scaling factor for the k-th standard data field; This represents the preset minimum scaling threshold, which is 0.1 in this embodiment; max() represents the maximum value function.

[0069] It should be understood that the molecular part This represents the geometric distance between the actual pixel coordinates of the image column and the predicted coordinates of the mapped curve. The smaller this distance, the closer the positions. A local coordinate scaling factor is introduced in the denominator, and to prevent division by zero errors caused by excessive image compression leading to a local coordinate scaling factor approaching 0, a preset minimum scaling threshold is introduced. Ensure that the denominator always has numerical stability.

[0070] In the area where the image is stretched, i.e. When the value is large, the positional error of the same pixel distance will be diluted by the larger denominator in the formula, thus calculating a smaller penalty value. This means that the algorithm has a higher tolerance for positional deviations in the stretched area; in areas where the image is compressed or normal, i.e. When the value is small, the denominator is small, and the penalty value is sensitive to positional error, which means that the algorithm maintains strict positional constraints in this region.

[0071] In addition, the smoothing coefficient is used to control the normalization scale of the positional deviation on the total cost.

[0072] The content incompatibility penalty is determined based on the dominant data type of the column: if the types do not match (e.g., date matches amount), a large fixed penalty value is assigned. In this example, it is set to... Twice the theoretical maximum value, the specific formula is: Otherwise, the value is 0. It should be noted that if the candidate column of the image is determined to be of "mixed / unknown type" (e.g., inaccurate OCR recognition or messy content), in order to ensure the robustness of the algorithm, the content incompatibility penalty is usually set to 0, allowing it to be matched by position features.

[0073] S302: Optimal path search based on dynamic programming.

[0074] After constructing the cost matrix, a dynamic programming algorithm is used to search for a matching path with the minimum total cost, thereby determining the final correspondence.

[0075] The specific implementation steps are as follows: (1) State definition: Let Indicates the previous Each standard data field successfully matched the previous one. The nth image candidate column, and the nth The first standard data field exactly matches the first... Minimum cumulative cost when there are 10 candidate columns of images.

[0076] (2) State transition: In order to calculate It needs to start from the previous line (i.e., the first line). To find the optimal predecessor node among all possible matching positions of (a set of standard data fields), the corresponding state transition equation is: In the formula, Indicates the previous Each standard data field successfully matched the previous one. The nth image candidate column, and the nth The first standard data field exactly matches the first... The minimum cumulative cost when considering 10 candidate image columns; min() represents the minimum value function; Indicates the first The candidate columns of the image are matched to the first... The matching difference measure of each standard data field.

[0077] It should be understood that the specific meaning of this equation is: the minimum cumulative cost of the current step is equal to the minimum value among all legal states in the previous step plus the matching cost of the current step itself. The constraints are as follows: This ensures that the matching order of the image columns must be from left to right, conforming to the physical layout rules.

[0078] (3) Backtracking extraction: according to arrive After iterating through all the cumulative minimum costs in sequence, start from the last row (the first row) The node with the smallest difference metric value among the standard data fields is selected as the endpoint. Then, the process is reversed along the predecessor node index of the record to reconstruct the optimal matching path with the minimum global total cost.

[0079] Finally, based on the correspondence determined by this path, the corresponding text content is extracted from the image candidate columns and output as structured business data. Simultaneously, the matching points on this path will also be used for closed-loop calibration later.

[0080] S4: Along the optimal matching path of each document, analyze the positional deviation between the actual position and the predicted position of each successfully matched standard data field, and update the original standard field width ratio in combination with the local coordinate scaling factor to obtain the final width ratio. Regenerate the reference coordinate sequence and trigger the full-process calculation.

[0081] This step in the application aims to leverage the systematic nature of document sharing within the same processing batch to eliminate inherent environmental errors introduced by factors such as scanner optical distortion, printer mechanical wear, or template version fine-tuning. To distinguish between random noise and systematic errors, this application uses the local coordinate scaling factor as a measure of data confidence and adds normalization constraints to ensure closed-loop stability.

[0082] S401: Extraction of position deviation numerical values.

[0083] After matching a single document and extracting structured text data, diagnostic data is also collected. Along the optimal matching path, standard data fields for each successful match are calculated. Positional deviation value The specific formula is as follows: In the formula: This indicates that in the optimal matching path, it is related to the th The first standard data field that matches The actual center coordinates of each candidate column of images; Represents the prediction center coordinates of the k-th standard data field; This represents the standard center position ratio of the k-th standard data field.

[0084] This value Stripped of the The explained nonlinear distortion components retain only the residual deviations between the current document features and the standard template parameters. Aggregate all features in the current processing batch. The deviation data generated from these documents will be used for subsequent statistical analysis.

[0085] S402: Weighted bias statistics based on scaling factor.

[0086] In order to extract stable systematic biases from data containing random noise, this application performs weighted statistics and uses local coordinate scaling factors to evaluate the reliability of each bias data point.

[0087] Specifically, for the first batch within the batch The first document Calculate the statistical weight of each standard data field. To suppress noise in the stretching region and prevent numerical overflow, the weight calculation formula is as follows: In this formula: The local scaling factor at that location, when When the area is large (stretched region), the pixel distribution is sparse and the positioning random error is large. Therefore, the calculated weight is small, which plays a role in suppressing noise. A preset minimum scaling threshold (0.1 in this embodiment) is used to prevent scaling when the image is severely compressed. When the weight approaches 0, the weight tends to infinity, leading to statistical distortion.

[0088] Based on this weight, calculate the... Batch-weighted mean deviation of each standard data field The specific formula is as follows: In the formula, B represents the number of copies of the document; Indicates the first in the batch The first document Statistical weights of standard data fields; This represents the standard data field for each successfully matched document in the i-th document within the batch. The positional deviation value.

[0089] Through this weighted calculation, the system effectively filters out unreliable data in the high-tension region and obtains a more accurate estimate that reflects the systematic offset of the equipment.

[0090] S403: Parameter correction and normalization update.

[0091] When the calculated weighted mean deviation When the value exceeds the preset ignore threshold (2 pixels in this embodiment), it indicates that there is a significant systematic offset between the current batch of documents and the preset standard template. At this time, this application performs parameter correction and update.

[0092] First, the pixel-level deviation is converted into a logical scaling adjustment and then added to the original standard field width ratio. Generate temporary correction values. The specific formula is as follows: In the formula, The average effective pixel width of documents within the batch. To update the step size coefficient (ranging from 0.5 to 1.0), which is used to control the stability of the correction, Indicates the first Batch-weighted mean deviation of each standard data field; This represents the standard field width ratio associated with the k-th standard data field.

[0093] Because independent modifications to each standard data field will cause the sum of the width ratios to no longer equal 1 (i.e., Using this parameter directly will cause subsequent coordinate system logic to crash. Therefore, a normalization update operation must be performed: This step ensures the corrected final width ratio. The sum of these values ​​strictly regresses to 1.

[0094] Finally, this application utilizes The original parameters are replaced, and the baseline coordinate calculation is retried (S201) to generate a new coordinate sequence adapted to the features of the current batch, and the full extraction process is executed again. A maximum recalculation threshold is set, which is set to 2 in this embodiment. When the number of times the full extraction process is retried reaches this threshold, the correction is forcibly terminated and the current optimal matching path is output. Since the new coordinate baseline has compensated for systematic deviations and has undergone normalization constraints, the re-extraction process will achieve extremely high matching accuracy.

[0095] The diagram illustrating the update of the original standard field width ratio is shown below. Figure 2 As shown.

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description; sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0097] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for extracting content from unstructured reports based on knowledge graphs, characterized in that: The method includes the following steps: For each report image to be processed, optical character recognition and orientation correction are performed to obtain the effective pixel width of the image. Within the effective pixel width range, the density spectrum corresponding to the horizontal coordinate of each pixel in the image is obtained. Combined with the text block content features corresponding to the preset range of each column of pixels in the image, candidate columns of the image are obtained. A pre-set ordered sequence of standard data fields is used. Based on the standard field width ratio associated with each standard data field and the field order, regression is performed on the set of all image candidate columns and the sequence of standard data fields to obtain a coordinate mapping polyline. Based on the changing trend of the coordinate mapping polyline, the local coordinate scaling factor is determined. Based on the image candidate columns and standard data fields, the differences between the actual and predicted locations and the differences between data types are analyzed. Combined with the local coordinate scaling factor, the matching difference metric is determined, a state matrix is ​​constructed, and dynamic programming is used to search for the optimal matching path of the image candidate columns. Along the optimal matching path of each document, analyze the positional deviation between the actual position and the predicted position of each successfully matched standard data field. Combined with the local coordinate scaling factor, update the original standard field width ratio to obtain the final width ratio, regenerate the reference coordinate sequence, and trigger the full-process calculation.

2. The method for extracting unstructured report content based on knowledge graphs as described in claim 1, characterized in that, The specific steps for obtaining the density spectrum corresponding to the horizontal coordinate of each pixel in the image are as follows: The position indicator function is set to 1 when the x-coordinate of each pixel in the image falls within the horizontal projection range of each text block; otherwise, it is set to 0. The product of the position indicator function value of the x-coordinate of each pixel in the image falling on each text block, the height of each text block, and the recognition confidence obtained by OCR for each text block is calculated. The products of the x-coordinate of each pixel in the image corresponding to all text blocks are accumulated to obtain the density spectrum corresponding to the x-coordinate of each pixel in the image.

3. The method for extracting unstructured report content based on knowledge graphs as described in claim 1, characterized in that, The image candidate column is specifically the column corresponding to the x-coordinate of the maximum point after the density spectrum has been smoothed.

4. The method for extracting unstructured report content based on knowledge graphs as described in claim 1, characterized in that, The obtained coordinate mapping polyline is specifically as follows: Add the standard field width ratios of all standard data fields preceding each standard data field, and then add the standard field width ratio corresponding to each standard data field with a preset weight to obtain the standard center position ratio of each standard data field. When the dominant data type of the image candidate column is completely consistent with the preset strong feature of the standard data field, the coordinate combination composed of the standard field width ratio and the standard center position corresponding to the image candidate column is established as a feature alignment anchor point. If the number of feature alignment anchor points is less than the preset value, a linear mapping model is used to obtain a coordinate mapping polyline; otherwise, an ordinal regression combined with piecewise linear interpolation is used to obtain a coordinate mapping polyline.

5. The method for extracting unstructured report content based on knowledge graphs as described in claim 4, characterized in that, The determination of the local coordinate scaling factor specifically involves: If a slope exists on the coordinate mapping polyline for each standard center position, the slope is used as the corresponding local coordinate scaling factor; otherwise, the maximum slope of the left and right segments of each standard center position in the coordinate mapping polyline is used as the corresponding local coordinate scaling factor.

6. The method for extracting unstructured report content based on knowledge graphs as described in claim 1, characterized in that, The matching difference metric is specifically determined by the sum of two parts: content incompatibility penalty and positional deviation penalty. Among them, the first The candidate columns of the image are matched to the first... The positional deviation penalty for each standard data field is denoted as... The calculation formula is as follows: In the formula, Indicates the center coordinates of the j-th candidate column of the image; Represents the prediction center coordinates of the k-th standard data field; This represents the standard center position ratio of the k-th standard data field; Indicates the preset smoothing coefficient; This represents the local coordinate scaling factor for the k-th standard data field; This represents the preset minimum scaling threshold; max() represents the maximum value function. If the first The dominant data type of the first image candidate column is the same as that of the second. If the preset strong features of a standard data field do not match, the penalty for content incompatibility is set to [value]. Twice the theoretical maximum value; otherwise, the content incompatibility penalty is 0.

7. The method for extracting unstructured report content based on knowledge graphs as described in claim 1, characterized in that, The state transition equation for dynamic programming search is as follows: In the formula, Indicates the previous Each standard data field successfully matched the previous one. The nth image candidate column, and the nth The first standard data field exactly matches the first... Minimum cumulative cost when there are 10 candidate columns of images; Indicates the previous Each standard data field successfully matched the previous one. The nth image candidate column, and the nth The first standard data field exactly matches the first... Minimum cumulative cost when there are 10 candidate columns of images; min{} represents the minimum value function; Indicates the first The candidate columns of the image are matched to the first... The matching difference measure of each standard data field, where i and j represent the index of the candidate image column.

8. The method for extracting unstructured report content based on knowledge graphs as described in claim 7, characterized in that, The process of obtaining the optimal matching path is as follows: select the node with the smallest matching difference metric value from the last standard data field as the endpoint, and backtrack along the predecessor node index of the record to obtain the optimal matching path; each node is specifically the element corresponding to the corresponding standard data field in the state matrix.

9. The method for extracting unstructured report content based on knowledge graphs as described in claim 1, characterized in that, The specific steps for updating the original standard field width ratio to obtain the final width ratio are as follows: Along the optimal matching path, the negative correlation mapping result between the local coordinate scaling factor and the maximum value of the preset minimum scaling threshold is used as the statistical weight of the corresponding standard data field. The position deviation values ​​of all successfully matched standard data fields are weighted and summed to obtain the batch weighted deviation mean of each standard data field. The batch weighted deviation mean is converted into a logical ratio adjustment amount and superimposed on the original standard field width ratio to generate a temporary correction value. After normalization, the final width ratio of each standard data field is obtained.

10. The method for extracting unstructured report content based on knowledge graphs as described in claim 9, characterized in that, The specific condition for updating the original standard field width ratio is that the absolute value of the batch-weighted deviation mean of each standard data field is greater than a preset ignore threshold.

Images