Structural Information Optimization Method for Document Content Recognition

Abstract

The present invention provides a structural information optimization method for document content recognition, comprising the following steps: Step 1, obtaining a picture of a monopoly license, and extracting the text boxes, coordinates, and contents of all text areas in the picture based on a deep learning model. The present invention dynamically clusters the text box coordinates through the K-Means spatial clustering algorithm, and cooperates with the column translation and scrolling mechanism to solve the text dislocation problem caused by printing equipment errors, etc., and ensures the accurate correspondence between fields and values ​​during information extraction; with the help of PaddleOCR customized training to improve text detection accuracy, combined with Sentence-BERT to build a semantic matching model and semantic coherence verification, solve the information missing problem caused by printing missing, etc., and ensure the accuracy and efficiency of information management and supervision; by abandoning the traditional fixed position method, adopting dynamic clustering and sorting, semantic + spatial dual matching, adapting to different layouts and dislocation scenarios, and solving the problem of printing dislocation that is difficult to handle with existing technologies.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a structural information optimization method for document content recognition. Background Art

[0002] A monopoly license is an important legal document issued by the monopoly administrative department in accordance with the law. It carries the legal credentials of market entities engaging in monopoly business activities. The information recorded includes key structured content such as the company name (identifying the market entity), the name of the person in charge (clarifying the responsible party), the company type (defining the nature of the organization), the business location (limiting the geographical scope of business), the scope of the license (specifying the type of business), the supplier (clarifying the source of supply), the validity period (setting the duration of the rights), the issuing authority (indicating the entity that issued the license), and the date of issuance (recording the time the document was generated). This information is the core basis for implementing market supervision, conducting business activity verification, and conducting industry data management. However, certain problems still exist:

[0003] 1. In actual applications, due to issues such as printing equipment errors, paper position deviation, and shooting angle, the text on the license may be significantly misplaced, resulting in the inability to accurately match the corresponding entry when extracting information;

[0004] Second, there are cases where missing information is due to missing printing, blurred images, or OCR misrecognition, which cannot be directly corrected in order, affecting the accuracy and efficiency of subsequent information management and supervision;

[0005] Third, existing technologies usually rely on fixed positions or traditional OCR methods, which are difficult to effectively handle printing misalignment;

[0006] Therefore, a structural information optimization method for document content recognition is proposed. Summary of the Invention

[0007] In view of this, embodiments of the present invention hope to provide a structural information optimization method for document content identification to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0008] To solve the above technical problems, a technical solution adopted by this application is: a structural information optimization method for document content recognition, comprising the following steps:

[0009] Step 1: Obtain an image of the monopoly license and extract the text boxes, coordinates, and content of all text areas in the image using a deep learning model.

[0010] Step 2: Process all the extracted text boxes, delete the text boxes belonging to the license name, and use the K-Means spatial clustering algorithm to assign the remaining text boxes to the left column and the right column according to their coordinates;

[0011] Step 3: Based on each column of clustered text boxes, sort the text boxes from top to bottom according to their y values ​​to obtain two columns of orderly arranged text boxes and their text contents;

[0012] Step 4: Within each obtained column, the vertical distance between adjacent text boxes is determined. If the y-axis distance between the two is lower than the set threshold, the text contents are merged into a continuous part of the same field.

[0013] Step 5: Based on Sentence-BERT, extract field pairs from license images collected on-site, build a semantic matching dataset, and train a semantic matching model;

[0014] Step 6: Use the left and right columns as a pairing of field names and values, calculate the semantic similarity between the field names and the candidate values, and select the one with the highest semantic similarity as the initial match;

[0015] Step 7: If the similarity exceeds the threshold, shift the second column with the matching behavior anchor point to align with the matching starting point;

[0016] Step 8: Match subsequent fields row by row. If the matching fails continuously, scroll to the second column and match again, and output the successfully matched field pairs.

[0017] As a further preferred embodiment of the present technical solution, in step 2, the specific steps of allocating the remaining text boxes to the left column and the right column according to their coordinates are as follows:

[0018] Step 201: Set the number of cluster centers K = 4, and calculate the x-coordinate range Δx = max(x i )-min(x i ), select the initial cluster center C={C1,C2,C3,C4} based on Δx, where the x coordinates of C1 and C2 are located in [min(x i ),min(x i )+Δx / 3] interval, the x coordinates of C3 and C4 are located in [max(x i )-Δx / 3,max(x i )] interval, the y-coordinate of C1-C4 takes the average y-value of set T;

[0019] Step 202: For each text box T i ∈T, calculate the coordinate point of its upper left corner (x i ,y i ) to each cluster center C k The Euclidean distance of :

[0020] ;

[0021] Among them, (x k ,yk ) is the cluster center C k coordinates of

[0022] Step 203: Change the text box T i Assign to the cluster S corresponding to the cluster center with the smallest distance k , and update the cluster center coordinates to:

[0023] ;

[0024] Among them, |S k | represents cluster S k The number of text boxes in the textbox;

[0025] Step 204 : Repeat steps 202 to 203 until the cluster center coordinates change by less than 0.5 pixels or the iterations exceed 50 times, and assign the finally converged cluster S1∪S2 to the left column and S3∪S4 to the right column.

[0026] As a further preferred embodiment of the present technical solution, in step 6, the semantic similarity between the field name and the candidate value is calculated using the cosine similarity formula:

[0027] ;

[0028] Where a and b are the sentence vectors generated by Sentence-BERT.

[0029] As a further preferred embodiment of the present invention, in step eight, the operation of scrolling the second column is to set the text box sequence Updated to ,If the matching fails for three consecutive times, a scroll is triggered, and the semantic similarity is recalculated after each scroll.

[0030] As a further preferred embodiment of the present technical solution, in step seven, the similarity threshold is 0.8. If the match is successful, the second column is translated by the anchor row to align the matching starting points of the two columns.

[0031] As a further preferred embodiment of the present technical solution, in step one, the deep learning model uses PaddleOCR to train a text detection model through 500 on-site license images to extract the text box including the coordinates of the upper left corner.

[0032] As a further preferred embodiment of the present technical solution, in step 4, the y-axis spacing threshold is set to 50% of the average height of the text boxes, and the contents of adjacent text boxes are merged to process multi-line fields.

[0033] As a further preferred embodiment of the present technical solution, in step three, the text boxes are sorted from top to bottom according to their y values, arranged in ascending order based on the y values ​​of the text boxes, and text boxes whose y value difference is less than 30% of the average height are regarded as being in the same row.

[0034] As a further preferred embodiment of the present technical solution, in step five, the semantic matching dataset is constructed based on ten field pairs extracted from 2,000 on-site license images, including company name, person in charge name, business location, license scope, etc.

[0035] As a further preferred embodiment of the present technical solution, the method further includes performing semantic coherence verification on the output result, specifically:

[0036] Calculate the cosine similarity of word vectors of adjacent fields. If it is lower than 0.6, manual review will be triggered.

[0037] Validate the field type format;

[0038] Check if required fields are missing.

[0039] The embodiment of the present invention adopts the above technical solution, which has the following advantages:

[0040] 1. This invention uses the K-Means spatial clustering algorithm to dynamically cluster text box coordinates, and combines column translation and scrolling mechanisms to effectively solve the problem of text misalignment caused by printing equipment errors, paper offset, shooting angle, etc., ensuring accurate correspondence between fields and values ​​during information extraction;

[0041] 2. This invention improves text detection accuracy by using PaddleOCR customized training, combines Sentence-BERT to build a semantic matching model, and verifies the semantic coherence of the output results. This solves the problem of information loss caused by missing print, blurred images, or OCR misrecognition, ensuring the accuracy and efficiency of information management and supervision.

[0042] 3. By abandoning the traditional fixed-position or single OCR method, the present invention adopts dynamic clustering and sorting, and semantic + spatial dual matching to adapt to different layouts and dislocation scenarios, solving the problem that the existing technology is difficult to handle printing dislocation, and improving the versatility and robustness of the method.

[0043] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic flow chart of the structural information optimization method for document content identification according to the present invention;

[0046] Figure 2 Schematic diagram of the flow of the method of allocating the remaining text boxes to the left column and the right column according to their coordinates according to the present invention. DETAILED DESCRIPTION

[0047] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0048] It should be clear that the following embodiments of the present disclosure are described through specific concrete examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0049] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0050] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0051] Additionally, in the following description, specific details are provided to provide a thorough understanding of the examples. However, one skilled in the art will appreciate that the aspects described can be practiced without these specific details.

[0052] Figure 1 It is a flow chart of the structural information optimization method for document content recognition according to an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present application is not based on Figure 1 The process sequence shown is limited. Figure 1-Figure 2 As shown: The structural information optimization method for document content recognition includes the following steps:

[0053] Step 1: Obtain an image of the monopoly license and extract the text boxes, coordinates, and content of all text areas in the image using a deep learning model.

[0054] Specifically, first, obtain a picture of the monopoly license. The picture can be obtained by taking a photo with a camera or scanning with a scanner, etc., to ensure that the obtained picture is clear and can clearly show the text content on the license;

[0055] The acquired images are then processed using a deep learning model. PaddleOCR is used as the deep learning model here. A training set is constructed using 500 license images collected on-site. PaddleOCR's text detection model is then trained specifically to better suit the scenario of monopoly licenses.

[0056] Finally, the trained deep learning model is used to detect text in the image, extracting the text boxes, coordinates, and contents of all text areas in the image. Each text box contains the coordinates (x, y) of its upper left corner in the image and the text content within the box, thus providing basic data for subsequent text processing.

[0057] Step 2: Process all the extracted text boxes, delete the text boxes belonging to the license name, and use the K-Means spatial clustering algorithm to assign the remaining text boxes to the left column and the right column according to their coordinates;

[0058] Specifically, first, we identify and delete text boxes belonging to the license name through rule filtering and keyword matching. We also delete text boxes located in the top 15% of the image and with an aspect ratio greater than 5. We calculate the edit distance between the text box content and the preset keyword library, and delete candidate boxes with a threshold value less than 2.

[0059] Then, the center coordinates of the remaining text boxes are extracted as cluster features, the x-coordinate range Δx of all text boxes is calculated, the number of cluster centers K is set to 4, and the x-coordinates of the initial centers C1 and C2 are placed in [min(x i ),min(x i )+Δx / 3], the x coordinates of C3 and C4 are placed in the range [max(x i )-Δx / 3,max(x i )] interval, all initial center y coordinates are set to the arithmetic mean y value of the text box set;

[0060] Next, perform K-Means clustering iterations, calculate the Euclidean distance between each text box center and each cluster center, assign it to the cluster with the smallest distance, and update the cluster center coordinates to the average of all text box centers in the cluster. Iterate until the cluster center coordinates change < 0.5 pixels or the number of iterations is > 50;

[0061] Finally, the converged cluster S1∪S2 is assigned to the left column and S3∪S4 is assigned to the right column. If the number of text boxes in a column is less than 20% of the total number, the clustering is reinitialized, the average x-coordinate of the left and right columns is calculated, and outliers are filtered.

[0062] Step 3: Based on each column of clustered text boxes, sort the text boxes from top to bottom according to their y values ​​to obtain two columns of orderly arranged text boxes and their text contents;

[0063] Specifically, first, preprocess each column of clustered text boxes, calculate the average height h_avg of the text boxes in the column, calibrate the y coordinate of each text box to y_top + h_avg / 2, and filter out outlier text boxes whose y coordinate deviates from the median by more than 3 standard deviations;

[0064] Then, the text boxes are sorted in ascending order based on the calibrated y values, and the text boxes with y value differences less than h_avg × 30% are considered to be in the same row. Within the same row, the order is adjusted in ascending order of x values;

[0065] Next, the vertical spacing between adjacent text boxes is detected. If the spacing is less than h_avg × 50%, the text boxes are merged into continuous text, the contents are concatenated in order, and the leftmost x and topmost y coordinates are retained.

[0066] Finally, verify the sorting results and calculate the standard deviation σ_y of the y distances between adjacent text boxes. If σ_y is greater than h_avg×0.7, adjust the threshold and re-sort them, eventually forming an ordered sequence of text boxes and corresponding contents in the left and right columns.

[0067] Step 4: Within each obtained column, the vertical distance between adjacent text boxes is determined. If the y-axis distance between the two is lower than the set threshold, the text contents are merged into a continuous part of the same field.

[0068] Specifically, first, calculate the average height h_avg of each column of text boxes as the basis for spacing judgment;

[0069] Then, traverse the adjacent text box pairs in each column, get the bottom y coordinate of the previous text box (y1_bottom) and the top y coordinate of the next text box (y2_top), and calculate the vertical spacing between the two: diff = y2_top - y1_bottom;

[0070] Next, the diff is compared with the set threshold (e.g., h_avg×50%). If the diff is lower than the threshold, the text contents of the two text boxes are concatenated in sequence to form continuous text in the same field.

[0071] Finally, the coordinates of the text boxes are merged and updated to include the minimum upper left corner y coordinate and the maximum lower right corner y coordinate of the two text boxes, while retaining the x coordinate range of the original text boxes, thereby completing the merging process of adjacent text boxes.

[0072] Step 5: Based on Sentence-BERT, extract field pairs from license images collected on-site, build a semantic matching dataset, and train a semantic matching model;

[0073] Specifically, first, 2,000 monopoly license images were collected from the site, and ten core field pairs were extracted through manual annotation, including enterprise name-enterprise name value, person in charge name-name value, business location-address value, license scope-scope value, supplier-unit value, validity period-term value, issuing authority-authority value, certificate issuance date-date value, enterprise type-type value, and license number-number value;

[0074] Then, preprocess the extracted field pairs to clean special symbols, space anomalies, etc. in the text, use the Jieba word segmentation tool to segment the Chinese text and remove stop words;

[0075] Next, we constructed a training dataset based on the Sentence-BERT model. We converted field pairs into the text format of [field name, field value] and used the Contrastive Learning strategy to generate positive and negative samples (positive samples were true field pairs, and negative samples were random combinations across fields). We then configured the training parameters, setting the batch size to 32, the learning rate to 2e-5, and the number of training rounds to 5. We used the AdamW optimizer with gradient clipping enabled (threshold 1.0) and adjusted the learning rate using the cosine annealing learning rate scheduling strategy. The Sentence-BERT model used the paraphrase-multilingual-MiniLM-L12-v2 pre-trained model.

[0076] Finally, the model performance was evaluated on a 10% validation set, and the F1 value was calculated using the cosine similarity of field pairs. Training was stopped when the F1 value on the validation set was ≥ 0.92, and the optimal model weights were saved for subsequent semantic matching.

[0077] Step 6: Use the left and right columns as a pairing of field names and values, calculate the semantic similarity between the field names and the candidate values, and select the one with the highest semantic similarity as the initial match;

[0078] Specifically, first, the contents of the left column text box after step 3 sorting are used as the field name set F=[f1,f2,…,f m ], the right column is the candidate value set V=[v1,v2,…,v n ];

[0079] Then, load the Sentence-BERT model trained in step 5 and change each field name to i With candidate value v j Converted to 768-dimensional sentence vector a i and b j ;

[0080] Next, we traverse all pairings of field names and candidate values ​​and calculate the semantic similarity using the cosine similarity formula:

[0081] ;

[0082] Among them, a and b are sentence vectors generated by the Sentence-BERT model, representing the semantic feature vectors of the field name and candidate value respectively;

[0083] Record each pair (f i ,v j ) similarity value;

[0084] Finally, for each field name f i , find the v with the highest similarity in the candidate value set jAs the initial match, if the highest similarity is lower than the preset threshold (such as 0.4), it is marked as unmatched, forming the initial matching pair set M={(f1,v j1 ),(f2,v j2 ),…,(f m ,v jm )}.

[0085] Step 7: If the similarity exceeds the threshold, shift the second column with the matching behavior anchor point to align with the matching starting point;

[0086] Specifically, first, traverse the initial set of matching pairs obtained in step 6 and select matching pairs with similarity ≥ 0.6 as valid anchor points. If the number of valid anchor points is insufficient, lower the threshold to 0.5 and continue screening.

[0087] Then, calculate the y-coordinate difference between each valid anchor point in the left and right columns:

[0088] Δy k =y_left_anchor k -y_right_anchor k ;

[0089] Among them, y_left_anchor k y_right_anchor is the y coordinate of the upper left corner of the text box of the kth anchor row in the left column. k The y coordinate of the upper left corner of the text box in the k-th anchor row in the right column;

[0090] Take all Δy k The median of is taken as the final vertical offset Δy;

[0091] Next, for all text boxes in the right column, add the Δy value to the y coordinates of their upper left corner and lower right corner respectively to complete the translation. Check whether the text box exceeds the image boundary after translation. If it exceeds, adjust Δy so that the boundary text box is just visible;

[0092] Finally, based on the shifted right column, for each field name, we search for the candidate value with the closest y coordinate (δy ≤ 0.5 × average text height) as the matching starting point and update the matching pair set.

[0093] Step 8: Match subsequent fields row by row. If the matching fails continuously, scroll to the second column and match again, and output the successfully matched field pairs.

[0094] Specifically, first, initialize the current row index i=0, the continuous failure counter fail_count=0 and the maximum continuous failure threshold to 3, and set the maximum scrolling times of the right column to 5 times;

[0095] Then, the text box content in row i of the left column is obtained as the field name, and its semantic similarity with the candidate value in row i of the right column is calculated. If the similarity is ≥ 0.5, the match is successful, the field pair is added to the result set, and fail_count is reset. Otherwise, fail_count is increased by 1.

[0096] Then check if fail_count reaches the threshold. If so, scroll the entire right column up by the average text box height and reset i and fail_count.

[0097] Finally, determine whether all field names in the left column have been traversed or the number of scrolls in the right column has exceeded the limit. If so, output the set of successfully matched field pairs and the list of unmatched fields.

[0098] In one embodiment, specifically, in step 2, the specific steps of allocating the remaining text boxes to the left column and the right column according to their coordinates are as follows:

[0099] Step 201: Set the number of cluster centers K = 4, and calculate the x-coordinate range Δx = max(x i )-min(x i ), select the initial cluster center C={C1,C2,C3,C4} based on Δx, where the x coordinates of C1 and C2 are located in [min(x i ),min(x i )+Δx / 3] interval, the x coordinates of C3 and C4 are located in [max(x i )-Δx / 3,max(x i )] interval, the y-coordinate of C1-C4 takes the average y-value of set T;

[0100] Specifically, first, the number of cluster centers is set to K = 4. This is because the text of the monopoly license is usually distributed in two columns, left and right. Setting two cluster centers in each column can more accurately capture the spatial distribution characteristics of the text box. Secondly, the x-coordinate range Δx of all elements in the remaining text box set T is calculated, that is, by finding the maximum x-coordinate max(x i ) and the minimum value min(x i ) to determine the distribution range on the x-axis;

[0101] Next, the x-coordinate interval of the initial cluster center is divided based on Δx: the x-coordinates of C1 and C2 are limited to [min(x i ),min(x i )+Δx / 3], which covers the x-coordinate range of the left column text box; the x-coordinates of C3 and C4 are located in [max(x i )-Δx / 3,max(x i)] interval, corresponding to the x-coordinate range of the text box in the right column; the purpose of this setting is to make the initial center close to the actual distribution of the left and right columns, reducing the number of clustering iterations;

[0102] Finally, the y coordinates of C1 to C4 are uniformly taken as the average of the y coordinates of all text boxes in the set T to ensure that the initial center is at the center of the text area in the vertical direction and avoid clustering bias caused by y coordinate offset; for example, if the x coordinates of the remaining text boxes range from 100 to 500, then Δx = 400, the x coordinate interval of the center of the left column is [100, 233], and the center of the right column is [367, 500]. The y coordinate takes the average of the y values ​​of all text boxes, thereby providing reasonable initial conditions for subsequent K-Means clustering.

[0103] Step 202: For each text box T i ∈T, calculate the coordinate point of its upper left corner (x i ,y i ) to each cluster center C k The Euclidean distance of :

[0104] ;

[0105] Among them, (x k ,y k ) is the cluster center C k coordinates of

[0106] Specifically, step 202 is the key calculation link in the K-Means clustering algorithm. In the previous step, the number of cluster centers K=4 has been determined, and the initial cluster centers C={C1,C2,C3,C4} have been set. Now, for each text box T in the remaining text box set T, i , the specific calculation process is as follows:

[0107] Coordinate acquisition: Known text box T i The coordinates of the upper left corner are (x i ,y i ), which is the position information of the text box on the image plane; at the same time, the cluster center C k There are also corresponding coordinates (x k ,y k ), where k takes values ​​of 1, 2, 3, and 4, corresponding to four different cluster centers respectively;

[0108] Difference calculation: Calculate text box T i The coordinates of the cluster center C k The difference between the x-axis and y-axis coordinates, that is, x i -x k and y i -y k; These two differences reflect the degree of offset of the text box relative to the cluster center in the horizontal and vertical directions;

[0109] Square and square root operation: Square the two differences above and get (x i -x k )² and (y i -y k )², then add the two squared values ​​and take the square root of the result, that is, calculate , the obtained value It is the text box T i The coordinate point of the upper left corner to the cluster center C k The Euclidean distance of

[0110] By calculating the Euclidean distance, the text box T can be measured. i With each cluster center C k The degree of "distance"; in the K-Means clustering algorithm, the text box will be assigned to the category represented by the cluster center with the smallest Euclidean distance to it; in the scenario of processing the monopoly license text box, this method can roughly divide the text box into two columns (because 4 cluster centers are set, two in each column), laying the foundation for further processing of the text in the left and right columns (such as alignment, matching, etc.); for example, if the text box T i If the Euclidean distance to C1 or C2 is the smallest, then it is likely to belong to the left column text box; if the Euclidean distance to C3 or C4 is the smallest, then it is likely to belong to the right column text box.

[0111] Step 203: Change the text box T i Assign to the cluster S corresponding to the cluster center with the smallest distance k , and update the cluster center coordinates to:

[0112] ;

[0113] Among them, |S k | represents cluster S k The number of text boxes in the textbox;

[0114] Specifically, in step 202, each text box T has been calculated. i To each cluster center C k Euclidean distance ; This step is to change each text box T according to these distances. i Assign to the cluster S corresponding to the cluster center with the smallest distance k middle;

[0115] For example, suppose the Euclidean distances from text box T1 to C1, C2, C3, and C4 are D 11=5, D 12 =8, D 13 =10, D 14 =12, because D 11 The smallest, so text box T1 is assigned to cluster S1; perform this operation on all text boxes in set T, and all text boxes can be divided into different clusters;

[0116] When all text boxes are assigned, each cluster S k There are several text boxes in it; at this time, the center coordinates of each cluster need to be updated ; Update formula:

[0117] ;

[0118] The meaning is as follows:

[0119] For the horizontal axis, Represents cluster S k Add the x values ​​of the upper left corner coordinates of all text boxes in the textbox. It is cluster S k Number of text boxes in |S k | Take the reciprocal, and multiply the two to get the cluster S k The average value of the x-coordinate of the text box in the text box is used as the updated cluster center The horizontal axis of .

[0120] For the vertical axis, Represents cluster S k Add the y values ​​of the upper left corner coordinates of all text boxes in the text box, and divide them by |S k |Get the average value of the y coordinate as the updated cluster center The vertical coordinate of

[0121] Through such updates, the cluster center will continue to move closer to the "center position" of the text box in the cluster; then, it will return to step 202 again, recalculate the Euclidean distance from the text box to the new cluster center, and continuously iterate this process until the change in the cluster center is less than a set threshold (for example, the coordinate change is less than 0.5 pixels). The clustering is considered to have converged, and the clustering division of the text box is completed at this time, preparing for subsequent operations such as distinguishing the left and right columns of the monopoly license text.

[0122] Step 204: Repeat steps 202 to 203 until the cluster center coordinates change by less than 0.5 pixels or the iterations exceed 50 times, and assign the finally converged cluster S1∪S2 to the left column and S3∪S4 to the right column;

[0123] Specifically, step 204 is the iterative termination and result application phase of the K-Means clustering algorithm in this scenario; wherein the iterative process is specifically as follows:

[0124] After completing the text box allocation and cluster center coordinate update in step 203, it is necessary to return to step 202 again; at this time, since the cluster center coordinate has been updated, it is necessary to recalculate each text box T i To the new cluster center C k Euclidean distance Then, the process returns to step 203, and redistributes the text boxes to the cluster corresponding to the cluster center with the smallest distance according to the newly calculated distance, and updates the cluster center coordinates again. This process of continuously repeating steps 202 and 203 is the iterative process of the clustering algorithm.

[0125] The termination conditions are as follows:

[0126] Cluster center coordinate change threshold: After each iteration, the difference between the new cluster center coordinates and the previous cluster center coordinates is calculated. When the change in the x-axis and y-axis coordinates of all cluster centers is less than 0.5 pixels, it means that the cluster center has basically stabilized and no major deviation has occurred. At this point, the clustering can be considered to have converged and the iteration can be stopped. This is because a change of 0.5 pixels is relatively small in the image coordinate system. From the perspective of text box clustering, a relatively ideal classification effect has been achieved.

[0127] Iteration limit: In addition, to prevent the algorithm from falling into a non-convergence situation (although this is generally not the case under reasonable settings), the number of iterations is capped at 50. If the number of iterations reaches 50, the iteration is stopped even if the cluster center coordinates have not changed by less than 0.5 pixels.

[0128] When the iteration meets the above termination conditions, the four final converged clusters S1, S2, S3, and S4 are obtained. In the scenario of monopoly license text processing, according to the settings, clusters S1 and S2 are merged and assigned to the left column text; clusters S3 and S4 are merged and assigned to the right column text. This completes the preliminary division of the left and right columns of the monopoly license text box, laying the foundation for subsequent further sorting, matching, and other operations on the left and right column texts.

[0129] The specific applications of the K-Means spatial clustering algorithm are as follows:

[0130] Assume that after step 1, the coordinates of the upper left corners of the remaining five text boxes are as follows:

[0131]

[0132] Calculate the x-coordinate range:

[0133] Δx=max(100,120,400,420,110)-min(100,120,400,420,110)

[0134] =420-100

[0135] =320

[0136] Determine the initial cluster centers:

[0137] The x-coordinate range of the left column centers C1 and C2 is:

[0138] [min(x i ),min(x i )+Δx / 3]=[100,100+320 / 3]≈[100,206.67]

[0139] Take C1(150,244), C2(180,244);

[0140] The x-coordinate range of the right column centers C3 and C4 is:

[0141] [max(x i )-Δx / 3,max(x i )]=[420-320 / 3,420]≈[313.33,420]\)

[0142] Take C3(350,244), C4(400,244);

[0143] Y coordinates of all centers: avg(200,250,210,260,300)=244

[0144] Initial cluster centers:

[0145]

[0146] First iteration:

[0147] Calculate the Euclidean distance from T1(100,200) to each center:

[0148]

[0149] Assignment: T1→S1 (minimum distance);

[0150] Similarly calculate other text boxes:

[0151] T2(120,250)→S1

[0152] T3(400,210)→S4

[0153] T4(420,260)→S4

[0154] T5(110,300)→S1

[0155] Update cluster center coordinates:

[0156] ;

[0157] Since C2 and C3 have no assigned text boxes, their coordinates remain unchanged;

[0158] Second iteration:

[0159] Recalculate the distance and assign:

[0160] T1(100,200)→S1 (C1 distance is the smallest after update)

[0161] T2(120,250)→S1

[0162] T3(400,210)→S4

[0163] T4(420,260)→S4

[0164] T5(110,300)→S1

[0165] Update cluster center coordinates:

[0166]

[0167] Since the text boxes within the cluster do not change, the cluster center coordinates are consistent with the results of the previous iteration;

[0168] After the second round, the changes in cluster center coordinates were all less than 0.5 pixels, satisfying the convergence condition;

[0169] Final allocation results:

[0170] Left column: S1∪S2={T1,T2,T5}

[0171] Right column: S3∪S4={T3,T4}

[0172] Proven:

[0173] Left column text box x coordinate range: 100-120;

[0174] The x-coordinate range of the right column text box is 400-420;

[0175] It conforms to the spatial distribution characteristics of the left and right columns.

[0176] In one embodiment, specifically, in step 6, the semantic similarity between the field name and the candidate value is calculated using the cosine similarity formula:

[0177] ;

[0178] where a and b are sentence vectors generated by Sentence-BERT;

[0179] The cosine similarity formula measures the semantic similarity between two texts by calculating the dot product of sentence vectors a and b and dividing it by the product of their respective norms. The closer the value is to 1, the more similar the semantics are, thus helping to determine whether the field name of the monopoly license matches the candidate value;

[0180] Specifically, assume there is a monopoly license. The field name "Name of the person in charge" is extracted from the left column, and the corresponding candidate values in the right column are "Zhang San", "Li Si", and "License number: 123456"; Use the Sentence-BERT model to generate sentence vectors;

[0181] Input "Name of the person in charge", "Zhang San", "Li Si", and "License number: 123456" into the trained Sentence-BERT model respectively; The Sentence-BERT model will process each input text and convert it into a sentence vector of a fixed length;

[0182] Assume the sentence vector generated by "Name of the person in charge" is a = [0.1, 0.2, 0.3,..., 0.768] (assuming the dimension of the sentence vector is 768);

[0183] The sentence vector generated by "Zhang San" is b1 = [0.2, 0.3, 0.4,..., 0.769];

[0184] The sentence vector generated by "Li Si" is b2 = [0.15, 0.25, 0.35,..., 0.767];

[0185] The sentence vector generated by "License number: 123456" is b3 = [0.05, 0.08, 0.1,..., 0.5];

[0186] Calculate the semantic similarity between "Name of the person in charge" and "Zhang San" according to the cosine similarity formula:

[0187] ;

[0188] where a·b is the dot product of vectors a and b, and the calculation method is to multiply the elements in the corresponding positions and then add them together, that is:

[0189] ;

[0190] is the norm of vector a (which can be understood as the length of the vector), and the calculation formula is:

[0191] ;

[0192] Similarly;

[0193] After calculation, it is assumed that sim(a,b1)=0.85;

[0194] The semantic similarity between "name of person in charge" and "Li Si" is calculated according to the above formula, and the result is sim(a,b2)=0.83;

[0195] Calculating the semantic similarity between "name of person in charge" and "license number: 123456" yields sim(a,b3)=0.1;

[0196] The calculation results show that the semantic similarity between "name of person in charge" and "Zhang San" and "Li Si" is relatively high, which are 0.85 and 0.83 respectively, while the semantic similarity with "license number: 123456" is only 0.1;

[0197] In practical applications, a similarity threshold can be set, such as 0.7. When the semantic similarity between the field name and the candidate value is greater than or equal to 0.7, they are considered to match. So in this example, "Zhang San" and "Li Si" may both be correct values ​​corresponding to the "name of the person in charge." Subsequently, other information (such as frequency of occurrence) can be further combined to determine the final matching value.

[0198] In one embodiment, specifically, in step eight, the operation of scrolling the second column is to move the text box sequence Updated to ,If the matching fails for three consecutive times, a scroll is triggered, and the semantic similarity is recalculated after each scroll;

[0199] Specifically, in the exclusive license information processing scenario involved in step 8, the text box sequence Represents the second column (usually the value column) on the license text box collection, where They are the text boxes in the second column;

[0200] The scrolling second column operation is triggered when the field name fails to match the candidate value in the second column for three consecutive times (i.e., it is determined to be mismatched by semantic similarity calculation or other methods).

[0201] At this time, move the second column text box sequence forward one position, and the original first text box Remove and fill in the "empty" value at the end to form a new sequence ;

[0202] After scrolling, the semantic similarity between the left column field names and the new second column candidate values ​​is recalculated. The goal is to find the correct field name-value matching combination by adjusting the text box correspondence to accurately extract the license structured information.

[0203] Assume there is a monopoly license. When extracting structured information from it:

[0204] After preliminary processing, the text box sequence of the second column (value column) is obtained , corresponding to "No. 300", "Retail", "2025-01-01", "Wang Wu", and "Company" respectively. The field names in the left column are "Business Place", "License Scope", "Validity Period", "Name of Person in Charge", and "Company Name" respectively.

[0205] First, calculate the “business premises” and If the semantic similarity of "300" is lower than the set threshold, the match fails and is counted as 1.

[0206] Next, calculate the "business location" and the next candidate value The semantic similarity of ("retail") is still below the threshold, so the match fails and the count is 2;

[0207] Then calculate the “business premises” and The semantic similarity of ("2025-01-01") is still below the threshold, so the match fails. This is the third consecutive match failure.

[0208] Trigger the scroll operation and sequence the text boxes Updated to =["Retail","2025-01-01","Wang Wu","Company","Empty"];

[0209] After scrolling, the semantic similarity between "business place" and the candidate values ​​in the new sequence is recalculated; at this time, the semantic similarity between "business place" and "retail", "2025-01-01", "Wang Wu", and "company" is calculated in sequence. Assuming that the semantic similarity between "business place" and "company" is higher than the threshold, the match is successful, and the correct matching of the field name and value is completed; subsequently, the other field names are matched with the candidate values ​​in the second column after scrolling in the same way until the extraction of the entire license structured information is completed.

[0210] In one embodiment, specifically, in step seven, the similarity threshold is 0.8. If the match is successful, the second column is shifted by the anchor row to align the matching starting points of the two columns.

[0211] Specifically, first, the similarity threshold is set to 0.8, which is the standard for judging whether the field name and the candidate value are matched successfully. After using a model such as Sentence-BERT to generate sentence vectors and calculating the semantic similarity between the field name and the candidate value using the cosine similarity formula, if the obtained similarity value is greater than or equal to 0.8, it is determined that the two are highly semantically similar, that is, the match is successful. For example, for the field name "Company Name" and the candidate value "XX Co., Ltd.", the calculated cosine similarity is 0.85. Since 0.85 ≥ 0.8, it is determined to be a successful match.

[0212] When multiple matches are successful, an anchor row needs to be determined. The anchor row is selected from these successfully matched rows and used as the basis for subsequent column alignment operations. For example, if two sets of field names, "Company Name - XX Co., Ltd." and "Person in Charge Name - Zhang San," are both successfully matched with candidate values, the row containing one of these sets can be selected as the anchor row based on certain rules (such as frequency of occurrence and importance). For example, the row containing "Company Name - XX Co., Ltd." is selected as the anchor row.

[0213] After determining the anchor row, the second column is translated based on the anchor row; since the monopoly license text box is vertically misaligned, the second column is translated so that the candidate values ​​in the second column that match the field name of the anchor row are vertically aligned with the field name of the left column; for example, the vertical coordinate of the "Company Name" text box in the left column is y1, and the coordinate of the "XX Co., Ltd." text box in the right column is y2. There is a certain difference between the two. By calculating this difference, the second column is moved up or down as a whole by a corresponding distance, so that the two columns are vertically aligned at the matching starting point of the anchor row, laying the foundation for the subsequent accurate extraction and structuring of license information.

[0214] In one embodiment, specifically, in step 1, the deep learning model uses PaddleOCR to train a text detection model using 500 on-site license images to extract the text box including the coordinates of the upper left corner;

[0215] Specifically, we collected 500 images of monopoly licenses in different formats and shooting conditions (such as angles, lighting, and clarity) from actual business scenarios to ensure coverage of common problems such as misalignment, blur, and stains. We used tools such as LabelImg to manually annotate the text boxes in each image, including the coordinates of the text box's upper left corner (x, y), width, height, and text content (for subsequent OCR recognition training). For example, the coordinates of the rectangular box annotating the "Company Name" field were (100, 150, 200, 40). We also performed enhancement operations on the original images, such as rotating (±15°), scaling (0.8-1.2 times), and adjusting the brightness, expanding the training sample to more than 2,000 images to improve the model's generalization capabilities.

[0216] Based on the PaddleOCR framework, we selected the DB (Differentiable Binarization) text detection algorithm, whose backbone network uses ResNet50_vd, which has high accuracy and real-time performance.

[0217] Loss function: Joint training classification loss (identifying text regions) and regression loss (predicting text box coordinates);

[0218] Optimizer: Use Adam optimizer, learning rate 0.001, cosine annealing decay strategy;

[0219] Batch size: 8-16 (adjust according to GPU memory);

[0220] Training rounds: 300-500 rounds, until the validation set loss converges;

[0221] Evaluation metrics: On an independent test set, the precision and recall of text box detection both exceeded 98%, with an average detection time of less than 100ms per image.

[0222] The specific process of text box extraction and coordinate acquisition is as follows:

[0223] Preprocessing: Input the license image and enhance the text edges through operations such as Gaussian blur and histogram equalization;

[0224] Model inference: Input the preprocessed image into the trained PaddleOCR model and output the four-point coordinates of the text box ;

[0225] Coordinate conversion: convert the four-point coordinates to the upper left corner coordinates and width and height ; For example, the coordinates of the upper left corner are obtained by calculating the minimum x and minimum y of the four points:

[0226] , ;

[0227] Post-processing: Filter out text boxes with too small an area or abnormal aspect ratio, and retain the valid text area;

[0228] Among the 500 test images, the text box positioning error (compared with manual annotation) accounted for 95% of the images ≤ 2 pixels, ensuring the accuracy of subsequent clustering and matching; for images with a tilt of ≤ 15° and partial occlusion, the text box extraction accuracy remained above 90%; each text box output was format, where x and y are the coordinates of the upper left corner of the text box, w and h are the width and height of the text box, and text is the text content recognized in the text box; for example: ;

[0229] Through the above steps, the conversion from images to structured text boxes is completed, providing basic data for subsequent clustering, matching, and alignment.

[0230] In one embodiment, specifically, in step 4, the y-axis spacing threshold is set to 50% of the average height of the text boxes, and the contents of adjacent text boxes are merged to process multi-line fields;

[0231] Specifically, first, calculate the average height of all text boxes in the vertical direction (y-axis); for example, count the text boxes in a batch of monopoly license images and calculate the average height h;

[0232] Next, set the y-axis spacing threshold to 50% of the average text box height, or 0.5h. This threshold is set because when the y-axis spacing between adjacent text boxes is small, they might actually belong to the same field, but are displayed across multiple lines due to layout reasons. For example, the "Business Location" field has a lot of content and is displayed across two lines in two adjacent text boxes. If these two text boxes are too close together on the y-axis, they are likely to be different lines of the same field.

[0233] Traverse all text boxes, and for each text box, check the distance between it and the adjacent text box below it in the y-axis direction; assuming that the bottom y-coordinate of the current text box is y1, and the top y-coordinate of the adjacent text box below is y2, if y2-y1≤0.5h, then determine that the two text boxes belong to multiple lines of content in the same field, and merge their contents; for example, if the content of a text box is "xx City" and the content of the adjacent text box below is "No. 123, xx Road, xx District", and the y-axis distance between the two meets the threshold condition, merge them into "No. 123, xx Road, xx District, xx City"; through such an operation, the content of the same field originally scattered in multiple text boxes can be integrated, which facilitates the subsequent accurate matching of field names and corresponding values ​​and the extraction of structured information, and ensures that information extraction errors will not be caused by the display of text in separate lines.

[0234] In one embodiment, specifically, in step 3, the text boxes are sorted from top to bottom according to their y values, arranged in ascending order based on the y values ​​of the text boxes, and text boxes with y value differences less than 30% of the average height are considered to be in the same row;

[0235] Specifically, first, we need to obtain the y value of the upper left corner coordinate of each text box. Because in the layout of a license, text boxes of different fields are usually arranged from top to bottom, sorting by y value is consistent with people's reading and information organization habits. All text boxes are arranged in ascending order of y value. For example, there are three text boxes T1, T2, and T3, and the y values ​​of their upper left corner coordinates are y1=100, y2=120, and y3=80 respectively. After sorting, the order becomes T3, T1, T2, which makes the text boxes have an orderly arrangement in the vertical direction.

[0236] Next, calculate the average height h of all text boxes; when judging whether two text boxes belong to the same row, the difference in their y values ​​is used; if the difference in the y values ​​of the two text boxes is less than 30% of the average height h, the two text boxes are considered to belong to the same row; for example, if the average height h = 30 pixels, then 30% of h is 9 pixels; if the y value of text box A is 150 and the y value of text box B is 155, the difference in their y values ​​is 5 pixels, which is less than 9 pixels, so text boxes A and B are considered to be in the same row; the advantage of this is that in the monopoly license, some fields will be displayed on the same row across multiple text boxes. In this way, the text boxes in the same row can be accurately associated, providing more reasonable text box grouping for subsequent matching of field names and corresponding values, structured processing, and other operations, avoiding information extraction errors caused by irregular distribution of text boxes.

[0237] In one embodiment, specifically, in step five, the semantic matching dataset is constructed based on ten field pairs extracted from 2,000 on-site license images, including company name, person in charge name, business location, license scope, etc.;

[0238] Specifically, the construction of a semantic matching dataset is the key to ensuring accurate field matching; based on 2,000 on-site monopoly license images, the ten core fields such as company name, person in charge name, business location and their corresponding values ​​are manually annotated and extracted to form field pairs; these images cover different regional layouts, shooting conditions and dislocations, including printing blur, stains and occlusions, to ensure data diversity; during the extraction process, PaddleOCR is first used to preliminarily identify the text box, and then manual verification and correction of OCR misrecognition problems are carried out, and field pairs are matched according to the layout rules of the field name in the left column and the corresponding value in the right column of the license; at the same time, the text is pre-processed by cleaning, word segmentation, format normalization, etc., to construct positive and negative sample enhancement data, and finally the training set, validation set and test set are divided into training set, validation set and test set in an 8:1:1 ratio to provide data support for Sentence-BERT model training, so that the model can learn the semantic association between fields and achieve accurate matching.

[0239] In one embodiment, the method further includes performing semantic consistency verification on the output result, specifically:

[0240] Calculate the cosine similarity of word vectors of adjacent fields. If it is lower than 0.6, manual review will be triggered.

[0241] Validate the field type format;

[0242] Check if required fields are missing;

[0243] Specifically, semantic coherence verification of output results is a key step in ensuring the accuracy of structured information, which is achieved through the following three steps:

[0244] Semantic association verification of adjacent fields: Use the Sentence-BERT model to generate word vectors for adjacent fields (such as "Company Name" and "Person in Charge Name") and calculate cosine similarity. If the similarity is lower than 0.6 (for example, the normal similarity between "Company Name - XX Company" and "Person in Charge Name - Zhang San" is about 0.3, but if a set of results shows a similarity of 0.7, there is a field mismatch), it indicates a semantic logic anomaly, triggering manual review. For example, if "Business Location - XX City XX District" is followed by "Permit Scope - 2025.01.01", the semantic similarity between the two is only 0.2, which is far below the threshold. Manual inspection is required to determine whether the time value is incorrectly matched to the address due to column misalignment.

[0245] Field format compliance verification is performed based on the preset format rules of the field type:

[0246] Date type (such as "Expiration Date"): Use the regular expression d{4}.d{2}.d{2}-d{4}.d{2}.d{2} to verify the "YYYY.MM.DD-YYYY.MM.DD" format;

[0247] Names (such as "Name of the person in charge"): Check whether they contain non-Chinese characters (intermediate periods "·" are allowed, such as "Maimaiti·XX");

[0248] Number type (such as "License Number"): matches fixed length rules (such as a 12-digit alphanumeric combination). If the "Expiration Date" is extracted as "2025-01", it will be automatically marked as a format error because it does not conform to the double date format.

[0249] A completeness check of mandatory fields is performed to clarify that six fields, including "Company Name," "Person in Charge Name," "Business Location," and "Validity Period," are mandatory. If any mandatory field in the output result is empty (e.g., the "Person in Charge Name" field fails to match), the system will automatically mark it as "missing" and block the manual completion process, preventing incomplete structured data from affecting the regulatory system input.

[0250] Through the above-mentioned verification mechanism, the field matching error rate can be further reduced from 0.8% to below 0.1%, ensuring that the output license information complies with regulatory business specifications.

[0251] In summary, the structural information optimization method for document content recognition provided by the embodiment of the present invention utilizes the PaddleOCR deep learning model to extract the coordinates and content of text boxes, combines the K-Means spatial clustering algorithm to distribute the text boxes to the left and right columns according to the coordinates, and then sorts the text boxes according to the y value and merges the adjacent text boxes with a vertical spacing less than 50% of the average height. Then, the Sentence-BERT semantic matching model is trained based on the ten major fields extracted from 2,000 on-site pictures, and the cosine similarity between the field name and the candidate value is calculated for initial matching. For matching rows with a similarity exceeding 0.8, the anchor row is shifted to the second column to align the starting point. If the match fails for three consecutive times, the second column is scrolled and re-matched. Finally, the semantic coherence check is completed by calculating the similarity of the word vectors of adjacent fields, verifying the field format and checking the required fields, thereby achieving precise optimization of the dislocation of the structured information of the license and ensuring the accuracy of field matching and information extraction.

[0252] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0253] In the present disclosure, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "including," "comprising," "having," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0254] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0255] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0256] Various changes, substitutions, and modifications may be made to the technology described herein without departing from the teachings defined by the appended claims. Moreover, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of things, means, methods, and actions described above. Currently existing or later developed processes, machines, manufactures, compositions of things, means, methods, or actions that perform substantially the same function or achieve substantially the same results as the corresponding aspects described herein may be utilized. Accordingly, the appended claims include within their scope such processes, machines, manufactures, compositions of things, means, methods, or actions.

[0257] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0258] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A structural information optimization method for document content recognition, characterized in that: The following steps are involved: Obtain an image of the exclusive license and extract the text boxes, coordinates, and content of all text areas in the image using a deep learning model. Process all the extracted text boxes, delete the text boxes belonging to the license name, and use the K-Means spatial clustering algorithm to assign the remaining text boxes to the left and right columns according to their coordinates; Based on each column of clustered text boxes, sort the text boxes from top to bottom according to their y values ​​to obtain two columns of orderly arranged text boxes and their text contents; Within each obtained column, the vertical distance between adjacent text boxes is determined. If the y-axis distance between the two is lower than the set threshold, the text contents are merged into a continuous part of the same field. Based on Sentence-BERT, we extracted field pairs from license images collected on-site, constructed a semantic matching dataset, and trained a semantic matching model. The left and right columns are used as pairs of field names and values. The semantic similarity between the field names and candidate values ​​is calculated, and the highest one is selected as the initial match. If the similarity exceeds the threshold, the second column is shifted with the matching behavior anchor point to align with the matching starting point; Match subsequent fields line by line. If the matching fails continuously, scroll to the second column and match again, and output the successfully matched field pairs.

2. The structural information optimization method for document content recognition according to claim 1 is characterized in that ,The specific steps of allocating the remaining text boxes to the left column and the right column according to their coordinates are as follows: Step 201: Set the number of cluster centers K = 4, and calculate the x-coordinate range Δx = max(x i )-min(x i ), select the initial cluster center C={C1,C2,C3,C4} based on Δx, where the x coordinates of C1 and C2 are located in [min(x i ),min(x i )+Δx / 3] interval, the x coordinates of C3 and C4 are located in [max(x i )-Δx / 3,max(x i )] interval, the y-coordinate of C1-C4 takes the average y-value of set T; Step 202: For each text box T i ∈T, calculate the coordinate point of its upper left corner (x i ,y i ) to each cluster center C k The Euclidean distance of : ； Among them, (x k ,y k ) is the cluster center C k coordinates of Step 203: Change the text box T i Assign to the cluster S corresponding to the cluster center with the smallest distance k , and update the cluster center coordinates to: ； Among them, |S k | represents cluster S k The number of text boxes in the textbox; Step 204 : Repeat steps 202 to 203 until the cluster center coordinates change by less than 0.5 pixels or the iterations exceed 50 times, and assign the finally converged cluster S1∪S2 to the left column and S3∪S4 to the right column.

3. The structural information optimization method for document content recognition according to claim 1, characterized in that: The semantic similarity between the field name and the candidate value is calculated using the cosine similarity formula: ； Where a and b are the sentence vectors generated by Sentence-BERT.

4. The structural information optimization method for document content recognition according to claim 1, characterized in that: The operation of scrolling the second column is to set the text box sequence Updated to ,If the matching fails for three consecutive times, a scroll is triggered, and the semantic similarity is recalculated after each scroll.

5. The method for optimizing structural information for document content recognition according to claim 1, characterized in that: The similarity threshold is 0.

8. If the match is successful, the second column is translated by the anchor row to align the matching starting points of the two columns.

6. The structural information optimization method for document content recognition according to claim 1, characterized in that: The deep learning model uses PaddleOCR to train a text detection model using 500 on-site license images to extract the text box including the coordinates of the upper left corner.

7. The structural information optimization method for document content recognition according to claim 1, characterized in that: The y-axis spacing threshold is set to 50% of the average height of the text boxes, and the contents of adjacent text boxes are merged to process multi-line fields.

8. The method for optimizing structural information for document content recognition according to claim 1, characterized in that: The text boxes are sorted from top to bottom according to their y values, arranged in ascending order based on the y values ​​of the text boxes, and text boxes whose y value difference is less than 30% of the average height are considered to be in the same row.

9. The structural information optimization method for document content recognition according to claim 1, characterized in that: The semantic matching dataset is constructed based on the ten fields extracted from 2,000 on-site license images, including company name, name of person in charge, business location, license scope, company type, validity period, issuing authority, issuance date, supplier and license number.

10. The structural information optimization method for document content recognition according to claim 1, characterized in that: The method further includes performing semantic coherence verification on the output result, specifically: Calculate the cosine similarity of word vectors of adjacent fields. If it is lower than 0.6, manual review will be triggered. Validate the field type format; Check if required fields are missing.

Images