An OCR-based waybill identification calibration method

Abstract

The present application relates to a kind of based on OCR with cargo list identification calibration method, comprising: by external OCR interface, with cargo list image is identified, and the key text block that company is contained in with cargo list is identified, with commodity database is matched, obtains supplier name;With cargo list table and table template are adapted, it is judged whether there is corresponding target table template;If there is, according to text block coordinates, text block is filled in target table template, and table information object is obtained;When the number of goods in table information object is not empty, it is determined that with cargo list table and target table template are adapted accurately, then determine that table information object is as identification result, and identification result includes attribute text and the number of goods in row information.This application extracts the picture data of with cargo list by combining OCR text recognition technology, and combines row and column coordinate check, multi-template adaptation and drugstore commodity database data, calibrates above-mentioned picture data, to improve the accuracy of with cargo list identification and the efficiency of warehousing.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and in particular to a method for recognizing and calibrating a goods accompanying list based on OCR. BACKGROUND

[0002] A goods accompanying list is a general term for a drug store goods accompanying list, a sales list, a warehouse-out list, etc. As a drug circulation enterprise or a medical institution purchases and sells drugs, a sales record must be established, and a voucher must be used to ensure that the bill, the account, the goods, and the payment are consistent.

[0003] Based on the general OCR text recognition technology provided by Ali, most of the text on the goods accompanying list can be initially recognized. However, due to the different levels of user photographing skills and the uneven quality of pictures, the directly recognized text cannot be directly used, which affects the accuracy of the goods accompanying list recognition and the efficiency of the goods accompanying list information storage. SUMMARY

[0004] Therefore, the present application provides a method for recognizing and calibrating a goods accompanying list based on OCR. The picture data of the goods accompanying list is extracted by combining the OCR text recognition technology, and the picture data is calibrated by combining the row and column coordinate verification, the multi-template adaptation, and the drug store commodity database, so as to improve the accuracy of the goods accompanying list recognition and the efficiency of the goods accompanying list storage.

[0005] According to a first aspect of some embodiments of the present application, a method for recognizing and calibrating a goods accompanying list based on OCR is provided, which includes the following steps:

[0006] An image of a goods accompanying list is obtained, the image is recognized through an external OCR interface, and a table of the goods accompanying list in a preset output format and text block coordinates referenced to the table are obtained;

[0007] The table coordinates referenced to the goods accompanying list and the text block position coordinates are calculated, the table coordinates are corresponded to the text block position coordinates one by one, and a calibrated goods accompanying list is obtained;

[0008] Key text blocks containing a company in the goods accompanying list are recognized, and the key text blocks are matched with a commodity database to obtain a supplier name of the goods accompanying list;

[0009] A table of the goods accompanying list corresponding to the supplier name is adapted to a plurality of pre-stored table templates, and it is determined whether a target table template exists;

[0010] If the target table template exists, the text blocks are filled into the target table template according to the text block coordinates, and a table information object is obtained;

[0011] When the product row number information in the table information object is not empty, it is determined that the accompanying order table and the target table template are accurately matched. Then, the table information object is determined as the recognition result, and the recognition result includes attribute text and product row number information.

[0012] Furthermore, before identifying the key text blocks of the company contained in the accompanying document, the process also includes: calculating the table coordinates and text block position coordinates with the accompanying document as a reference, and matching the table coordinates with the text block position coordinates one by one to obtain the calibrated accompanying document.

[0013] Furthermore, if the target table template exists, it also includes:

[0014] Obtain the coordinates of the first attribute column corresponding to several attribute texts in the header row of the accompanying order form, and the coordinates of the second attribute column corresponding to the header row of the target form template. The attribute texts include approval number, generic name, manufacturer, specifications, production date, expiration date, quantity, production batch number, unit price, and place of origin.

[0015] The coordinates of the second attribute column of several attribute texts are calibrated to the coordinates of their corresponding first attribute column.

[0016] Furthermore, after calibrating the coordinates of the second attribute column of the several attribute texts to their corresponding coordinates of the first attribute column, the method further includes:

[0017] Determine whether the accompanying order form meets the preset template adaptation conditions. The template adaptation conditions include: more than half of the attribute text of the accompanying order form is consistent with the attribute text of the target form template, or more than one-third of the attribute text of the accompanying order form is consistent with the attribute text of the target form template, and the coordinates of the first attribute column and the second attribute column of the approval number in the attribute text are consistent.

[0018] If it does not meet the requirements, then construct the corresponding general table based on the obtained coordinates of the first attribute column;

[0019] The text block of the accompanying order is filled into the general table to generate a corresponding table information object.

[0020] Furthermore, when the number of product rows in the table information object is empty, it is determined that the accompanying order table and the target table template are mismatched. Then, a corresponding general table is constructed based on the obtained coordinates of the first attribute column.

[0021] The text block of the accompanying order is filled into the general table to generate a corresponding table information object.

[0022] Furthermore, if the target table template does not exist, a corresponding general table is constructed based on the obtained coordinates of the first attribute column;

[0023] The text block of the accompanying order is filled into the general table to generate a corresponding table information object.

[0024] Furthermore, before obtaining the table information object, the following steps are also included:

[0025] The target table template is calibrated by using the text block position coordinates and table coordinates to obtain the table information object.

[0026] Further, key text blocks containing company information are identified in the accompanying order and matched with the product database to obtain the supplier name of the accompanying order, including:

[0027] Identify key text blocks containing the company name and sort them according to coordinates, then match them sequentially with the similarity of the names to supplier names in the product database;

[0028] The supplier name with the highest similarity is determined as the supplier name of the accompanying order.

[0029] Furthermore, the method also includes: matching the product name, product specifications, manufacturer, and approval number in the identification result with the sub-database in the product database corresponding to the target pharmacy to obtain a first identification result that matches the target pharmacy;

[0030] Based on the approval number and product name length in the identification result, a base score is calculated for the identification result. The base score is used to indicate the degree of matching between the identification result and the product database.

[0031] Select the first identification result that exceeds the base score, and match it again with the product database to obtain the second identification result that matches the product database;

[0032] The first and second recognition results are merged, and the top five recognition results are selected as the accurate recognition results.

[0033] Furthermore, the external OCR interface is the Alibaba OCR interface, and the preset output format is JSON format.

[0034] This application, based on existing OCR text recognition technology and combined with an existing product database, identifies keywords in the accompanying documents and adapts them to corresponding pre-stored target table templates. This allows for the retrieval of accompanying document data based on the target table templates, significantly improving the efficiency of document entry. Secondly, by combining the OCR recognition results with the absolute coordinates of the accompanying document, the application verifies the recognized coordinate information, resolving issues such as table tilting or text misalignment caused by user photography. Finally, this application also integrates with the pharmacy's corresponding product database, linking it to existing product information in the pharmacy. This allows for matching and calibration of the recognized information with the product database information, improving the accuracy of recognizing fuzzy text in the accompanying documents.

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Attached Figure Description

[0036] Figure 1 This is a typical example of a packing slip;

[0037] Figure 2 This is the accompanying order document obtained after OCR recognition;

[0038] Figure 3 This is a flowchart illustrating the steps of the OCR-based shipping document recognition and calibration method in the embodiments of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0040] It should be understood that the described embodiments are merely some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0041] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0042] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0043] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0044] like Figure 1 As shown, the accompanying delivery note is a general term for various documents included with a pharmacy's goods, such as sales slips and outbound delivery slips. As a pharmaceutical distribution company, medical institutions must establish sales records to ensure consistency between invoices, accounts, goods, and payments. The "manufacturer," "generic name," "dosage form," and "specifications" of the medicine on the accompanying delivery note are usually constant and only need to be checked during each inspection. However, the "batch number," "quantity," "production date," and "expiration date" are constantly changing and need to be manually entered into the inspection system each time.

[0045] Current warehousing technology using accompanying delivery notes utilizes general OCR text recognition technology, which can initially recognize most of the text on the delivery notes. However, due to varying user photography skills and inconsistent image quality, the directly recognized text is often unusable, affecting the accuracy of delivery note recognition and the efficiency of warehousing. Figure 2 As shown, some cells in the table have incorrect text recognition, and some text blocks overlap, making them unacceptable for verification.

[0046] To address the aforementioned issues, this application provides an OCR-based method for calibrating and recognizing accompanying delivery notes. Please refer to [link / reference]. Figure 3 The method includes the following steps:

[0047] Step S1: Obtain the accompanying order image, recognize the accompanying order image through an external OCR interface, and obtain the table of the accompanying order in a preset output format and the coordinates of the text blocks with the table as a reference.

[0048] The accompanying order image was obtained by taking a photo. The external OCR interface can be the Alibaba OCR interface, and the default output format is JSON. The coordinates of the table and text blocks are obtained through the OCR interface, and the coordinates of the text blocks correspond to the coordinates of the cells in the table. For example, if the coordinates of the text block "Production Batch Number" are (1,5), and its corresponding cell coordinates are (1,5), then the text block "Production Batch Number" is located in the fifth cell of the first row in the accompanying order table.

[0049] Step S2: Identify key text blocks containing the company name in the accompanying order and match them with the product database to obtain the supplier name of the accompanying order. This product database can be obtained through the ElasticSearch search engine database. Different suppliers will have different templates for their accompanying orders. If each accompanying order form is identified one by one as usual, identifying accompanying orders from the same supplier will waste more time. Therefore, the accompanying orders can be simply distinguished based on different supplier names to facilitate subsequent identification steps.

[0050] Step S3: Match the accompanying order form corresponding to the supplier name with multiple pre-stored form templates, and determine whether a corresponding target form template exists.

[0051] Different suppliers have different pre-stored form templates, so the pre-stored form templates can be quickly retrieved based on the current supplier name, reducing the recognition error of the accompanying order form.

[0052] Step S4: If the target table template exists, the text block is filled into the target table template according to the text block coordinates to obtain a table information object.

[0053] This table information object is used to indicate the data obtained after the text content of the accompanying invoice is filled in according to the pre-stored template. Specifically, after filling in the data, the target table template is divided into rows and columns according to certain rules, and the attribute text data of each column and the corresponding product data of each row are obtained, which constitutes the table information object.

[0054] Step S5: When the product row number information in the table information object is not empty, it is determined that the accompanying order table and the target table template are accurately matched. Then, the table information object is determined as the recognition result, and the recognition result includes attribute text and product row number information.

[0055] If the product row number information is not empty, it indicates that the template contains valid product data, and that the template is accurately matched with the accompanying order.

[0056] This application obtains the OCR recognition results, identifies keywords within them, and matches them with a product database to obtain the supplier name. It then determines the template corresponding to the supplier name and fills the identified text blocks into the template, enabling quick and convenient warehousing of goods accompanied by delivery orders. This recognition-based warehousing solution replaces manual warehousing calibration, improving the accuracy and efficiency of warehousing.

[0057] In a preferred embodiment, before identifying the key text blocks of the company contained in the accompanying order, the method further includes: calculating the table coordinates and text block position coordinates with the accompanying order as a reference, and matching the table coordinates with the text block position coordinates one by one to obtain the calibrated accompanying order.

[0058] Because the OCR interface may omit table data in the returned data, there will be some mismatch between the coordinates of the text blocks and the coordinates of the cells in the table. In this case, the coordinates of the text blocks and the cells in the table can be matched by using the table coordinates and text block position coordinates as a reference on the accompanying invoice.

[0059] The template corresponding to the supplier may not be completely identical to the supplier's accompanying order. Template adaptation only ensures consistency in table layout; for example, the header row may have five columns, but the text information within that header row may not be consistent. To ensure the accuracy of the text block's corresponding template, the target table template corresponding to the supplier needs to be checked and confirmed. Therefore, in a preferred embodiment, step S4, if the target table template exists, further includes:

[0060] Step S411: Obtain the coordinates of the first attribute column corresponding to several attribute texts in the header row of the accompanying order form, and the coordinates of the second attribute column corresponding to the header row of the target form template. The attribute texts include approval number, generic name, manufacturer, specifications, production date, expiration date, quantity, production batch number, unit price, and place of origin.

[0061] Step S412: Align the coordinates of the second attribute column of the several attribute texts to the coordinates of their corresponding first attribute column.

[0062] The header row in the packing slip is used to set the attribute text for the goods. The target table template also confirms this by checking whether the attribute text settings in the header row are consistent with those in the packing slip. Specifically, this is done by identifying the coordinates of different attribute texts. For example, if the first attribute column coordinate of "Approval Number" in the packing slip is (1, 3), while the second attribute column coordinate of "Approval Number" in the target table template is (1, 5), this indicates that the "Approval Number" is inconsistent between the packing slip and the target table template. It needs to be calibrated to ensure that the second column coordinate of "Approval Number" in the target table template is also (1, 3). This ensures that the content in the target table template corresponds one-to-one with the packing slip, reducing errors in content adaptation.

[0063] There are three abnormal situations during the process of determining whether a suitable template exists based on the supplier name and identifying the supplier based on the suitable template: First, although a target table template exists, the currently stored template corresponding to the supplier is no longer compatible due to updates to the supplier's accompanying order; second, although the existing template is confirmed to match the accompanying order successfully, the populated table does not contain product information; third, no suitable template exists. The quickest solution is to reconstruct a general template based on the current text coordinates and the obtained attribute column coordinates.

[0064] Based on the first scenario described above, in a preferred embodiment, after confirming the supplier template, the method further includes:

[0065] Step S421: Determine whether the accompanying order form meets the preset template adaptation conditions. The template adaptation conditions include: more than half of the attribute text of the accompanying order form is consistent with the attribute text of the target form template, or more than one-third of the attribute text of the accompanying order form is consistent with the attribute text of the target form template, and the coordinates of the first attribute column and the second attribute column of the approval number in the attribute text are consistent.

[0066] Step S422: If it does not meet the requirements, construct the corresponding general table based on the obtained coordinates of the first attribute column.

[0067] Step S423: Fill the text block of the accompanying order into the general table to generate a corresponding table information object.

[0068] If the target table template does not meet the above adaptation conditions, it indicates that the supplier's accompanying document table has updated its template. In this case, a more suitable general table needs to be reconstructed to avoid recognition errors caused by low compatibility between the accompanying document table and the template. In a specific example, the text blocks of the accompanying document are filled into the general table, corresponding to the generated table information object. Similar to step S4 above, where the text block values ​​are filled into the target table template, the filling is done by matching the coordinates of the text block to the coordinates of the attribute column and the coordinates of the cells in the table.

[0069] Based on the second scenario described above, in a preferred embodiment, the following steps are included: Step S431: When the product row number information in the table information object is empty, it is determined that the accompanying order table and the target table template are mismatched. Then, based on the obtained coordinates of the first attribute column, a corresponding general table is constructed. Step S432: The text blocks of the accompanying order are filled into the general table, and a corresponding table information object is generated.

[0070] Based on the third scenario described above, in a preferred embodiment, the following steps are included:

[0071] Step S441: If the target table template does not exist, construct a corresponding general table based on the obtained coordinates of the first attribute column. Step S442: Fill the general table with the text block from the accompanying document, and generate a corresponding table information object.

[0072] In one specific embodiment, before obtaining the table information object, the method further includes:

[0073] The target table template is calibrated using text block position coordinates and table coordinates to obtain the table information object. This step serves the same purpose as calibrating the OCR recognition result: both use the accompanying document as a reference, ensuring that the text block position coordinates and table coordinates on that document are absolute coordinates, preventing errors and omissions, and enabling precise calibration of the table data.

[0074] In one specific embodiment, step S2 involves identifying key text blocks related to the company contained in the accompanying order and matching them with the product database to obtain the supplier name of the accompanying order, including:

[0075] Step S21: Identify key text blocks containing the company and sort them according to coordinates, then match them sequentially with the similarity of supplier names in the product database.

[0076] Supplier names are typically company names such as XX Company, XX Limited Liability Company, or XX Corporation. Therefore, key text blocks containing the company name can be identified. In other embodiments, these can also be text blocks related to the supplier's name, such as "enterprise" or "group." The coordinate sorting can be either Y-coordinate sorting or X-coordinate sorting.

[0077] Step S22: Determine the supplier name with the highest similarity as the supplier name of the accompanying order.

[0078] The higher the similarity, the closer the current text block is to the supplier's name. Specifically, the top five text blocks can be selected and compared with the Elasticsearch product database.

[0079] Since the product data on the packing slip and the product data in the pharmacy system may not be exactly the same, some matching rules are needed for accurate inventory calibration. In a preferred embodiment, the method further includes:

[0080] The product name, specifications, manufacturer, and approval number in the identification results are matched with the sub-database of the product database corresponding to the target pharmacy to obtain a first identification result that matches the target pharmacy. Product data for different pharmacies may differ from the ES product database; therefore, it is necessary to match the content coarsely calibrated using the ES product database with the pharmacy's sub-database.

[0081] Based on the approval number and product name length in the identification result, a base score is calculated for the identification result. This base score indicates the degree of matching between the identification result and the product database. The base score can be set according to different pharmacy rules. For example, if the pharmacy's product data covers a wide range of product types, its base score can be set lower; conversely, it can be set higher.

[0082] A first identification result exceeding the baseline score is selected and matched again with the product database to obtain a second identification result that matches the product database.

[0083] The first and second recognition results are merged, and the top five recognition results are selected as the accurate recognition results.

[0084] Whether it's the first or second identification result, when retrieving data from the product database, each row in each result will have a score. The score is determined by the database based on the similarity between the matching criteria and the results. Although the first and second identification results are retrieved from two different databases (the product database and the pharmacy sub-database), the two databases have roughly the same data volume and structure, so the scoring criteria are essentially the same. Therefore, the results can be sorted according to their combined scores, with higher scores ranking higher.

[0085] This step involves correlating and matching the acquired data with the sub-database of the current pharmacy. Previous steps, based on the ES product database, performed calibration using general logic, detached from the product data in the target pharmacy. This step, however, uses the product data of the target pharmacy for calibration, clearly defining the calibration target and resulting in more accurate calibration results.

[0086] This application, based on existing OCR text recognition technology and combined with an existing product database, identifies keywords in the accompanying documents and adapts them to corresponding pre-stored target table templates. This allows for the retrieval of accompanying document data based on the target table templates, significantly improving the efficiency of document entry. Secondly, by combining the OCR recognition results with the absolute coordinates of the accompanying document, the application verifies the recognized coordinate information, resolving issues such as table tilting or text misalignment caused by user photography. Finally, this application also integrates with the pharmacy's corresponding product database, linking it to existing product information in the pharmacy. This allows for matching and calibration of the recognized information with the product database information, improving the accuracy of recognizing fuzzy text in the accompanying documents.

[0087] It should be understood that the embodiments of this application are not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from their scope. The scope of the embodiments of this application is limited only by the appended claims. The embodiments described above merely illustrate several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the embodiments of this application, and these all fall within the protection scope of the embodiments of this application.

Claims

1. An OCR-based waybill recognition calibration method, characterized in that, The method comprises the following steps: An image of the accompanying invoice is obtained, the image of the accompanying invoice is recognized through an external OCR interface, and a table of the accompanying invoice in a preset output format and text block coordinates referenced to the table are obtained; Table coordinates referenced to the accompanying invoice and text block position coordinates are calculated, the table coordinates are corresponded to the text block position coordinates one by one, and the accompanying invoice is calibrated to obtain a calibrated accompanying invoice; Key text blocks containing a company in the accompanying invoice are recognized and matched with a commodity database to obtain a supplier name of the accompanying invoice; The table of the accompanying invoice corresponding to the supplier name is adapted to a plurality of pre-stored table templates, and it is determined whether a target table template exists; If the target table template exists, the text blocks are filled into the target table template according to the text block coordinates to obtain a table information object, which specifically comprises the following sub-steps: A first attribute column coordinate corresponding to a plurality of attribute texts in a table header row of the table of the accompanying invoice and a second attribute column coordinate corresponding to the attribute texts in a table header row of the target table template are obtained, wherein the attribute texts include an approval number, a generic name, a manufacturer, a specification, a production date, an expiration date, a quantity, a production batch number, a unit price and a place of production; The second attribute column coordinates of the attribute texts are calibrated to the first attribute column coordinates corresponding thereto; It is determined whether the table of the accompanying invoice meets a preset template adaptation condition, the template adaptation condition comprises that more than half of the attribute texts of the table of the accompanying invoice are consistent with the attribute texts of the target table template, or more than one third of the attribute texts of the table of the accompanying invoice are consistent with the attribute texts of the target table template, and the first attribute column coordinates and the second attribute column coordinates of the approval number in the attribute texts are consistent; If not, a corresponding general table is constructed according to the obtained first attribute column coordinates; The text blocks of the accompanying invoice are filled into the general table to correspondingly generate a table information object; If the target table template does not exist, a corresponding general table is constructed according to the obtained first attribute column coordinates; The text blocks of the accompanying invoice are filled into the general table to correspondingly generate a table information object; When commodity row number information in the table information object is not empty, it is determined that the table of the accompanying invoice is accurately adapted to the target table template, and the table information object is determined as a recognition result, the recognition result comprises attribute texts and commodity row number information.

2. The OCR-based accompanying invoice recognition and calibration method according to claim 1, wherein: When commodity row number information in the table information object is empty, it is determined that the table of the accompanying invoice is incorrectly adapted to the target table template, and a corresponding general table is constructed according to the obtained first attribute column coordinates; The text blocks of the accompanying invoice are filled into the general table to correspondingly generate a table information object.

3. The OCR-based waybill identification and calibration method of claim 1, wherein, Before obtaining the table information object, the following steps are further included: The target table template filled through the text block position coordinates and the table coordinates is calibrated to obtain the table information object.

4. The OCR-based waybill identification and calibration method of claim 1, wherein, Identifying a key text block containing a company in the waybill and matching it with a commodity database to obtain a supplier name of the waybill, comprising: Identifying a key text block containing a company and sorting according to coordinates, and sequentially matching the similarity of the supplier name in the commodity database; Determining the supplier name with the maximum similarity as the supplier name of the waybill.

5. The OCR-based waybill identification and calibration method according to any one of claims 1-4, characterized in that, The method further comprises: Matching the commodity name, commodity specification, manufacturer, and approval number in the identification result with a sub-database in the commodity database corresponding to a target pharmacy to obtain a first identification result conforming to the target pharmacy; According to the approval number and the length of the commodity name in the identification result, a base score of the identification result is calculated, which is used to indicate the matching degree of the identification result and the commodity database; Selecting the first identification result exceeding the base score and matching it with the commodity database again to obtain a second identification result conforming to the commodity database; Merging the first identification result and the second identification result and selecting the top five identification results as accurate identification results.

6. The OCR-based waybill identification calibration method according to claim 5, characterized in that: The external OCR interface is an Ali OCR interface, and the preset output format is a JSON format.

Images