Intelligent track-following card data identification and entry method

Abstract

The invention discloses a track-following card data intelligent identification and entry method, and solves the problems of low efficiency and high error rate of traditional manual entry of enterprise track-following card data. According to the scheme, the method comprises the following steps: pre-establishing identification templates of track-following cards in various formats, and uploading track-following card image scanning header two-dimensional codes to match the identification templates; cutting a corresponding field region according to coordinate information defined in the template; identifying and extracting field contents in the table, decoding the right two-dimensional code of each process, and forming a complete process record by the right two-dimensional code and the field data of the row; each process record is indexed according to the line number or the unique process ID in the two-dimensional code; and the system arranges the structured identification data and transmits the structured identification data to an enterprise internal ERP, MES or quality system. According to the scheme, the data processing efficiency is improved, data traceability is realized, template management and recognition model continuous optimization are supported, an automatic closed-loop process from paper information to system data is constructed, and the method is an important basic tool for promoting enterprise intelligent manufacturing and informatization construction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial informatization, in particular to a kind of intelligent identification and entry method of follow-up card data. BACKGROUND

[0002] Follow-up card is also called "process flow card" or "process tracking card", and is an important production management tool for recording and tracking product flow, processing, inspection and other information in each process in manufacturing enterprises. It usually exists in paper form and flows with the product in the production link. Its main functions include: 1) guiding production operation, by clearly indicating product number, process route, process requirements and other information, to help operators correctly perform each process; 2) recording process execution, such as operation time, operator, equipment number, processing parameters, test results, etc., to ensure that each step of operation is traceable; 3) realizing quality traceability, when product quality problems occur, the responsible process and related personnel can be quickly found through the follow-up card, to improve problem response and improvement efficiency; 4) promoting production process control, by recording and transmitting production progress information in real time, to provide basis for scheduling management, capacity analysis and work time statistics. Overall, follow-up card is a key carrier connecting production site and management system, ensuring the orderly and efficient operation of production.

[0003] Under the current digital management environment, the information on the follow-up card is often required to be integrated with MES (Manufacturing Execution System) or ERP (Enterprise Resource Planning System), at which time the data on the paper follow-up card needs to be entered into the system.

[0004] However, there are the following difficulties in actual data entry: the operator needs to check each card and manually enter the information, which cannot keep up with the production rhythm of the product batch, resulting in low efficiency; due to unclear handwriting, fatigue work and other factors, misreading, misfilling and other situations are prone to occur, affecting data accuracy; due to the variety of card formats and complexity of handwritten information, field extraction and standardization processing are difficult; manually entered data is difficult to be linked with MES, ERP and other systems in real time, affecting the enterprise's quick response and decision-making on production status and quality problems. SUMMARY

[0005] The present application aims to provide a follow-up card data intelligent identification and entry method to solve the problems raised in the background.

[0006] The present application provides a follow-up card data intelligent identification and entry method, comprising the following steps:

[0007] S1, pre-establishing identification templates for multiple formats of follow-up cards in the system;

[0008] S2, standardize the scanned uploaded follow-up card image, and match a recognition template after scanning a two-dimensional code on a right side of a table head of the follow-up card;

[0009] S3, crop a corresponding field area from the image according to coordinate information defined in the template;

[0010] S4, the system recognizes and extracts all field contents in the table, recognizes and decodes a two-dimensional code on a right side of each process, and combines the two-dimensional code and all field data of the row into a complete process record;

[0011] S5, the system independently encapsulates each complete process record, and indexes according to a row number or a unique process ID in the two-dimensional code;

[0012] S6, the system sorts the structured recognition data, and transmits the data to an ERP, MES or quality system in an enterprise through an interface or a database pushing mode.

[0013] Further, the step S1 further includes:

[0014] S11, an operator manually labels positions of each field on a clear follow-up card sample using a graphical template labeling tool, and names each area, and the fields include: table head fields (such as a work order number, a material code, a product name and the like), table fields (such as a "process name", a "test result" and a "signature" of each row of processes), two-dimensional code positions (a table head two-dimensional code and a two-dimensional code of each row of processes), a handwritten field area, a signature image area, a printed text area and the like, each field corresponds to an image area, and a rectangle is used to represent the image area, and a coordinate form of the rectangle is a four-tuple: R i =(x i ,y i ,w i ,h i ), wherein x i ,y i is a left upper corner coordinate of the area, w i ,h i is a width and a height, all field coordinates are defined based on an original image resolution, and coordinate normalization or image alignment is needed during recognition.

[0015] S12, for a table type follow-up card, each card can include a fixed or non-fixed number of process rows, and a structure rule of a table area needs to be defined in the template, and the structure rule includes: a multi-row starting area R start =(x0,y0,w0,h0); a row height h row ; a row upper limit N max ; relative positions of columns in the row; in addition, a relative position of a two-dimensional code on a right side of each row needs to be determined, which is used for subsequent automatic detection and binding of data of the row.

[0016] S13, define the field type in the table, such as printed text, handwritten field, signature image, two-dimensional code, time field, which is directly determined by the field type which OCR model, preprocessing strategy and post-checking method to be used subsequently.

[0017] S14, assign a unique number to each template, which is written into the card table header two-dimensional code, realize "scan card to identify template" entry, two-dimensional code structure is JSON structure.

[0018] Further, the step S2 further comprises: since the card is usually scanned or photographed in a factory environment, the original image may have problems such as rotation, tilt, shadow, uneven illumination, stain, etc., and image standardization processing must be done: calculate the transformation matrix M using Hough straight line edge detection to restore the image to a normal viewing angle; calculate the main direction angle θ by detecting the horizontal line in the image, and automatically rotate the image; use CLAHE (contrast limited histogram equalization) to improve text clarity and reduce background interference.

[0019] After image standardization processing, the OCR engine is used to recognize the two-dimensional code, and the structured data format is obtained by analyzing the two-dimensional code content: the template number is used to find a specific template in the local or remote database, each template structure includes field coordinates, recognition type, line structure, etc., and after finding the template, it is mounted to the current recognition task, at this time all recognition steps such as image correction, field cropping, table cutting, etc. will be operated according to this template, so as to realize multi-template dynamic switching when recognizing multiple formats of tables, and realize mixed scanning recognition.

[0020] Further, the step S3 further comprises: the system extracts the position information of each field in the image according to the currently loaded template, and each field region is stored in the template as: i = (x i ,y i ,w i ,h i ), crop the corresponding region from the image to obtain an "image sub-block" of each field, which is used for recognition in the next stage; In the operation, slight displacement in the field image is inevitable, so the system adds a redundant boundary to the coordinate region to improve fault tolerance; the system records the corresponding field name and field type for each image block, which is used for classification processing in the next stage.

[0021] Further, the step S4 further comprises: for the extracted field image, the system assigns a recognition engine according to the field type: printed text uses standard OCR engine such as PaddleOCR, EasyOCR, Tesseract; handwritten content uses a special handwriting recognition model such as CRNN, TrOCR, GPT-4V fine-tuned; signature image remains the image and does not do text recognition; date / number field OCR recognition result is subsequently formatted and verified by regular expression. After the content recognition is completed and extracted, according to the table structure defined in the template, the system can confirm the position of the two-dimensional code on the right side of each process line by line, parse the two-dimensional code to get the unique ID identification, and the system combines the two-dimensional code unique ID identification with the field area recognition result of the line to form a structured "process data item", and the multiple data item records are independent of each other.

[0022] Further, the step S5 further comprises: the MES / ERP system within the enterprise usually requires standardization of data format, therefore the system performs standardization processing on the recognition result of each follow-up card: automatically converts the field name and field value structure according to the template, and uniformly encapsulates into standard key-value structure JSON or database table format.

[0023] Further, the step S6 further comprises: field verification is required before uploading, each field recognition value has a confidence score, and the content below the set threshold is automatically marked as "manual review is required", and the recognition image and intermediate result are saved for man-machine collaborative review. Through the POST JSON request, the structured data is pushed to the interface of the MES or middle platform system in real time: each card is pushed once after recognition, and each process can be pushed in pieces, supporting result callback and writing state feedback. If the on-site network is unstable or needs to be imported after manual review, the data can be first landed in the intermediate database, and then synchronized into the warehouse by the background system according to the business rules.

[0024] The continuous learning and optimization capability of the recognition system is built, when the manual modification of the recognition result on the interface is recorded, the system automatically records the "image + correct value" sample, adds it to the training set, and is used for fine-tuning of the handwriting recognition model and the field classifier model periodically; the images of frequently incorrect fields are clustered and analyzed to determine whether the template is biased, the font style is abnormal, the field structure is fuzzy, etc., to guide the subsequent template optimization or shooting specification adjustment.

[0025] Each row of data in the follow-up card is bound with the two-dimensional code on the right side to form an independent process record, the system uploads each process one by one or selectively, the management personnel can search according to process, time, personnel, card in multiple dimensions, each record can be independently reviewed, modified, confirmed and uploaded, which adapts to the management needs of the enterprise in multi-role collaboration, phased processing and abnormal tracking.

[0026] The application provides a follow-up card data intelligent identification and input method.

[0027] The application realizes automatic conversion of the whole process from paper record to enterprise system data: the identification template is determined through the table head two-dimensional code, and the system can adapt to follow-up cards of various formats; the two-dimensional code on the right side of each process is used to realize process level data binding, so that each row of data can be independently identified and accurately traced. The information on the whole card is automatically extracted by OCR, two-dimensional code analysis and image processing technology, and is structured and packaged and uploaded to the MES / ERP system, so that data real-time input and business immediate response are realized. The system supports template version management and continuous learning optimization of the identification model, and guarantees long-term adaptability and accuracy. The scheme not only greatly improves the efficiency and reliability of production data input, but also enhances the transparency of process management and the responsibility tracing ability, and is an important infrastructure for promoting the upgrade of factories from manual management to intelligent manufacturing. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A flowchart of the steps of the follow-up card identification method.

[0029] Figure 2 A follow-up card template.

[0030] Figure 3 A follow-up card template with simulated filled content. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0032] In one or more embodiments, the application provides a follow-up card data intelligent identification and input method as shown in Figure 1 The method comprises the following steps:

[0033] S1, an identification template of a follow-up card of various formats is pre-established in the system;

[0034] S2, the scanned follow-up card image is standardized, and the right side two-dimensional code of the follow-up card table head is scanned and matched to determine the identification template after the standardization is completed;

[0035] S3, according to the coordinate information defined in the template, the corresponding field area is cropped from the image;

[0036] S4. The system identifies and extracts the content of all fields in the table, identifies and decodes the QR code on the right side of each process, and combines the QR code identifier with all field data of the row to form a complete process record.

[0037] S5. The system independently encapsulates each complete process record and indexes it by the line number or the unique process ID in the QR code;

[0038] S6. The system organizes structured identification data and transmits it to the enterprise's internal ERP, MES, or quality systems via interface or database push.

[0039] by Figure 2 Taking the above-mentioned card template as an example, in this embodiment, in step S1, the card is uploaded. Figure 3 The example card with filled content shown is processed to ensure that the image resolution is not less than 300 DPI, and scaled to a uniform size of A42480×3508px. The standard image width and height W_std and H_std are recorded for subsequent field coordinate normalization.

[0040] Use the LabelImg annotation tool to manually create rectangular annotations for the field areas and data item cell areas on the image, and assign logical names to each area: Header fields (including branch factory, group, product number, part number, date, furnace number, batch number); Table fields (including process, process name, process technical requirements, processor signature, self-inspection result (date, measured value), special inspection result (inspection result, date, signature), remarks, and identification area); QR code fields (including header QR code and QR code for each row); Handwritten fields (including self-inspection result and special inspection result); Image fields (including handwritten signature blocks for processor signature and special inspection result signature); and date field. All rectangular coordinates use the standard format: R. i =(x i ,y i ,w i ,h i ), where (x i ,y i ) is the coordinate of the top left corner, (w i ,h i () represents the width and height, in pixels.

[0041] In this embodiment, since the tracking card is a table containing multiple rows of process records, a multi-row table structure and the position of each row's fields are defined to avoid repeatedly selecting field content for each row. In a specific embodiment, the table's starting area, row height, and number of rows are marked: {"table_start":{"x":5,"y":130},"row_height":48,"max_rows":5}. The position of each row's fields is defined, along with the offset_x and width of each column's fields relative to the entire row: "table_fields":[

[0042] {"name":"Process Number","offset_x":5,"width":38,"type":"printed_text"},

[0043] {"name":"Process Name","offset_x":43,"width":95,"type":"printed_text"},

[0044] {"name":"Processor's Signature","offset_x":288,"width":55,"type":"handwritten"},

[0045] {"name":"Test Result","offset_x":495,"width":75,"type":"printed_text"},

[0046] {"name":"QR code","offset_x":910,"width":90,"type":"qrcode"}];

[0047] The offset positions of all fields are not listed here; only the fields mentioned above are used as examples. The actual pixel position of the content corresponding to a certain field in the i-th row is: X field =X table +offset x Y field =Y table +i·row_height.

[0048] After determining the position information of the field content, recognition strategies are configured for handwritten, printed, and image content respectively, and the recognition strategies are written into the template. The template is configured with a unique number, and a two-dimensional code is written to realize "scan code to recognize template"; the two-dimensional code is printed in the upper right corner of the table header, and its coordinate position is defined in the template, which can be stored as {"field_name":"template two-dimensional code","type":"qrcode","position":{"x":870,"y":38,"w":120,"h":120}}. After the template configuration is completed, it is stored in the database in the.json file format.

[0049] In this embodiment, in step S2, since the tracking card is usually scanned or photographed in a factory environment, the original image may have problems such as rotation, tilt, shadow, uneven lighting, stains, etc., and image standardization processing must be performed: a transformation matrix M is calculated using Hough straight line edge detection to restore the image to a normal viewing angle; the main direction angle θ is calculated by detecting the horizontal line in the image, and the image is automatically rotated; CLAHE (contrast limited histogram equalization) is used to improve text clarity and reduce background interference.

[0050] After image standardization processing, the OCR engine uses the OpenCV+pyzbar algorithm to recognize the two-dimensional code on the right side of the table header, and the two-dimensional code content is encoded using JSON. The system checks the field structure to ensure that the content parsing is safe and reliable. The recognized two-dimensional code data is parsed into a structured format, and the template number in it is used to search for the template definition in the local or remote database. The corresponding template structure including field coordinates, recognition type, and row structure is searched from the template library and mounted to the current recognition task. At this time, all recognition steps such as image correction, field cropping, and table splitting will be operated according to this template.

[0051] In this embodiment, in step S3, the system extracts the region coordinates of each field and its corresponding filled content according to the currently loaded template, and crops the corresponding region from the image. Each field region has been defined in the template as R i =(x i ,y i ,w i ,h i ), and the system uses these coordinates to perform cropping operations in the processed image to obtain an "image sub-block" for each field, which is used for recognition in the next stage. Note that the coordinates defined in the template need to be dynamically scaled according to the resolution of the original image to adapt to different scanning resolutions; considering that there is inevitable slight displacement in the field image, the system adds a certain redundancy boundary (such as ±5px) to the coordinate region to improve fault tolerance; for each image block, the system records its corresponding field name, field type, such as OCR, handwritten, two-dimensional code, and image preservation, which is used for classification processing in the next stage.

[0052] In this embodiment, in step S4, for the extracted field image in each row, the system assigns a recognition engine according to the field type: PaddleOCR engine is used for printed text; TrOCR special handwriting recognition model is used for handwritten content; signature image is kept as image and no text recognition is performed; OCR recognition result of date / number field is subsequently formatted and checked by regular expression. After content recognition and extraction, according to the table structure defined in the template, including starting position, row height, maximum number of rows, etc., the system can judge the position of the two-dimensional code on the right side of each operation row, parse the two-dimensional code to get the unique ID, and combine the two-dimensional code unique ID with the field area recognition content result of the row to form a structured “operation data item”. Multiple data item records are independent of each other.

[0053] In this embodiment, in step S5, the MES / ERP system within the enterprise usually requires uniform data format standards, so the system needs to first standardize the recognition results of each trace card. In a specific embodiment, the overall json structure is defined as follows:

[0054]

[0055]

[0056] template_id is the recognition template number, header_qr is the original content recognized by the header two-dimensional code of the trace card, which is the unique number or business code of the trace card; card_fields field is the public information of the whole card, i.e. the header information; operations is the independent data item of each operation row: op_id is the unique number generated for the operation record, op_qr is the original content recognized by the two-dimensional code on the right side of the operation, which is used to identify the code of the operation in the business system. The system automatically converts the field name and field value structure according to the template and uniformly encapsulates it into a standard JSON format.

[0057] In this embodiment, in step S6, field verification is performed before uploading. Each field recognition value has a confidence score. Content below the set threshold is automatically marked as “manual review required”, and the recognition image and intermediate result are saved for human-machine collaborative review. Through a POST JSON request, structured data is pushed to the interface of the MES or middle platform system in real time: each card is pushed once after recognition, and each operation can be pushed in pieces, supporting result callback and writing state feedback. If the on-site network is unstable or needs to be imported after manual review, the data can be first dropped into the intermediate database, and then the background system can be used to synchronize the data into the database according to the business rules.

[0058] Each row of data in the card is bound to the two-dimensional code on its right to form an independent process record. The system uploads each process one by one or selectively. Management personnel can search by process, time, personnel, and card in multiple dimensions. Each record can be independently reviewed, modified, confirmed, and uploaded, which meets the management needs of multi-role collaboration, phased processing, and abnormal tracking in enterprises.

[0059] The system has the ability to continuously learn and optimize. After manual modification of the recognition result on the interface, the system automatically records the "image + correct value" sample and adds it to the training set for fine-tuning of the handwriting recognition model and field classifier model. It also clusters and analyzes images of frequently incorrect fields to determine whether the template is biased, the font style is abnormal, or the field structure is ambiguous, etc. to guide subsequent template optimization or shooting specification adjustment.

[0060] In this embodiment, if there are abnormalities such as misalignment of fields and structural misplacement in recognition, the system should prompt that the current template may have been invalidated and suggest re-sampling to generate a new version.

[0061] Support for "template inheritance": new templates can copy the field structure based on old templates and only adjust part of the coordinates or recognition logic.

[0062] Support for "template rollback": historical cards can still be parsed using old templates, ensuring that historical data can be continuously parsed.

[0063] In this embodiment, the recognized data can not only be uploaded to ERP / MES, but also be linked with more systems, such as:

[0064] Backfill each process data to the quality inspection system and traceability system;

[0065] Store the signature image in the quality responsibility chain management platform;

[0066] Use metadata such as recognition time and signature time for performance system analysis;

[0067] In the warehouse system, automatically generate barcodes or packaging slips based on the recognition data.

Claims

1. A method for intelligent identification and entry of data in a follow-up card, characterized in that, The method comprises the following steps: S1, a plurality of format tracking card recognition templates are pre-established in the system; S2, the tracking card image uploaded by scanning is standardized, and the recognition template is determined by matching the two-dimensional code on the right side of the tracking card table header after the standardization is completed; S3, according to the coordinate information defined in the template, the corresponding field area is cut out from the image; S4, the system extracts all field contents in the table, decodes the two-dimensional code on the right side of each process, and combines the two-dimensional code mark and all field data of the row into a complete process record; S5, the system independently encapsulates each complete process record, and indexes according to the row number or the unique process ID in the two-dimensional code; S6, the system arranges the structured recognition data, and transmits the structured recognition data to the ERP, MES or quality system in the enterprise through an interface or a database pushing mode.

2. The method of claim 1, wherein, In step S1, the steps include: S11, the operator manually selects and names the positions of various fields on the follow-up card sample diagram, and these fields include: table header field, table field, two-dimensional code field, handwritten field, signature image field, date field, and the image area of each field is constructed into a four-tuple: R i =(x i ,y i ,w i ,h i ), wherein (x i ,y i ) is the left upper corner coordinate of the area, (w i ,h i ) is the width and height of the area, and all field coordinates are defined based on the original image resolution, and coordinate normalization or image alignment is required during identification; S12, the structure rules of the table area in the template are defined, including: a plurality of row starting areas, the height of each row, the upper limit of the number of rows, the relative position of each field column in the row, and the relative position of the two-dimensional code on the right side of each row; S13, the field types in the table are defined, including printed text, handwritten field, signature image, two-dimensional code and time field, and the field type directly determines which OCR model, preprocessing strategy and post-checking method to be used subsequently; S14, a unique number is allocated to each template, and the number is written into the card table header two-dimensional code, so that the "card scanning and template recognition" entrance is realized, and the two-dimensional code structure is in JSON structure.

3. The method of claim 1, wherein, In step S2, first, image preprocessing is performed: a transformation matrix M is calculated by using Hough straight line edge detection to restore the image to a normal viewing angle; the main direction angle θ is calculated by detecting the horizontal line in the image, and the image is automatically rotated; the CLAHE (contrast limited histogram equalization) is used to improve the text clarity and reduce background interference; after the processing is completed, the OpenCV+pyzbar algorithm is used for two-dimensional code recognition; the two-dimensional code content is encoded in JSON, and the template number is parsed after the system checks the structure; the template number is used to load the corresponding template structure, to drive the subsequent image correction and field recognition, and to realize the dynamic switching and mixed scanning recognition of multiple templates.

4. The method of claim 1, wherein, In step S3, after the image standardization is completed, the system extracts the coordinate information of each field according to the currently loaded template, cuts out the corresponding area from the image, and obtains an "image sub-block" of each field, which is used for recognition in the next stage; in the operation, slight displacement of the on-site image is inevitable, so the system increases the redundant boundary of the coordinate area to improve the fault tolerance; the system records the corresponding field name and field type for each image block, which is used for classification processing in the next stage.

5. The method of claim 1, wherein, In step S4, the system allocates a recognition engine according to the field type: a standard OCR engine is used for printed text, a special handwriting recognition model is used for handwritten content, a signature image is reserved, and a regular expression is used for format verification of the OCR recognition result of the date / number field; The system combines each row right two-dimensional code and the field recognition result of the row to form a structured "process data item", so that each row is an independent structured data record.

6. The method of claim 1, wherein, In step S5, the MES / ERP system inside the enterprise requires data format standardization, so the system standardizes the recognition results of each trace card: according to the template, the field name and field value structure are automatically converted, and the standard key-value structure JSON or database table format is uniformly packaged.

7. The method of claim 1, wherein, In step S6, field verification is performed before uploading. Each field recognition value has a confidence score. If the score is below the set threshold, the content is automatically marked as "manual review required". The recognition image and intermediate results are saved for human-machine collaborative review. Through a POST JSON request, structured data is pushed to the MES or middle platform system interface in real time: each card is pushed once after recognition, and each process can be pushed in sections. It supports result callback and state feedback. If the network is unstable or needs to be imported after manual review, the data can be first dropped into the intermediate database, and then the background system can be synchronized according to the business rules to import it into the database.

8. The method of claim 7, wherein, The system has continuous learning and optimization capabilities. When manual modifications are made to the recognition results on the interface, the system automatically records the "image + correct value" samples and adds them to the training set for fine-tuning of the handwriting recognition model and field classifier model. The system analyzes the images of frequently incorrect fields to determine whether the template is biased, the font style is abnormal, or the field structure is ambiguous, etc. to guide subsequent template optimization or shooting specification adjustment.

9. The method of claim 1, wherein, Each row of data in the trace card is bound to the two-dimensional code on its right side to form an independent process record. The system uploads each process one by one or selectively. Management personnel can search by process, time, personnel, and card in multiple dimensions. Each record can be independently reviewed, modified, confirmed, and uploaded to meet the management needs of multi-role collaboration, phased processing, and abnormal tracking in enterprises.

Images