Machining drawing processing information extraction method based on YOLO + OCR

Abstract

The invention relates to the technical field of computer vision and intelligent information processing, in particular to a machining drawing information extraction method based on YOLO + OCR, which comprises the following steps: sequentially carrying out graying processing, Gaussian filtering noise reduction and Canny edge detection on a machining drawing, and extracting and screening drawing frames and drawing areas; detecting a text group in the drawing by adopting a YOLO-OBB algorithm; and performing discrete classification prediction on the rotation angle of the detected text group image. According to the method, text pixel loss caused by direct zooming is avoided through drawing preprocessing, the 360-degree text direction problem is solved by utilizing a YOLO-OBB algorithm and a four-classification neural network model, PMI information is aligned and recognized in combination with a special symbol model, global parameters are extracted through natural language processing, and finally errors are monitored and corrected. The problems of text pixel loss, insufficient direction processing, poor symbol and text association and the like in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of computer vision and intelligent information processing technology, specifically a method for extracting machining information from mechanical processing drawings based on YOLO+OCR. Background Technology

[0002] With the rapid development of intelligent manufacturing and digital factories, the demand for automated processing of drawing information in the machining field is becoming increasingly urgent. Currently, in the manufacturing industry, the machining drawings submitted by users to factories are typically two-dimensional or three-dimensional exported drawings, containing a large amount of unstructured special machining requirements, such as surface roughness, dimensional tolerances, and geometric tolerances. This critical information is often scattered across different areas of the drawing in the form of text annotations or professional symbols, currently relying mainly on manual interpretation. Traditional manual interpretation methods require engineers to check each annotation on the drawing item by item, taking an average of 1-3 hours per drawing for complex parts, severely hindering the automation process of process design. According to industry statistics, approximately 37% of defects in non-standard parts machining stem from misinterpretation of special requirements in the drawings.

[0003] Visual recognition also presents the following technical challenges:

[0004] The text distribution characteristics are complex: the technical requirements stipulate that the text is scattered in areas such as the title bar and enlarged details, and is often tilted at 15° to 75° to adapt to the viewing angle. Traditional OCR has an accuracy rate of less than 60% for recognizing this type of tilted text, which is insufficient to meet the accuracy requirements.

[0005] Technical symbol recognition barrier: Diameter symbol Countersunk hole Mechanical symbols are composed of line segments in vector graphics, which existing OCR cannot recognize at all, and will disrupt the coherence of the text, affecting the overall information extraction.

[0006] Structured representation has special requirements: there are many "symbol + parameter" structures in machining (such as "Ra0.8"), which need to recognize both symbol semantics and text numerical values ​​at the same time. Existing technologies are difficult to take into account the correlation between the two.

[0007] To address the aforementioned issues, existing technologies attempt to extract information using optical character recognition (OCR) and machine learning algorithms, specifically convolutional neural networks (CNN).

[0008] However, the existing technical solutions have obvious drawbacks in use:

[0009] Directly scaling the drawing causes text pixels to be lost;

[0010] The 360° text orientation issue remains unresolved.

[0011] The relationship between technical symbols and text is not adequately addressed.

[0012] These shortcomings result in a low average recognition accuracy of existing systems in real industrial scenarios, failing to meet the demands of intelligent manufacturing for automated extraction of drawing information. Summary of the Invention

[0013] The purpose of this invention is to provide a method for extracting machining information from machining drawings based on YOLO+OCR, so as to solve the problems mentioned in the background art.

[0014] To achieve the above objectives, the present invention provides the following technical solution: a method for extracting machining information from machining drawings based on YOLO+OCR, comprising the following steps:

[0015] S1. Drawing preprocessing: The received digitized image files of machining drawings are sequentially processed by grayscale conversion, Gaussian filtering for noise reduction, and Canny edge detection. The drawing frame and drawing area are separated by contour extraction and filtering.

[0016] S2, OBB text block detection: The YOLO-OBB algorithm is used to detect text blocks in drawings. The algorithm divides the input image into an S×S grid and predicts the probability of containing text, text category, and rotation bounding box parameters for each grid.

[0017] S3. Text Correction: Construct a four-class neural network model to perform discrete classification prediction on the rotation angle of the detected text cluster image. The classification labels are 0°, 90°, 180°, and 270°. Based on the prediction results, the text cluster is rotated to the 0° direction using a rotation matrix to complete the correction. After rotation, irrelevant background is cropped out.

[0018] S4, PMI Recognition: The content of the corrected text is extracted using OCR technology. The recognition rate is optimized by combining the OCR engine. PMI information, including tolerances, materials and surface treatment, is extracted. Accurate recognition is achieved by aligning the information of the detection results with the OCR recognition results through a special symbol model.

[0019] S5. Global Information Extraction: Based on the results of the frame region and text recognition, natural language processing technology is used to extract the design global parameters through keyword matching and entity recognition.

[0020] S6. Error Detection and Feedback: Real-time monitoring and analysis of the recognition results during the information extraction process to correct potential recognition errors.

[0021] Preferably, in step S1, the drawing frame includes metadata including title, drawing unit, date and version number, and the drawing area includes part design and annotation information.

[0022] Preferably, the Canny edge detection in step S1 includes gradient calculation, non-maximum suppression, and double threshold connection steps, used to extract the frame edge and the outline of the drawing area.

[0023] Preferably, in step S2YOLO-OBB algorithm, the rotated bounding box parameters are represented as b = (x, y, w, h, θ), where (x, y) are the center coordinates of the bounding box, (w, h) are the width and height, and θ is the rotation angle.

[0024] Preferably, in step S2, the text group encompasses the tilted text in the title bar and the enlarged view area, with a tilt angle ranging from 15° to 75°.

[0025] Preferably, in step S4, the OCR technology first performs noise reduction and binarization preprocessing on the corrected text, and then performs character recognition to extract the content. The OCR engine includes PP-OCRv5 and Tesseract.

[0026] Preferably, the special symbols in step S4 include diameter symbols. Countersunk hole Twenty-seven categories of mechanical symbols, including geometrical tolerance symbols, are detected and identified using a special symbol detection model.

[0027] Preferably, the keywords in step S5 include "Tolerance", "Material", and "Surface Treatment", which are combined with the domain knowledge base to achieve semantic understanding and structured output.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] This invention preprocesses drawings by converting them to grayscale, using Gaussian filtering for noise reduction, and employing Canny edge detection to separate the drawing frame from the drawing area, avoiding text pixel loss caused by direct scaling. It uses the YOLO-OBB algorithm to detect text clusters, recognizing tilted text from 15° to 75° and predicting rotation bounding box parameters. A four-class neural network model is constructed to classify the text cluster rotation angle, correcting it to 0° and cropping the background to solve the 360° text orientation problem. OCR technology is used to extract the corrected text content, and a special symbol model is combined to achieve information alignment and identify PMI information. Based on the frame and text recognition results, natural language processing technology is used to extract global parameters. Finally, the recognition results are monitored and analyzed to correct errors. This invention solves the problems of low efficiency and easy misreading in traditional manual interpretation, as well as text pixel loss, insufficient orientation processing, and poor correlation between symbols and text in existing technologies. It improves the automation and accuracy of information extraction from machining drawings, is applicable to fields such as machining, and can be extended to drawing processing in other industries. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the overall process of the YOLO+OCR-based method for extracting machining information from machining drawings according to the present invention.

[0031] Figure 2 This is a schematic diagram illustrating the alignment of information between the special symbol detection results and the OCR recognition results.

[0032] Figure 3 This is a diagram illustrating PMI information. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Please see Figure 1-3 This invention provides a technical solution: a method for extracting machining information from machining drawings based on YOLO+OCR, comprising the following steps:

[0035] S1. Drawing Preprocessing: The received digitized image files of machining drawings are sequentially processed using grayscale conversion, Gaussian filtering for noise reduction, and Canny edge detection. The drawing frame and drawing area are separated through contour extraction and filtering. The drawing frame contains metadata including the title, drafting unit, date, and version number, while the drawing area contains part design and annotation information. The specific method for separating the drawing frame and drawing area through contour extraction and filtering is as follows:

[0036] S11. Grayscale Conversion: Converts color drawings to grayscale images to simplify subsequent processing.

[0037] I gray (x,y)=0.299R+0.587G+0.114B;

[0038] S12, Gaussian filter noise reduction:

[0039] Applying a Gaussian filter to smooth the image and suppress noise:

[0040] Filtered image: I smooth =I gray ·G;

[0041] S13, Canny Edge Detection: The Canny algorithm is used to detect edges. The steps include gradient calculation, non-maximum suppression, and double threshold connection, which are used to extract the edges of the frame and the outline of the drawing area.

[0042] S14. Contour Extraction and Filtering: Using contour detection algorithms (such as OpenCV's findContours), the boundaries of the drawing frame are identified. Area and shape thresholds are set to filter contours that conform to the characteristics of the drawing frame, thus completing the separation of the drawing frame from the drawing area. The separation of the drawing frame from the drawing area is achieved by filtering contours based on area thresholds (5% to 30%) and rectangularity thresholds (≥0.8).

[0043] By effectively separating the drawing frame from the drawing area during the drawing preprocessing stage, a clean and interference-free input environment is provided for subsequent information extraction, significantly improving recognition accuracy and efficiency. In traditional methods, the complexity of drawings often leads to low efficiency in information extraction. This patent, however, employs techniques such as grayscale conversion, Gaussian filtering, and Canny edge detection to significantly improve the readability and processing efficiency of the drawings. This process lays a solid foundation for subsequent text detection and character recognition, ensuring data integrity and accuracy. Simultaneously, the drawing frame information obtained in the preprocessing stage lays the foundation for global information extraction, guaranteeing data integrity.

[0044] S2, OBB Text Block Detection: The YOLO-OBB algorithm is used to detect text clusters in drawings. It can effectively identify text regions at any angle, including tilted text in title blocks and magnified view areas, with tilt angles ranging from 15° to 75°. YOLO-OBB introduces an Oriented Bounding Box (OBB), which can detect text regions at any rotation angle, adapting to the diverse directions of text in mechanical drawings. The algorithm divides the input image into an S×S grid and predicts the probability of containing text, text category, and rotation bounding box parameters b = (x, y, w, h, θ) for each grid, where (x, y) are the center coordinates of the bounding box, (w, h) are the width and height, and θ is the rotation angle. The training process uses the IOU with the rotated bounding box as the loss function to optimize the detection accuracy.

[0045] The OBB text block detection provides the spatial location and orientation of text clusters, offering accurate input for text correction and OCR recognition. This technology significantly improves the detection capability of slanted text in mechanical drawings, promoting the automation of information extraction. Compared to traditional horizontal bounding boxes (HBB), YOLO-OBB introduces oriented bounding boxes (OBB), greatly enhancing the detection capability of slanted text. The application of this technology not only improves the accuracy of text detection in mechanical drawings but also promotes the automation of information extraction, reducing the need for manual intervention.

[0046] S3. Text Correction: A four-class neural network model is constructed to perform discrete classification prediction of the rotation angle of the detected text cluster image. The classification labels are 0°, 90°, 180°, and 270°. Based on the prediction results, the text cluster is rotated to the 0° direction using a rotation matrix to complete the correction. After rotation, irrelevant background is cropped to remove it. The purpose of the correction process is to rotate the text cluster to the standard direction (0°) to improve the accuracy of subsequent OCR recognition. A deep learning-based four-class model is used to perform discrete classification prediction of the rotation angle of the text cluster. The specific steps are as follows:

[0047] S31. Design of Angle Classification Model:

[0048] A four-class neural network model was constructed, with classification labels Θ = {0°, 90°, 180°, 270°}. The model input consisted of detected text cluster images, and the output was the probability distribution of the four classes. The four-class neural network model adopted a CNN architecture with 3 convolutional layers and 2 fully connected layers. The input was a 224×224 pixel grayscale image of the text cluster, with convolutional kernel sizes of 3×3, 5×5, and 3×3, respectively. The activation function was ReLU, and the output layer used the softmax function to output the probability distribution of the four angle classes: 0°, 90°, 180°, and 270°. The training dataset contained 100,000 mechanical drawing text samples (covering annotations of parts made of metal, plastic, etc.), with a balanced sample size for each angle class. The loss function used was cross-entropy loss, and samples that were easily confused with 180° and 0° were assigned a weight of 1.5 times. After 50 rounds of iterative training, the accuracy on the validation set stabilized at 98.2%.

[0049] S32, Angle Prediction:

[0050] Predict the rotation category of the text cluster using a model:

[0051] Wherein, P(θ|I text The model uses the input text image I. text The predicted probability of belonging to the angle category θ;

[0052] S33, Image Correction:

[0053] According to the prediction angle Rotate the text image to 0°: Wherein, rotation matrix Defined as:

[0054]

[0055] S34. Cutting and Post-processing:

[0056] After rotation, the image is cropped to remove irrelevant background and ensure the clarity and integrity of the corrected text area;

[0057] This step employs a four-class classification model for angle prediction correction. By discretizing the text direction for prediction, it avoids the error accumulation caused by continuous angle estimation in traditional methods, thus improving the robustness and accuracy of rotation angle discrimination. This method ensures the uniformity of the text image's orientation, significantly improving the performance and efficiency of OCR recognition. The corrected text region is clearer, providing higher accuracy for subsequent character recognition and ensuring the effective extraction of complex text information, thus providing a solid guarantee for the accurate extraction of complex text information in mechanical drawings.

[0058] S4, PMI Recognition: The content of the corrected text is extracted using OCR technology. Combined with the OCR engine to optimize the recognition rate, PMI information including tolerances, materials and surface treatment is extracted. Accurate recognition is achieved by aligning the information of the detection results with the OCR recognition results through a special symbol model. Optical character recognition (OCR) technology is a technology that converts characters in an image into an editable text format. It is widely used in various document digitization and information extraction scenarios.

[0059] In mechanical drawings, text blocks typically contain important information such as descriptive text, special symbols, and tolerance representations. OCR technology can extract this textual information for further processing and analysis. The OCR system first preprocesses the input image, including denoising and binarization, to improve recognition accuracy. Then, the system analyzes the characters in the image, identifying the specific content of each character.

[0060] To improve recognition accuracy, multiple OCR engines, such as PP-OCRv5 and Tesseract, were compared and optimized. Through continuous parameter adjustment and model training, a high recognition rate was achieved on different types of mechanical drawings, ensuring that the extracted information is complete and accurate.

[0061] The special symbol detection model is an improvement on YOLOv5, adding three detection heads for small-sized symbols. The anchor box size is designed based on the average pixel size of the symbol (5×5 to 30×30). Association is achieved by calculating the spatial Euclidean distance between the symbol and adjacent text (with a threshold of 5 pixels), for example, the diameter symbol. When the distance to the number '5.5' is less than the threshold, it is automatically combined into...

[0062] like Figure 2 The process shown in this patent involves aligning the information from the special symbol detection results with the OCR recognition results to obtain... Figure 3 The PMI information shown;

[0063] PMI information includes key manufacturing information such as tolerances, materials, and surface treatment. Accurately identifying PMI helps to: ensure quality control in the manufacturing process, support design optimization and process improvement, and provide data support for product lifecycle management.

[0064] S5. Global Information Extraction: Based on the bounding box region and text recognition results, Natural Language Processing (NLP) technology is used to extract the design global parameters through keyword matching and entity recognition. The specific steps are as follows:

[0065] Keyword matching and entity recognition (NER): Tolerance, Material, Surface Treatment.

[0066] By combining domain knowledge bases, semantic understanding and structured output can be achieved;

[0067] Global information extraction provides crucial data support for mechanical design and manufacturing. By extracting tolerance, material, and surface treatment information, engineers can better understand design intent and ensure product consistency and quality throughout the manufacturing process. Keyword matching and entity recognition enable rapid acquisition of critical information, enhancing the intelligence level of information processing. Global information extraction not only helps engineers better understand design intent but also ensures product consistency and quality during manufacturing, providing vital references for subsequent product maintenance and upgrades.

[0068] S6. Error Detection and Feedback: Real-time monitoring and analysis of the recognition results during the information extraction process to correct potential recognition errors, including:

[0069] The system monitors OCR recognition confidence in real time (triggers secondary recognition when it falls below 0.7) and symbol-text association distance (a distance exceeding 10 pixels is considered an association error). For errors with clear rules, such as character inversion caused by incorrect text correction angle, the system automatically triggers correction based on the detection results.

[0070] Correcting angle errors: Re-call the four-class classification model to predict the angle of the text group, or reverse the correct angle based on the OCR pre-recognition results (e.g., when recognizing "upside down" characters, automatically rotate 180° to correct them).

[0071] Incorrect symbol or parameter combination: Invoking a domain knowledge base (e.g.) '4x' and (Based on the matching rules), the system automatically matches similar values ​​within the same area and corrects them based on the context.

[0072] For errors that the system cannot determine, generate a structured error report:

[0073] Report content includes: error location, error type, original identification results, and suggestions for suspected correct results;

[0074] Interactive interface: Highlights error areas with visual tools, allowing for direct manual modification and submission of corrective results;

[0075] Model iteration: The training set is expanded by rotating and scaling the manually corrected data. The four-class classification model, symbol recognition model and rule base are updated every 500 data points to improve the recognition accuracy in subsequent stages.

[0076] By monitoring and analyzing the recognition results in real time through error detection and feedback steps, potential recognition errors can be identified and corrected promptly. This mechanism further improves the reliability of the entire information extraction process and ensures the accuracy of the final output data.

[0077] This invention boasts excellent adaptability, capable of handling the processing needs of various types of mechanical drawings. Whether complex two-dimensional drawings or three-dimensional models, they can all be effectively automated through the technical framework of this patent. This flexibility makes this patent widely applicable in practical applications, meeting the needs of different industries and fields. By automating the extraction of drawing information, it reduces manual intervention, improves production efficiency, and lowers the human error rate, providing strong support for enterprises' digital transformation. With the continuous development of intelligent manufacturing, this invention will bring higher productivity and competitiveness to the industry, greatly promoting the progress of intelligent manufacturing. Through automated information extraction and processing, enterprises can significantly reduce labor costs and improve production efficiency. This process not only reduces errors caused by manual interpretation but also shortens the production cycle, bringing higher economic benefits to enterprises. According to industry statistics, after adopting this patented technical solution, enterprises can improve efficiency in the drawing processing stage by more than 50%, significantly reducing production costs.

[0078] This invention is not only applicable to the field of machining, but can also be extended to drawing information processing in other industries such as construction and electronics. This cross-domain applicability makes this patent have great application potential in different fields, and can support the intelligent transformation of more industries.

[0079] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0080] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for extracting machining information from machining drawings based on YOLO+OCR, characterized in that: Includes the following steps: S1. Drawing preprocessing: The received digitized image files of machining drawings are sequentially processed by grayscale conversion, Gaussian filtering for noise reduction, and Canny edge detection. The drawing frame and drawing area are separated by contour extraction and filtering. S2, OBB text block detection: The YOLO-OBB algorithm is used to detect text blocks in drawings. The algorithm divides the input image into an S×S grid and predicts the probability of containing text, text category, and rotation bounding box parameters for each grid. S3. Text Correction: Construct a four-class neural network model to perform discrete classification prediction on the rotation angle of the detected text cluster image. The classification labels are 0°, 90°, 180°, and 270°. Based on the prediction results, the text cluster is rotated to the 0° direction using a rotation matrix to complete the correction. After rotation, irrelevant background is cropped out. S4, PMI Recognition: The content of the corrected text is extracted using OCR technology. The recognition rate is optimized by combining the OCR engine. PMI information, including tolerances, materials and surface treatment, is extracted. Accurate recognition is achieved by aligning the information of the detection results with the OCR recognition results through a special symbol model. S5. Global Information Extraction: Based on the results of the frame region and text recognition, natural language processing technology is used to extract the design global parameters through keyword matching and entity recognition. S6. Error Detection and Feedback: Real-time monitoring and analysis of the recognition results during the information extraction process to correct potential recognition errors.

2. The method for extracting machining information from machining drawings based on YOLO+OCR according to claim 1, characterized in that: In step S1, the drawing frame contains metadata including the title, drawing unit, date, and version number, while the drawing area contains part design and annotation information.

3. The method for extracting machining information from machining drawings based on YOLO+OCR according to claim 2, characterized in that: The Canny edge detection in step S1 includes gradient calculation, non-maximum suppression, and double threshold connection steps, which are used to extract the edges of the frame and the outline of the drawing area.

4. The method for extracting machining information from machining drawings based on YOLO+OCR according to claim 3, characterized in that: In the S2YOLO-OBB algorithm, the parameters of the rotated bounding box are represented as b = (x, y, w, h, θ), where (x, y) are the coordinates of the bounding box center, (w, h) are the width and height, and θ is the rotation angle.

5. The method for extracting machining information from machining drawings based on YOLO+OCR according to claim 4, characterized in that: In step S2, the text group includes the tilted text in the title bar and the enlarged view area, with the tilt angle ranging from 15° to 75°.

6. The method for extracting machining information from machining drawings based on YOLO+OCR according to claim 5, characterized in that: In step S4, the OCR technology first performs noise reduction and binarization preprocessing on the corrected text, and then performs character recognition to extract the content. The OCR engine includes PP-OCRv5 and Tesseract.

7. The method for extracting machining information from machining drawings based on YOLO+OCR according to claim 6, characterized in that: The special symbols in step S4 include diameter symbols. Countersunk hole Twenty-seven categories of mechanical symbols, including geometrical tolerance symbols, are detected and identified using a special symbol detection model.

8. The method for extracting machining information from machining drawings based on YOLO+OCR according to claim 7, characterized in that: In step S5, the keywords include "Tolerance", "Material", and "Surface Treatment", which are combined with the domain knowledge base to achieve semantic understanding and structured output.

Images