2D drawing feature recognition method, system, equipment and medium

Abstract

The present invention provides a 2D drawing feature recognition method, system, device, and medium, comprising: inputting a two-dimensional drawing and a corresponding three-dimensional model; automatically extracting all relevant feature information from the three-dimensional model; automatically extracting all annotation information from the two-dimensional drawing by constructing a deep learning intelligent algorithm model and an image algorithm; obtaining a mapping relationship between the two-dimensional drawing and the three-dimensional model through projection and mathematical calculation; integrating the feature information, annotation information, and the mapping relationship between the two-dimensional drawing and the three-dimensional model, with the system automatically determining which part of the three-dimensional drawing feature the extracted annotation information corresponds to, and mapping the annotation information recognized from the two-dimensional drawing to a three-dimensional feature list; displaying the annotation information corresponding to the features of the three-dimensional drawing; allowing the user to review the two-dimensional drawing annotation information for accuracy; and outputting the annotated three-dimensional model and feature annotation data details. The present invention can automatically extract and identify annotation feature information from two-dimensional drawings and generate structured tabular information.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile production and processing in the manufacturing industry, and in particular to a 2D drawing feature recognition method, system, equipment and medium. Background Art

[0002] As manufacturing capabilities improve, product complexity and precision are also increasing, and the role of information technology in promoting the development of manufacturing is becoming increasingly apparent. However, a large number of small and medium-sized manufacturing enterprises in China are still in the 2D process design stage based on 2D engineering drawings. 2D drawings (.pdf, .jpg, .png files) contain a large amount of process feature information required for manufacturing, such as dimensions, text annotations, tolerances, etc., which are an important basis for part production and processing. However, 2D drawing recognition mainly relies on experienced process engineers, which cannot be effectively informationized. The application of artificial intelligence technology can realize the intelligent recognition of 2D drawings, automatically extract, transform, and analyze the annotation information of 2D drawings and apply it to automated flexible production lines. This not only solves the problem of scarce process technicians, but also removes obstacles to the digitalization process of domestic small and medium-sized manufacturing enterprises.

[0003] At present, the 3D drawing feature recognition in the industry is relatively mature. NX can be used to add custom development components to realize the feature recognition of three types of drawings: holes, faces, and slots in specific scenarios. Normal AutoCAD and NX do not contain some specific shape feature combinations that require secondary development. The ability of secondary development in the whole process has a greater impact on the effect and efficiency. There are also cases of automatic recognition of two-dimensional drawings in the market, but the recognition method mainly relies on logical rules, resulting in only the recognition of fixed drawing templates. It cannot effectively recognize newly added template types or drawings with undefined annotation symbols. It is also difficult to effectively distinguish the corresponding part space information of two-dimensional drawings and those with shadows and special backgrounds. Most recognition systems do not have the function of human interaction, and the recognition model itself does not have the function of intelligent feedback self-learning based on human interaction, resulting in low stability and poor robustness of two-dimensional drawing recognition.

[0004] Utility model patent publication number CN210598815U discloses an automatic drilling machine based on drawing recognition and intelligent positioning. The machine comprises a rectangular platform supported by crawler-type longitudinal synchronous travel mechanisms on both sides. Crawler-type transverse synchronous travel mechanisms with raised and lowered positions are mounted on the other two sides of the platform. In the raised position, the transverse synchronous travel mechanism is higher than the longitudinal synchronous travel mechanism, while in the lowered position, the transverse synchronous travel mechanism is lower than the longitudinal synchronous travel mechanism. The platform is equipped with outriggers with raised and lowered positions. In the raised position, the bottom of the outrigger is higher than the bottom of the longitudinal synchronous travel mechanism, while in the lowered position, the bottom of the outrigger is lower than the bottom of the transverse synchronous travel mechanism. A raiseable drilling machine is mounted in the center of the platform. This utility model uses a combination of two-dimensional drawings and three-dimensional models for positioning to assist in the recognition of two-dimensional drawings. However, it is limited to cases where the drawing annotations are digital and cannot guarantee recognition rates for special symbols or undefined annotation types. The model lacks self-learning capabilities, resulting in poor stability and a limited range of applications. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the present invention provides a 2D drawing feature recognition method, system, device and medium.

[0006] According to the present invention, a 2D drawing feature recognition method, system, device and medium are provided, and the solution is as follows:

[0007] In a first aspect, a method for identifying features of a 2D drawing is provided, the method comprising:

[0008] Step S1: input two-dimensional drawings and corresponding three-dimensional models;

[0009] Step S2: Automatically extract all annotated feature information from the 3D model;

[0010] Step S3: Automatically extract all annotation information from the two-dimensional drawing by building a proprietary AI deep learning algorithm model and image algorithm. The annotation information includes related information such as annotation data, annotation arrows, and annotation lines.

[0011] Step S4: Obtaining the mapping relationship between the two-dimensional drawing and the three-dimensional model through projection and mathematical calculation;

[0012] Step S5: Integrate the feature information, annotation information, and the mapping relationship between the 2D drawing and the 3D model. The system automatically determines which feature of the 3D drawing the extracted annotation information corresponds to, and maps the annotation information identified on the 2D drawing to the 3D feature list.

[0013] Step S6: displaying the annotation information of the corresponding features of the three-dimensional drawing;

[0014] Step S7: The user verifies whether the two-dimensional drawing annotation information is correct;

[0015] Step S8: Outputting the annotated 3D model and feature annotation data details;

[0016] The step S3 specifically includes:

[0017] Step S3.1: Collect multiple automotive manufacturing process drawings and divide them into a training set and a test set in an 8:2 ratio.

[0018] Step S3.2: Based on the Paddle framework, a deep learning algorithm for identifying annotation information in two-dimensional drawings is constructed. The deep learning algorithm extracts dimensions, text annotations, form and position tolerances, surface roughness, and their locations from two-dimensional drawings.

[0019] Step S3.3: Based on the two-dimensional image information and annotation symbol information collected in step S3.1, train the neural network model, using accuracy as the main evaluation indicator and recall rate as an auxiliary tuning indicator;

[0020] Step S3.4: Based on an image algorithm, all arrows and annotation lines in the two-dimensional drawing image and their positions are identified based on standard two-dimensional drawing arrow shape and annotation line shape matching;

[0021] The step S4 specifically includes:

[0022] Step S4.1: For each face of the two-dimensional drawing, extract the circumscribed rectangle of the two-dimensional drawing and crop the face containing only the drawing information for matching;

[0023] Step S4.2: Using an image algorithm, identify the annotation arrows and annotation lines on the two-dimensional drawing and match them with the annotation arrows;

[0024] Step S4.3: For the three-dimensional drawing, a series of features are read from the .STEP file through feature extraction. The positions of the features are clustered to predict the positions of potential two-dimensional sections. The potential two-dimensional sections are matched with the drawing through projection and hashing algorithms. The ends of the identified annotation lines are detected in turn to see if they are close to the features and corresponding matches are made.

[0025] Preferably, step S7 includes:

[0026] Step S7.1: If the user verifies that the 2D drawing information corresponds to the 3D feature mapping accurately and the 2D drawing annotation information is correct, proceed to the next step;

[0027] Step S7.2: If the user verifies that the annotation information extracted from the two-dimensional drawing is inaccurate, or the corresponding three-dimensional feature mapping is incorrect, the user is guided to manually correct the incorrect annotation or matching information through the interactive interface; at this time, the user's operation information will be saved and added to the deep learning model of the reinforcement learning that has been constructed as a data source for positive cases.

[0028] Preferably, correcting the erroneous annotation or matching information in step S7.2 specifically includes: identifying the two-dimensional drawing with its own image recognition algorithm model and the image algorithm identifying the annotation information results, adding missed features, and deleting erroneously matched feature connections.

[0029] Preferably, the step S7.2 also includes: by reasoning about the matched features, the program can intelligently remind the user on the interactive interface which features are still not matched; at the same time, the manually corrected marking information will be transmitted to the algorithm model corresponding to the two-dimensional to three-dimensional information as learning data for the self-learning algorithm model.

[0030] In a second aspect, a 2D drawing feature recognition system is provided, the system comprising:

[0031] Module M1: Input 2D drawings and corresponding 3D models;

[0032] Module M2: Automatically extract all relevant feature information from the 3D model;

[0033] Module M3: Automatically extract all annotation information from 2D drawings by building its own AI deep learning algorithm model and image algorithm. The annotation information includes annotation data, annotation arrows, annotation lines and other related information.

[0034] Module M4: Obtain the mapping relationship between 2D drawings and 3D models through projection and mathematical calculations;

[0035] Module M5: Integrate feature information, annotation information, 2D drawings, and 3D model mappings. The system automatically determines which feature of the 3D drawing the extracted annotation information corresponds to, and maps the annotation information identified on the 2D drawing to the 3D feature list.

[0036] Module M6: Display the annotation information of the corresponding features of the 3D drawing;

[0037] Module M7: The user reviews whether the 2D drawing annotation information is correct;

[0038] Module M8: Output annotated 3D model and feature annotation data details;

[0039] The module M3 specifically includes:

[0040] Module M3.1: Collect multiple automotive processing drawings and divide them into training and test sets in an 8:2 ratio.

[0041] Module M3.2: Based on the Paddle framework, a deep learning algorithm is constructed for the recognition of annotation information in 2D drawings. The deep learning algorithm extracts dimensions, text annotations, form and position tolerances, surface roughness, and their locations from 2D drawings.

[0042] Module M3.3: Based on the 2D image information and annotation symbol information collected in Module M3.1, train the neural network model, using accuracy as the primary evaluation metric and recall as an auxiliary tuning metric;

[0043] Module M3.4: Image-based algorithms identify all arrows and annotation lines and their positions in 2D drawings based on standard 2D drawing arrow shape and annotation line shape matching;

[0044] The module M4 specifically includes:

[0045] Module M4.1: For each face of the 2D drawing, extract the circumscribed rectangle of the 2D drawing and crop the face containing only the drawing information for matching;

[0046] Module M4.2: Identify annotation arrows and annotation lines on 2D drawings through image algorithms and match them with annotation arrows;

[0047] Module M4.3: For 3D drawings, a series of features are read from the .STEP file through feature extraction. The positions of the features are clustered to predict the positions of potential 2D sections. The potential 2D sections are matched with the drawings through projection and hashing algorithms. The ends of the identified annotation lines are sequentially detected to see if they are adjacent to the features and the corresponding matches are made.

[0048] Preferably, the module M7 includes:

[0049] Module M7.1: If the user verifies that the 2D drawing information corresponds to the 3D feature mapping accurately and the 2D drawing annotation information is correct, proceed to the next step;

[0050] Module M7.2: If the user verifies that the annotation information extracted from the 2D drawing is inaccurate, or the corresponding 3D feature mapping is incorrect, the user is guided through the interactive interface to manually correct the incorrect annotation or matching information. At this time, the user's operation information will be saved and added to the deep learning model of the established reinforcement learning as a data source for positive cases.

[0051] Preferably, the correction of erroneous annotation or matching information in the module M7.2 specifically includes: recognition of the two-dimensional drawing's own image recognition algorithm model and the image algorithm recognition annotation information results, adding missed features, and deleting erroneously matched feature connections.

[0052] Preferably, the module M7.2 also includes: by reasoning about the matched features, the program can intelligently remind the user in the interactive interface which features are still not matched; at the same time, the manually corrected marking information will be transmitted to the algorithm model corresponding to the two-dimensional to three-dimensional information as learning data for the self-learning algorithm model.

[0053] According to a third aspect, a device is provided, comprising:

[0054] one or more processors;

[0055] a storage device for storing one or more programs,

[0056] When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the steps in the method.

[0057] In a fourth aspect, a computer-readable storage medium storing a computer program is provided, wherein the computer program implements the steps in the method when executed by a processor.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. This invention uses computer vision technology and graphics processing algorithms to achieve accurate recognition of two-dimensional part drawings by combining two-dimensional drawings with three-dimensional models. This solves the human resource limitations and cumbersome operations of drawing information recognition in real-world scenarios. It also solves the recognition bias problem caused by the lack of spatial information in simple two-dimensional drawings, thus reducing human resource constraints.

[0060] 2. At the same time, the present invention constructs a model with self-learning ability, which can continuously improve the algorithm's feature recognition stability and accuracy by manually adjusting information during use, thereby achieving the effect of reducing costs and increasing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0062] Figure 1 To identify the flow chart;

[0063] Figure 2 Three-dimensional models at different angles;

[0064] Figure 3 A list of two-dimensional drawings and the corresponding results of extracted annotation information. DETAILED DESCRIPTION

[0065] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0066] The embodiment of the present invention provides a 2D drawing feature recognition method, which automatically extracts and identifies annotation feature information (including dimensions, text annotations, position tolerances, surface roughness, symbols, arrows, etc.) from 2D drawings by combining 2D parts with 3D models to generate structured table information. Figure 1 As shown, the method specifically includes:

[0067] Step 1: Input the 2D drawings and the corresponding 3D models, refer to Figure 2 As shown, the 3D model displays all model-related feature information (holes, slots, faces, etc.), including their specific locations and processing information (such as the order of holes, the characteristics of each order, etc.). The 2D drawing displays annotation information, and the main annotations include information such as dimensions, text annotations, position tolerances, surface roughness, and symbols.

[0068] Step 2: The system automatically extracts all relevant feature information (holes, slots, faces, etc.) from the 3D model, including their specific locations and processing information (such as the order of holes, features of each order, etc.).

[0069] The basic principle of 3D model feature extraction and recognition is to extract geometric and manufacturing information from the part entity, then compare and instantiate the feature class based on the extracted and recognized information. The feature recognition methods involved in this example are mainly boundary-based feature recognition methods, which can be divided into the following three types. Different methods are currently used according to the matching of different feature types:

[0070] (1) Rule-based feature recognition: Define rules to determine feature boundaries, and then perform feature recognition based on these rules. For example, define the feature rules for a hole: start with the opening face, ensure the hole axis is coplanar, connect all the faces of the hole sequentially, and finally determine whether it ends at a valid hole surface. This method identifies part features by comparing the relationship between model boundary facts and feature rules.

[0071] (2) Graph feature recognition. This type of method uses faces as nodes and the connection relationship between faces as arcs to represent the graph model of the part boundary. The difference between this face adjacency graph and the general face edge graph is that the convexity and concavity of the face are represented by the attributes of the arc, so that the graph information of the boundary pattern is more complete. The feature search strategy of the graph-based feature recognition method is subgraph matching, that is, by comparing and matching the legend in the part boundary library with the features of the graph in the actual model. Therefore, how to improve the efficiency of the graph search algorithm and replace the matching of all subgraphs with classification matching is one of the key issues of the graph-based feature recognition method.

[0072] (3) Feature Recognition Based on Traces: Any feature instance will inevitably leave a feature trace in the part CAD model. After the features intersect, the original feature boundary disappears, replaced by new feature constraints. Therefore, based on these change traces, part features can be gradually identified. Given the current maturity of solid boundary representation technology, specific feature traces can be geometric topology information or other CAD information of design features. Currently, feature traces are mainly determined based on topology information and design features.

[0073] Step 3: Automatically extract all annotation information from the two-dimensional drawing through self-training neural network algorithm and image algorithm. The annotation information includes annotation data, annotation arrows and annotation lines and their position information.

[0074] Step S3.1: Collect 600 automotive processing drawings and divide them into a training set and a test set with an 8:2 ratio. According to the "National Standard for Mechanical Drawing", 128 common mechanical drawing symbols and their meanings are summarized. These symbols cover most of the annotation symbols in two-dimensional drawings.

[0075] Step S3.2: Based on the paddle framework, a deep learning algorithm is constructed for the recognition of two-dimensional drawing annotation information. The recognition algorithm can extract the dimensions, text annotations, type and position tolerances, surface roughness and their position information in the two-dimensional drawing images.

[0076] The specific technical implementation is as follows:

[0077] The essence of 2D drawing inspection is to identify annotated numbers, text, and symbols, and their corresponding locations. Technical implementation presents several challenges, including the diverse nature of text in natural scenes, with varying sizes, orientations, lengths, shapes, and languages. Furthermore, overlapping text or high text density can affect the final text detection results. Commonly used text detection methods include regression-based and segmentation-based methods. Within the project's deep learning framework, the currently selected technical solution is the segmentation-based DBNet3 method.

[0078] The following figure illustrates the working principle of DBNet. To address the time-consuming post-processing issues associated with segmentation-based methods, which require thresholding for binarization, DBNet proposes a learnable threshold method and cleverly designs a binarization function that approximates a step function. This allows the segmentation network to learn the threshold for text segmentation end-to-end during training. Automatically adjusting the threshold not only improves accuracy but also simplifies post-processing, enhancing text detection performance.

[0079] By building the DBNet model described above, we can identify the bounding box positions of annotation information on 2D drawings. The next step is to identify the specific text and symbol content within the bounding boxes obtained from the annotation information. We use a CTC-based approach. Due to the specific nature of text recognition, the input data contains a large amount of contextual information. The convolutional kernel characteristics of convolutional neural networks (CNNs) make them more focused on local information and lack the ability to model long-term dependencies. Therefore, using CNNs alone makes it difficult to discover contextual connections between text.

[0080] To address this issue, we first used a convolutional network (CRNN)4 to extract image features and simultaneously introduced a bidirectional LSTM (Long Short-Term Memory) to enhance context modeling. Finally, the output feature sequence was fed into a CTC module, which inferred the connectivity between characters and directly decoded the sequence. This architecture was proven effective and performed well in this text recognition task.

[0081] CTC (Connectionist Temporal Classification) is a good way to let the network automatically learn alignment, which is very suitable for speech recognition and handwriting recognition. For the input sequence X = [x1, x2, ..., xT], we want to get the corresponding output sequence Y = [y1, y2, ..., yU]. We need CTC to effectively complete two tasks, one is the calculation of the loss function, and the other is forward reasoning: LossFunction: For a given input, a model will be trained to maximize the probability of the correct output sequence; that is, we need to calculate the conditional probability p(Y|X), because for backpropagation, the function p(Y|X) is differentiable. Inference: Now that the model has been trained, for a given X, we need to use it to infer Y, which means finding: Y* = argmaxY(Y|X). In this way, CTC can get an approximate solution without spending too much effort.

[0082] Step S3.3: Based on the two-dimensional image information and annotation symbol information collected in step S3.1, train the above neural network model, using accuracy as the main evaluation indicator and recall rate as an auxiliary tuning indicator. Currently, the accuracy of this algorithm model has reached more than 90%.

[0083] Step S3.4: Based on an image algorithm, all arrows and annotation lines in the two-dimensional drawing image and their positions are identified based on standard two-dimensional drawing arrow shape and annotation line shape matching.

[0084] Step 4: Obtain the mapping relationship between the 2D drawing and the 3D model through projection and mathematical calculation.

[0085] The steps are as follows:

[0086] 4.1: For each face of the 2D drawing, we use an algorithm to extract the circumscribed rectangle of the drawing and crop the face containing only the drawing information for matching.

[0087] 4.2: In addition, image algorithms are used to identify arrows and annotation lines on two-dimensional drawings and match them with arrows. Arrow recognition mainly uses normalized correlation (normalized cross-correlation, hence NCC) and normalized squared difference matching algorithms. The template area is rotated and slid on the input image. The similarity with the input image is calculated using a cross-correlation matching algorithm to obtain the position of the arrow in the image. The distance between the arrow and the position and content of the annotation information of the own image recognition algorithm model are matched to determine the correspondence between the annotation of the own image recognition algorithm model and the information of the annotated arrow. The vertical position information of the arrow tip is extracted, and an image method is used for iteration to find the extension range of the annotation line in the drawing and select the position. The following is an explanation of the normalized correlation algorithm algorithm.

[0088] The normalized square difference matching calculation formula is as follows. This method normalizes the square difference method so that the input result is scaled between 0 and 1. When the template and the sliding window completely match, the calculated value is 0. The lower the match between the two, the larger the calculated value. T(x′, y′) represents the template image, I(x, y) represents the original image, the width of the cut template image is w and the height is h, and R represents the matching result.

[0089]

[0090] The specific arrow image matching process is as follows:

[0091] 1. Select a sliding window of the same size as the template in the image to be matched.

[0092] 2. Compare the grayscale value of each pixel in the sliding window with the corresponding pixel in the template, and calculate the similarity between the template and the sliding window.

[0093] 3. Slide the sliding window from the upper left corner to the right, slide it down one line after sliding to the rightmost point, and start sliding again from the leftmost point. Record the similarity between the template and the sliding window calculated after each movement.

[0094] 4. Compare the similarities of all positions and select the sliding window with the largest similarity as the candidate matching result.

[0095] As the name implies, NCC is used to normalize the correlation between the targets to be matched. It compares the original pixel with the target pixel by constructing a 3*3 neighborhood matching window at the position of the pixel to be matched p(px,py).

[0096] p'(px+d,py) also constructs a neighborhood matching window to establish an objective function to measure the correlation of the matching window. The premise of constructing the correlation window is that the two frames have been corrected to a horizontal position, that is, the optical centers are on the same horizontal line, and the epipolar lines are horizontal. Otherwise, the matching process can only be completed in the direction of the tilted epipolar line, which will consume more computing resources. The correlation degree is defined by the following formula:

[0097]

[0098] The value of NCC(p,d) will be between -1 and 1. Wp is the matching window mentioned above, I1(x,y) is the pixel value of the original image, and I1(px,py) is the mean value of the pixels within the original window. (x+d,y) is the pixel value of the corresponding point in the target image offset by d dd in the x-x direction. I2(px+d,py) is the mean value of the pixels in the matching window of the target image. If NCC = -1, NCC = -1, or NCC = -1, the two matching windows are completely uncorrelated. Conversely, if NCC = 1, NCC = 1, or NCC = 1, the two matching windows are highly correlated.

[0099] The specific arrow image matching process is as follows:

[0100] 1. Image acquisition: Capture images through a calibrated binocular camera. Of course, you can also use two monocular cameras to combine into a binocular camera.

[0101] 2. Epipolar Correction: The purpose of this correction is to align the epipolar lines of two image frames, or to align the optical centers of two image frames on the same horizontal line. This correction facilitates subsequent NCC operations.

[0102] 3. The distortion information in the internal parameters obtained by calibration can be used to remove the distortion of the image.

[0103] 4. After correction by the correction function, the camera's correction transformation R and the new projection matrix P are obtained. The next step is to dedistort the left and right views and obtain the remapping matrix.

[0104] 5. Feature matching: This is the step where we use NCC to do matching. The matching method is as described above. The best match is the one with the highest correlation in the right view and the pixel to be tested in the left view on the same horizontal line. After the matching is completed, we need to record its disparity d, that is, the difference between the horizontal direction xl of the pixel to be tested and the horizontal direction xr of the matching pixel d = xr -xl, finally we can get a disparity map D DD with the same size as the original image.

[0105] 6. Depth recovery: Through the disparity map D DD obtained by the above matching results, we can simply use similar triangles to infer the depth map with the left view as the reference system.

[0106] 4.3: For 3D drawings, using hole features as an example, we use our hole extraction program to extract them. Feature extraction reads a series of hole features from the .STEP file, including their shape, specific location, etc. By clustering the locations of these features, we can predict the location of potential 2D sections: using projection and hashing algorithms, we can match the potential 2D sections with the drawings. After this matching step is completed, we can narrow the range of holes to be matched to those related to this section. Next, we simply need to check whether the end of the annotation line extracted in step 4.2 is close to this hole to make the corresponding match.

[0107] Step 5: Integrate the feature information, annotation information, 2D drawings, and 3D model mapping relationships. The system automatically determines which part of the 3D drawing feature the extracted annotation information corresponds to, and maps the annotation information identified in the 2D drawing to the 3D feature list.

[0108] Step 6: Display the annotation information of the corresponding features of the 3D drawing.

[0109] Step 7: The user reviews whether the 2D drawing annotation information is correct.

[0110] Specifically, step S7 includes:

[0111] Step S7.1: If the user verifies that the 2D drawing information corresponds to the 3D feature mapping accurately and the 2D drawing annotation information is correct, proceed to the next step;

[0112] Step S7.2: If the user verifies that the extracted annotation information from the 2D drawing is inaccurate, or that the corresponding 3D feature mapping is incorrect, the user is guided through the interactive interface to manually correct the incorrect annotation or matching information. Specifically, the user can correct the annotation information recognized by the 2D drawing's own image recognition algorithm model, add missed features, and delete incorrectly matched feature connections. At this time, the user's operation information is saved and added as a data source for positive cases to the established reinforcement learning deep learning model. The deep learning model will continuously optimize and improve the model's robustness based on the user's correction information, making the entire 2D drawing annotation information extraction results more accurate and realizing intelligent self-learning capabilities.

[0113] By reasoning about matched features, the program can also intelligently alert users on this interface to any unmatched features. For example, a hole has three levels, but only the annotations for the first two levels match. In this case, a notification indicating the third level is missing will be displayed. Simultaneously, the manually corrected annotations are fed into the algorithm model that converts 2D to 3D information, serving as learning data for the self-learning algorithm.

[0114] Step 8: Output the annotated 3D model and feature annotation data details.

[0115] The present invention also provides a 2D drawing feature recognition system, which specifically includes:

[0116] Module M1: Input 2D drawings and corresponding 3D models.

[0117] Module M2: The system automatically extracts all relevant feature information (holes, slots, surfaces, etc.) from the 3D model, including their specific locations and processing information (such as the order of holes, features of each order, etc.).

[0118] Module M3: The system automatically extracts all annotation information from two-dimensional drawings through its own image recognition algorithm model technology, mainly including information such as dimensions, text annotations, type and position tolerances, surface roughness and symbols, etc. Through image algorithms, it recognizes arrows and annotation lines on two-dimensional drawings.

[0119] Module M4: Obtain the mapping relationship between two-dimensional drawings and three-dimensional models through projection and mathematical calculations.

[0120] Module M5: Integrate the mapping relationship between feature information, annotation information, 2D drawings and 3D models. The system automatically determines which part of the 3D drawing feature the extracted annotation information corresponds to, and maps the annotation information identified in the 2D drawing to the 3D feature list.

[0121] Module M6: Displays the annotation information of the corresponding features of the three-dimensional drawing.

[0122] Module M7: The user reviews whether the annotation information on the 2D drawing is correct.

[0123] Specifically, module M7 includes:

[0124] Module M7.1: If the user verifies that the 2D drawing information corresponds to the 3D feature mapping accurately and the 2D drawing annotation information is correct, proceed to the next step;

[0125] Module M7.2: If the user verifies that the annotation information extracted from the 2D drawing is inaccurate, or the corresponding 3D feature mapping is incorrect, the interactive interface will guide the user to manually correct the incorrect annotation or matching information. Specifically, the user can correct the annotation information recognition results of the 2D drawing's own image recognition algorithm model, add missed features, and delete incorrectly matched feature connections.

[0126] By reasoning about matched features, the program can also intelligently alert users on this interface to any unmatched features. For example, a hole has three levels, but only the annotations for the first two levels match. In this case, a notification indicating the third level is missing will be displayed. Simultaneously, the manually corrected annotations are fed into the algorithm model that converts 2D to 3D information, serving as learning data for the self-learning algorithm.

[0127] Module M8: Output annotated 3D model and feature annotation data details.

[0128] The embodiments of the present invention provide a 2D drawing feature recognition method, system, device and medium. Through computer vision technology and graphics processing algorithms, a combination of two-dimensional drawings and three-dimensional models is adopted to realize accurate recognition of two-dimensional part drawings. This solves the human resource limitations and tedious operations of drawing information recognition in real scenarios, and solves the recognition deviation problem caused by the actual spatial information of simple two-dimensional drawings, thereby reducing human resource limitations. At the same time, a model with self-learning ability is constructed, which can continuously improve the algorithm's feature recognition stability and accuracy by manually adjusting information during use, thereby achieving the effect of reducing costs and increasing efficiency.

[0129] Throughout the entire process, users do not need to worry about how specific holes and features are processed. The main manpower spent is on correcting the recognition results. The following is the ratio of correction time to annotation time.

[0130] The original process complexity is O(N×M) → If there are N features, each feature requires M time to mark and process.

[0131] The current process complexity is: O(N0×M0), N0<N, m0<<M→ In the inspection process, only a portion of the features require manpower to correct, and the workload is much reduced compared to before; and what needs to be corrected are only some quick intermediate results, without the need to mark them from scratch, which greatly reduces the time consumption.

[0132] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0133] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A 2D drawing feature recognition method, characterized in that: include: Step S1: input two-dimensional drawings and corresponding three-dimensional models; Step S2: Automatically extract all annotated feature information from the 3D model; Step S3: Automatically extract all annotation information from the two-dimensional drawing by building a proprietary AI deep learning algorithm model and image algorithm. The annotation information includes related information such as annotation data, annotation arrows, and annotation lines. Step S4: Obtaining the mapping relationship between the two-dimensional drawing and the three-dimensional model through projection and mathematical calculation; Step S5: Integrate the feature information, annotation information, and the mapping relationship between the 2D drawing and the 3D model. The system automatically determines which feature of the 3D drawing the extracted annotation information corresponds to, and maps the annotation information identified on the 2D drawing to the 3D feature list. Step S6: displaying the annotation information of the corresponding features of the three-dimensional drawing; Step S7: The user verifies whether the two-dimensional drawing annotation information is correct; Step S8: Outputting the annotated 3D model and feature annotation data details; The step S3 specifically includes: Step S3.1: Collect multiple automotive manufacturing process drawings and divide them into a training set and a test set in an 8:2 ratio. Step S3.2: Based on the Paddle framework, a deep learning algorithm for identifying annotation information in two-dimensional drawings is constructed. The deep learning algorithm extracts dimensions, text annotations, form and position tolerances, surface roughness, and their locations from two-dimensional drawings. Step S3.3: Based on the two-dimensional image information and annotation symbol information collected in step S3.1, train the neural network model, using accuracy as the main evaluation indicator and recall rate as an auxiliary tuning indicator; Step S3.4: Based on an image algorithm, all arrows and annotation lines in the two-dimensional drawing image and their positions are identified based on standard two-dimensional drawing arrow shape and annotation line shape matching; The step S4 specifically includes: Step S4.1: For each face of the two-dimensional drawing, extract the circumscribed rectangle of the two-dimensional drawing and crop the face containing only the drawing information for matching; Step S4.2: Using an image algorithm, identify the annotation arrows and annotation lines on the two-dimensional drawing and match them with the annotation arrows; Step S4.3: For three-dimensional drawings, a series of features are read from the .STEP file through feature extraction. The positions of the features are clustered to predict the positions of potential two-dimensional sections. The potential two-dimensional sections are matched with the drawings through projection and hashing algorithms. The ends of the identified annotation lines are detected in turn to see if they are close to the features, and corresponding matches are made. The annotation features corresponding to the annotation information can be determined based on the specified directions of the annotation lines and arrows.

2. The 2D drawing feature recognition method according to claim 1, characterized in that: The step S7 comprises: Step S7.1: If the user verifies that the 2D drawing information corresponds to the 3D feature mapping accurately and the 2D drawing annotation information is correct, proceed to the next step; Step S7.2: If the user verifies that the annotation information extracted from the two-dimensional drawing is inaccurate, or the corresponding three-dimensional feature mapping is incorrect, the user is guided to manually correct the incorrect annotation or matching information through the interactive interface; at this time, the user's operation information will be saved and added to the deep learning model of the reinforcement learning that has been constructed as a data source for positive cases.

3. The 2D drawing feature recognition method according to claim 2, characterized in that: Correcting the erroneous annotation or matching information in step S7.2 specifically includes: using the two-dimensional drawing's own image recognition algorithm model and the image algorithm to recognize the annotation information results, adding features that were missed, and deleting erroneously matched feature connections.

4. The 2D drawing feature recognition method according to claim 2, characterized in that: The step S7.2 also includes: by reasoning about the matched features, the program can intelligently remind the user on the interactive interface which features are still not matched; at the same time, the manually corrected marking information will be transmitted to the algorithm model corresponding to the two-dimensional to three-dimensional information as learning data for the self-learning algorithm model.

5. A 2D drawing feature recognition system, characterized in that: include: Module M1: Input 2D drawings and corresponding 3D models; Module M2: Automatically extract all relevant feature information from the 3D model; Module M3: Automatically extract all annotation information from 2D drawings by building its own AI deep learning algorithm model and image algorithm. The annotation information includes annotation data, annotation arrows, annotation lines and other related information. Module M4: Obtain the mapping relationship between 2D drawings and 3D models through projection and mathematical calculations; Module M5: Integrate feature information, annotation information, 2D drawings, and 3D model mappings. The system automatically determines which feature of the 3D drawing the extracted annotation information corresponds to, and maps the annotation information identified on the 2D drawing to the 3D feature list. Module M6: Display the annotation information of the corresponding features of the 3D drawing; Module M7: The user reviews whether the 2D drawing annotation information is correct; Module M8: Output annotated 3D model and feature annotation data details; The module M3 specifically includes: Module M3.1: Collect multiple automotive processing drawings and divide them into training and test sets in an 8:2 ratio. Module M3.2: Based on the Paddle framework, a deep learning algorithm is constructed for the recognition of annotation information in 2D drawings. The deep learning algorithm extracts dimensions, text annotations, form and position tolerances, surface roughness, and their locations from 2D drawings. Module M3.3: Based on the 2D image information and annotation symbol information collected in Module M3.1, train the neural network model, using accuracy as the primary evaluation metric and recall as an auxiliary tuning metric; Module M3.4: Image-based algorithms identify all arrows and annotation lines and their positions in 2D drawings based on standard 2D drawing arrow shape and annotation line shape matching; The module M4 specifically includes: Module M4.1: For each face of the 2D drawing, extract the circumscribed rectangle of the 2D drawing and crop the face containing only the drawing information for matching; Module M4.2: Identify annotation arrows and annotation lines on 2D drawings through image algorithms and match them with annotation arrows; Module M4.3: For 3D drawings, a series of features are read from the .STEP file through feature extraction. The positions of the features are clustered to predict the positions of potential 2D sections. The potential 2D sections are matched with the drawings through projection and hashing algorithms. The ends of the identified annotation lines are sequentially detected to see if they are adjacent to the features and the corresponding matches are made.

6. The 2D drawing feature recognition system according to claim 5, characterized in that: The module M7 includes: Module M7.1: If the user verifies that the 2D drawing information corresponds to the 3D feature mapping accurately and the 2D drawing annotation information is correct, proceed to the next step; Module M7.2: If the user verifies that the annotation information extracted from the 2D drawing is inaccurate, or the corresponding 3D feature mapping is incorrect, the user is guided through the interactive interface to manually correct the incorrect annotation or matching information. At this time, the user's operation information will be saved and added to the deep learning model of the established reinforcement learning as a data source for positive cases.

7. The 2D drawing feature recognition system according to claim 6, characterized in that: Correcting erroneous annotation or matching information in the module M7.2 specifically includes: using the image recognition algorithm model to recognize the two-dimensional drawing and the image algorithm to recognize the annotation information results, adding features that were missed, and deleting erroneously matched feature connections.

8. The 2D drawing feature recognition system according to claim 6, characterized in that: The module M7.2 also includes: by reasoning about the matched features, the program can intelligently remind the user on the interactive interface which features are still not matched; at the same time, the manually corrected marking information will be transmitted to the algorithm model corresponding to the two-dimensional to three-dimensional information as learning data for the self-learning algorithm model.

9. A device, characterized in that The device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the method according to any one of claims 1 to 4.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images