Methods, systems, devices, and media for matching features of a drawing to numbers

Abstract

The application provides a kind of drawing feature and digital matching method, system, equipment and medium, comprising: obtaining three-dimensional model and automatically extracting all relevant feature information;Obtain two-dimensional drawing and automatically extract all label information;Overall projection is carried out to three-dimensional model, and the model transformation projection image and projection information corresponding to three-dimensional model are generated;The model transformation projection image and two-dimensional drawing are carried out hash calculation, and the section projection picture position of the three-dimensional model corresponding to two-dimensional drawing is found;According to feature information, label information, two-dimensional drawing and step S3, determine three-dimensional model projection image and projection information, determine which part of feature of three-dimensional model corresponding to label information, and correspond label information to three-dimensional feature list;Output labeled three-dimensional model and feature label data details.The application can improve the efficiency and accuracy of two-dimensional drawing information recognition in production and processing, solve the problem of feature label information that two-dimensional drawing cannot be automatically recognized in automobile production.

Description

Technical Field

[0001] This invention relates to the field of automotive manufacturing engineering technology, specifically to a method, system, device, and medium for matching drawing features with numbers. Background Technology

[0002] In automotive manufacturing, annotated 2D drawings are common processing guidelines and standards. While 2D annotated drawings and 3D model files are readily available, feature recognition combining 2D drawings and 3D models primarily relies on experienced process engineers, but these engineers are scarce, hindering company production. Furthermore, there are currently no mature tools on the market for intelligent recognition of 2D drawing annotations, severely limiting the pace of digital transformation in the manufacturing industry. Given the pressure of data-driven transformation in manufacturing, the recognition of annotations in 2D drawings combined with 3D models is the first step in production; therefore, achieving automated and intelligent recognition of 2D drawings and 3D models is urgently needed.

[0003] While there are simple precedents for identifying 2D drawings and 3D model data, the main drawbacks of this method are that the matched 3D models have simple and fixed structures, limited scenarios, and relatively fixed 2D drawings with no complex overlaps or shadows. Most of the processing relies on computer graphics rather than image recognition technology, making it difficult to handle diverse drawing features and information. Currently, a combination of AI visual recognition technology and image algorithms can address these issues.

[0004] Utility model patent CN209096370U discloses a drawing table for easy positioning and calibration of a robotic arm. The table includes a main body with a drawing slot and a robotic arm mounting slot, both located on the same side. The drawing slot is used for positioning the drawing paper, and the robotic arm mounting slot is used for mounting the robotic arm. A pen positioning block is also provided on the table for correcting the stopping height of the robotic arm. This utility model, corresponding to drawing paper and a three-dimensional robotic arm positioning and mounting slot, is limited to robotic arms with fixed drawing paper patterns and structures. It struggles to support positioning for other types of drawing paper. Feature and data matching relies heavily on rule-based implementation, resulting in poor flexibility and limited applicability. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method, system, device, and medium for matching drawing features with numbers.

[0006] According to the present invention, a method, system, device, and medium for matching drawing features with numbers are provided, and the solution is as follows:

[0007] Firstly, a method for matching drawing features with numbers is provided, the method comprising:

[0008] Step S1: Input the 3D model and automatically extract all relevant feature information; input the 2D drawing and automatically extract all annotation information;

[0009] Step S2: Project the entire 3D model to generate a model transformation projection image and projection information corresponding to the 3D model;

[0010] Step S3: Perform a hash calculation on the transformed projection image of the model and the two-dimensional drawing, and find the image with the highest hash similarity, which is the position of the cross-sectional projection image of the three-dimensional model corresponding to the two-dimensional drawing;

[0011] Step S4: Based on the feature information, annotation information, two-dimensional drawing, and step S3, determine the three-dimensional model projection image and projection information, determine which part of the three-dimensional model the annotation information corresponds to, and map the annotation information to the three-dimensional feature list;

[0012] Step S5: Output the annotated 3D model and feature annotation data details;

[0013] Step S1 includes: inputting a 3D model and automatically extracting all relevant feature information; inputting a 2D drawing and automatically extracting all annotation information.

[0014] Extract all relevant feature information of the 3D model by using custom development components of Siemens NX software;

[0015] The annotation information in two-dimensional drawings is extracted by combining AI computer vision technology and image algorithms.

[0016] Preferably, the AI ​​computer vision calculation process is as follows:

[0017] 1) Collect multiple processing drawings and divide them into training and testing sets according to a set scale;

[0018] 2) Based on the Paddle framework, DBNet and RCNN neural network models are combined to construct a deep learning algorithm for recognizing annotation information in two-dimensional drawings. The recognition algorithm extracts dimensions, text annotations, form and position tolerances, surface roughness and their locations from two-dimensional drawing images.

[0019] 3) Based on two-dimensional drawings and annotation information, train a neural network model, using accuracy as the main evaluation index and recall as an auxiliary optimization index;

[0020] 4) Based on image algorithms, identify all arrows and annotation lines in two-dimensional drawings and their positions by matching the shapes of arrows and annotation lines in the two-dimensional drawings.

[0021] Preferably, step S2 includes: generating projection information of the 3D model using a combination of OpenGL and OpenCV, and the process of OpenGL projecting the 3D model into a 2D target scene.

[0022] 1) Fix the camera on a tripod and point it at the scene to observe it from different positions;

[0023] 2) Arrange the scene so that the positions of each object in the photo move, rotate, or zoom in and out of the objects in the scene according to rules;

[0024] 3) Select the camera lens and adjust the magnification. When displaying an object, select how the object is projected onto the screen.

[0025] 4) Determine the size of the photo, zoom in or out, and draw the graphic to occupy the entire screen or a portion of the screen.

[0026] Preferably, OpenGL's process of displaying a 3D model on a 2D screen specifically includes:

[0027] 1) Coordinate transformation, which is described using a transformation matrix;

[0028] 2) The final 3D model will be displayed in a rectangular window; the portion outside this window will be cropped.

[0029] 3) After cropping, the final drawing is plotted on a 2D screen. The correspondence between the transformed coordinates and screen pixels is established, i.e., viewport coordinate transformation. Different model transformation projections are sliced ​​along the x, y, and z axes to generate the model transformation projection image and projection information corresponding to the 3D model.

[0030] Preferably, step S3 includes:

[0031] Step S3.1: Scale the images, uniformly scaling the image size to an N*N data matrix, resulting in a total of N... 2 1 pixel;

[0032] Step S3.2: Convert to grayscale image to unify the input standard for the next step, and convert all non-single-channel images to single-channel grayscale images;

[0033] Step S3.3: Calculate the DCT, select the M*M data matrix in the upper left corner, and calculate the N*N data matrix corresponding to the discrete cosine transform of the N*N data matrix;

[0034] Step S3.4: Reduce the DCT and take the M*M sub-region at the top left corner of the N*N data matrix obtained in the previous step;

[0035] Step S3.5: Calculate the average value. From the previous step, obtain an M*M integer matrix G, and calculate the average value of all elements in this matrix. Assume its value is a;

[0036] Step S3.6: Calculate the fingerprint and initialize the phash value of the input image;

[0037] Step S3.7: After obtaining the phash value of the image, compare the Hamming distance of the phash values of the two images. A group of images with a Hamming distance less than 10 are similar images.

[0038] Preferably, the specific content of the step S3.6 includes: Calculate the fingerprint and initialize the phash of the input image;

[0039] Traverse each pixel of the matrix G row by row from left to right;

[0040] If the element G(i,j) in the i-th row and j-th column >= a, then phash += "1";

[0041] If the element G(i,j) in the i-th row and j-th column < a, then phash += "0".

[0042] In a second aspect, a system for matching drawing features and numbers is provided. The system includes:

[0043] Module M1: Input a three-dimensional model and automatically extract all relevant feature information; input a two-dimensional drawing and automatically extract all annotation information;

[0044] Module M2: Perform an overall projection on the three-dimensional model to generate a model transformation projection image and projection information corresponding to the three-dimensional model;

[0045] Module M3: Perform hash calculation on the model transformation projection image and the two-dimensional drawing, and find the image with the highest hash similarity, which is the position of the cross-sectional projection image of the three-dimensional model corresponding to the two-dimensional drawing;

[0046] Module M4: Determine the three-dimensional model projection image and projection information according to the feature information, annotation information, two-dimensional drawing, and step S3, determine which part of the features of the three-dimensional model the annotation information corresponds to, and map the annotation information to the three-dimensional feature list;

[0047] Module M5: Output the three-dimensional model with annotations and the details of the feature annotation data;

[0048] The module M1 includes: Input a three-dimensional model and automatically extract all relevant feature information, input a two-dimensional drawing and automatically extract all annotation information;

[0049] Extract all relevant feature information of the 3D model by using custom development components of Siemens NX software;

[0050] The annotation information in two-dimensional drawings is extracted by combining AI computer vision technology and image algorithms.

[0051] Preferably, the AI ​​computer vision calculation process is as follows:

[0052] 1) Collect multiple processing drawings and divide them into training and testing sets according to a set scale;

[0053] 2) Based on the Paddle framework, DBNet and RCNN neural network models are combined to construct a deep learning algorithm for recognizing annotation information in two-dimensional drawings. The recognition algorithm extracts dimensions, text annotations, form and position tolerances, surface roughness and their locations from two-dimensional drawing images.

[0054] 3) Based on two-dimensional drawings and annotation information, train a neural network model, using accuracy as the main evaluation index and recall as an auxiliary optimization index;

[0055] 4) Based on image algorithms, identify all arrows and annotation lines in a 2D drawing and their positions by matching the shapes of arrows and annotation lines in the 2D drawing;

[0056] The module M2 includes: generating projection information of a 3D model using a combination of OpenGL and OpenCV, and the process of projecting the 3D model into a 2D target scene using OpenGL:

[0057] 1) Fix the camera on a tripod and point it at the scene to observe it from different positions;

[0058] 2) Arrange the scene so that the positions of each object in the photo move, rotate, or zoom in and out of the objects in the scene according to rules;

[0059] 3) Select the camera lens and adjust the magnification. When displaying an object, select how the object is projected onto the screen.

[0060] 4) Determine the size of the photo, zoom in or out, and draw the graphic to occupy the entire screen or a portion of the screen;

[0061] OpenGL completes the display of a 3D model on a 2D screen in the following ways:

[0062] 1) Coordinate transformation, which is described using a transformation matrix;

[0063] 2) The final 3D model will be displayed in a rectangular window; the portion outside this window will be cropped.

[0064] 3) After cropping, it is finally drawn on a two-dimensional screen, establishing the correspondence between the transformed coordinates and the screen pixels, that is, the viewport coordinate transformation, and performing different model transformation projections by slicing along the x, y, and z axes to generate the model transformation projection image and projection information corresponding to the three-dimensional model.

[0065] The module M3 includes:

[0066] Module M3.1: Scale the picture, uniformly scale the picture size to an N*N data matrix, and obtain a total of N 2 pixel points;

[0067] Module M3.2: Convert to grayscale image, unify the next input standard, and convert non single-channel pictures to single-channel grayscale images;

[0068] Module M3.3: Calculate the DCT, and select the upper left corner M*M data matrix to calculate the corresponding N*N data matrix after the discrete cosine transform of the N*N data matrix;

[0069] Module M3.4: Shrink the DCT, and take the upper left corner M*M sub-region of the N*N data matrix obtained in the previous step;

[0070] Module M3.5: Calculate the average value. Obtain an M*M integer matrix G from the previous step, calculate the average value of all elements in this matrix, and assume its value is a;

[0071] Module M3.6: Calculate the fingerprint and initialize the phash value of the input picture;

[0072] After obtaining the phash value of the picture, compare the Hamming distance of the phash values of two pictures. A group of pictures with a Hamming distance less than 10 are similar pictures;

[0073] The module M3.6 specifically includes: calculating the fingerprint and initializing the phash of the input picture;

[0074] Traverse each pixel of the matrix G row by row from left to right;

[0075] If the element G(i,j) in the i-th row and j-th column >= a, then phash += "1";

[0076] If the element G(i,j) in the i-th row and j-th column < a, then phash += "0".

[0077] In a third aspect, a device is provided, characterized in that the device includes:

[0078] One or more processors;

[0079] A storage device for storing one or more programs,

[0080] When the one or more programs are executed by the one or more processors, the one or more processors perform the steps in the method.

[0081] Fourthly, a computer-readable storage medium storing a computer program is provided, characterized in that the computer program, when executed by a processor, implements the steps in the method.

[0082] Compared with the prior art, the present invention has the following beneficial effects:

[0083] 1. This invention achieves spatial correspondence between 3D drawings and 2D views by using 3D model projection technology and image hashing algorithm, thereby realizing the matching between feature information in 3D drawings and annotation data in 2D views, solving the problem of isolated data information between 2D drawing annotations and 3D model features, which makes it impossible to automate information mapping.

[0084] 2. This invention significantly improves the efficiency and accuracy of two-dimensional drawing information recognition in production and processing, replacing the traditional method of manually recognizing drawings, reducing design costs, and accelerating the automation process of process design. Attached Figure Description

[0085] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0086] Figure 1 The main process data diagram;

[0087] Figure 2 It is a two-dimensional diagram;

[0088] Figure 3 This is a 3D projection image. Detailed Implementation

[0089] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0090] This invention provides a method for matching drawing features with numbers, referring to... Figure 1 As shown, the method specifically includes:

[0091] Step S1: Obtain the 3D model and automatically extract all relevant feature information (holes, slots, surfaces, etc.), including their specific locations and processing information (such as the order of the hole, the features of each order, etc.). Obtain the 2D drawing and automatically extract all annotation information, mainly including dimensions, text annotations, form and position tolerances, surface roughness and symbols, etc.

[0092] Currently, the extraction of all relevant feature information of 3D models is mainly achieved by custom-developing components for the Siemens NX software. The native Siemens NX software can only extract feature annotation information of holes, surfaces, and grooves in simple scenes. It cannot effectively identify complex scenes and feature combinations of certain shapes. However, the features of drawings in actual processing and production are complex and varied. Based on this, the company organized personnel to carry out secondary development of the components in the Siemens NX software. Currently, it can meet the feature recognition needs of most scenarios and ensure high accuracy of feature recognition.

[0093] By combining AI computer vision technology and image algorithms, annotation information in 2D drawings can be extracted. Currently, the industry's common method for extracting annotation information from 2D drawings is mainly rule-based, resulting in its applicability only to certain types of drawings, and requiring the addition of new rules as new drawings are added. Drawings outside the established rules cannot be effectively identified. Our company has innovatively solved this problem by combining AI vision algorithms and image algorithms. The specific AI vision computing process is as follows:

[0094] (1) Collect 600 automotive processing drawings and divide them into training and test sets in an 8:2 ratio. Based on the "National Standard for Mechanical Drawing", summarize 128 common symbols and their meanings in mechanical drawings. These symbols cover most of the annotation symbols in two-dimensional drawings.

[0095] (2) Based on the Paddle framework, and based on the combination of DBNet and RCNN neural network models, a deep learning algorithm for recognizing annotation information in two-dimensional drawings is constructed. The recognition algorithm can extract the dimensions, text annotations, form and position tolerances, surface roughness and their positions in two-dimensional drawing images.

[0096] (3) Based on the collected two-dimensional image information and labeled symbol information, a neural network model is trained. Accuracy is used as the main evaluation index and recall is used as the auxiliary optimization index. Currently, the accuracy of this algorithm model has reached over 90%.

[0097] (4) Based on image algorithms, based on standard two-dimensional drawing arrow shape and annotation line shape matching, identify all arrows and annotation lines and their positions in two-dimensional drawing images.

[0098] Step S2 mainly involves: projecting the 3D model as a whole to generate the model transformation projection image and projection information corresponding to the 3D model. OpenGL is a common technology platform for 3D projection in the industry. OpenGL's perspective projection model is similar to the pinhole projection model of a regular camera; its projection matrix corresponds to the camera's intrinsic parameter matrix K, and the observation matrix corresponds to the camera's extrinsic parameter matrix [R|T]. Considering project compatibility, this project mainly uses a combination of OpenGL and OpenCV to generate the 3D model's projection information. The reasons are: firstly, the coordinate system used by the OpenGL projection model is different from that of OpenCV, and the K and [R|T] obtained by OpenGL cannot be directly used; secondly, for Clipping, OpenGL's projection matrix needs to project points into NDC space.

[0099] The process by which OpenGL projects a 3D model into a 2D target scene is similar to taking a picture with a camera:

[0100] (1) Fix the camera on a tripod and let it point at the scene to observe the scene from different positions (view transformation);

[0101] (2) Arrange the scene so that the positions of each object in the photo are moved, rotated or enlarged and shrunk according to the rules (model transformation);

[0102] (3) When selecting a camera lens and adjusting the magnification (focusing) to display an object, you can choose how the object is projected onto the screen (projection transformation);

[0103] (4) Determine the size of the photo, whether to enlarge or shrink the photo, draw the graphic, and whether to occupy the entire screen or a part of the screen (viewport transformation).

[0104] In OpenGL, various transformations are implemented through matrix operations. Specifically, when a transformation command is issued, it generates a 4x4 transformation matrix (object coordinates in OpenGL are always homogeneous coordinates, i.e., (x, y, z, w), so all transformation matrices are 4x4 matrices). The current matrix is ​​multiplied by this transformation matrix to generate a new current matrix. For example, for vertex coordinates v, the transformation command is usually issued before the vertex coordinate command. If the current matrix is ​​C, and the matrix formed by the transformation command is M, then after issuing the transformation command, the new current matrix generated is CM. This matrix is ​​then multiplied by the vertex coordinates v to form the new vertex coordinates CMv. The above process illustrates that the last transformation command before drawing the vertex in the program acts on the vertex first. This also shows that in OpenGL programming, the actual transformation order is the reverse of the specified order.

[0105] OpenG is used to display 3D models on a 2D screen. This is done in three main parts:

[0106] 1) Coordinate Transformation: Coordinate transformation is described using transformation matrices. This transformation process in 3D drawing involves knowledge of spatial geometry and matrix operations. 3D coordinates are typically defined using homogeneous coordinates. Transformation matrix operations can be categorized into viewing, modeling, and projection operations. These operations can include selection, translation, scaling, orthographic projection, perspective projection, etc.

[0107] 2) Since the final 3D model needs to be displayed in a rectangular window, the parts outside this window need to be clipped to improve drawing efficiency. In 3D graphics, clipping means discarding the parts outside the clipping plane.

[0108] 3) When finally rendering to the display (2D screen), it is necessary to establish the correspondence between the transformed coordinates and the screen pixels. This is usually called "viewport transformation". Different model transformation projections are generated along the x, y, z axes to produce the model transformation projection image and projection information corresponding to the 3D model.

[0109] Step S2: Project the entire 3D model, referring to... Figure 2 and Figure 3 As shown:

[0110] By transforming the model while keeping the camera in a fixed position, objects are moved away from the camera to place them appropriately in the scene. Setting the object's position involves rotating, translating, and scaling the model. By default, the center of the object model (i.e., the drawing coordinate system) is located at the center of the world coordinate system. Different model transformation projections are implemented along the x, y, and z axes to generate the corresponding model transformation projection image and projection information for the 3D model.

[0111] Step S3: Using an image hash algorithm, the 3D model transformation projection image and the 2D model drawing generated in step S2 are hashed together. The image with the highest hash similarity is the location of the 3D model cross-section projection image corresponding to the 2D drawing.

[0112] Specifically, step S3 includes:

[0113] Step S3.1: Scale the images, uniformly scaling the image size to an N*N data matrix, resulting in a total of N... 2 1 pixel;

[0114] Step S3.2: Convert to grayscale image and standardize the input for the next step. Convert all non - single - channel images to single - channel grayscale images;

[0115] Step S3.3: Calculate the DCT and select the upper - left M*M data matrix. The DCT is a special Fourier transform that transforms the image from the pixel domain to the frequency domain. The DCT matrix represents coefficients of increasing frequencies from the upper - left to the lower - right corner. However, except for the upper - left corner, the coefficients elsewhere are 0 or close to 0. Therefore, only the low - frequency region in the upper - left corner is retained. Calculate the discrete cosine transform of the N*N data matrix to obtain the corresponding N*N data matrix; in this embodiment, N is equal to 32 and M is equal to 8;

[0116] Step S3.4: Shrink the DCT. Take the upper - left 8x8 sub - region of the 32x32 data matrix obtained in the previous step;

[0117] Step S3.5: Calculate the average value. Obtain an 8x8 integer matrix G from the previous step and calculate the average value of all elements in this matrix. Assume its value is a;

[0118] Step S3.6: Calculate the fingerprint and initialize the phash of the input image;

[0119] Traverse each pixel of matrix G row by row from left to right;

[0120] If the element G(i,j) in the i - th row and j - th column >= a, then phash += "1";

[0121] If the element G(i,j) in the i - th row and j - th column < a, then phash += "0".

[0122] Step S3.7: After obtaining the phash value of the image, compare the Hamming distance of the phash values of two images. A pair of images with a Hamming distance less than 10 are similar images.

[0123] Step S4: Integrate the drawing feature information, annotation information, two - dimensional drawing data, and determine the three - dimensional model projection image and information in step 3. According to the recognition feature position information of the projection image and the position information marked on the two - dimensional drawing, determine which part of the three - dimensional drawing features the extracted annotation information corresponds to, and map the annotation information recognized from the two - dimensional drawing to the three - dimensional feature list.

[0124] Step S5: Output the three - dimensional model with annotations and the details of the feature annotation data.

[0125] The present invention also provides a system for matching drawing features with numbers, and this system includes:

[0126] Module M1: Automatically extracts all relevant feature information (holes, slots, surfaces, etc.) from the 3D model, including their specific location and processing information (such as the order of the hole, the features of each order, etc.). Automatically extracts all annotation information from the 2D drawing, mainly including dimensions, text annotations, form and position tolerances, surface roughness and symbols, etc.

[0127] Currently, the extraction of all relevant feature information of 3D models is mainly achieved by custom-developing components for the Siemens NX software. The native Siemens NX software can only extract feature annotation information of holes, surfaces, and grooves in simple scenes. It cannot effectively identify complex scenes and feature combinations of certain shapes. However, the features of drawings in actual processing and production are complex and varied. Based on this, the company organized personnel to carry out secondary development of the components in the Siemens NX software. Currently, it can meet the feature recognition needs of most scenarios and ensure high accuracy of feature recognition.

[0128] By combining AI computer vision technology and image algorithms, annotation information in 2D drawings can be extracted. Currently, the industry's common method for extracting annotation information from 2D drawings is mainly rule-based, resulting in its applicability only to certain types of drawings, and requiring the addition of new rules as new drawings are added. Drawings outside the established rules cannot be effectively identified. Our company has innovatively solved this problem by combining AI vision algorithms and image algorithms. The specific AI vision computing process is as follows:

[0129] (1) Collect 600 automotive processing drawings and divide them into training and test sets in an 8:2 ratio. Based on the "National Standard for Mechanical Drawing", summarize 128 common symbols and their meanings in mechanical drawings. These symbols cover most of the annotation symbols in two-dimensional drawings.

[0130] (2) Based on the Paddle framework, and based on the combination of DBNet and RCNN neural network models, a deep learning algorithm for recognizing annotation information in two-dimensional drawings is constructed. The recognition algorithm can extract the dimensions, text annotations, form and position tolerances, surface roughness and their positions in two-dimensional drawing images.

[0131] (3) Based on the collected two-dimensional image information and labeled symbol information, a neural network model is trained. Accuracy is used as the main evaluation index and recall is used as the auxiliary optimization index. Currently, the accuracy of this algorithm model has reached over 90%.

[0132] (4) Based on image algorithms, based on standard two-dimensional drawing arrow shape and annotation line shape matching, identify all arrows and annotation lines and their positions in two-dimensional drawing images.

[0133] Step S2 mainly involves: projecting the 3D model as a whole to generate the model transformation projection image and projection information corresponding to the 3D model. OpenGL is a common technology platform for 3D projection in the industry. OpenGL's perspective projection model is similar to the pinhole projection model of a regular camera; its projection matrix corresponds to the camera's intrinsic parameter matrix K, and the observation matrix corresponds to the camera's extrinsic parameter matrix [R|T]. Considering project compatibility, this project mainly uses a combination of OpenGL and OpenCV to generate the 3D model's projection information. The reasons are: firstly, the coordinate system used by the OpenGL projection model is different from that of OpenCV, and the K and [R|T] obtained by OpenGL cannot be directly used; secondly, for Clipping, OpenGL's projection matrix needs to project points into NDC space.

[0134] The process by which OpenGL projects a 3D model into a 2D target scene is similar to taking a picture with a camera:

[0135] (1) Fix the camera on a tripod and let it point at the scene to observe the scene from different positions (view transformation);

[0136] (2) Arrange the scene so that the positions of each object in the photo are moved, rotated or enlarged and shrunk according to the rules (model transformation);

[0137] (3) When selecting a camera lens and adjusting the magnification (focusing) to display an object, you can choose how the object is projected onto the screen (projection transformation);

[0138] (4) Determine the size of the photo, whether to enlarge or shrink the photo, draw the graphic, and whether to occupy the entire screen or a part of the screen (viewport transformation).

[0139] In OpenGL, various transformations are implemented through matrix operations. Specifically, when a transformation command is issued, it generates a 4x4 transformation matrix (object coordinates in OpenGL are always homogeneous coordinates, i.e., (x, y, z, w), so all transformation matrices are 4x4 matrices). The current matrix is ​​multiplied by this transformation matrix to generate a new current matrix. For example, for vertex coordinates v, the transformation command is usually issued before the vertex coordinate command. If the current matrix is ​​C, and the matrix formed by the transformation command is M, then after issuing the transformation command, the new current matrix generated is CM. This matrix is ​​then multiplied by the vertex coordinates v to form the new vertex coordinates CMv. The above process illustrates that the last transformation command before drawing the vertex in the program acts on the vertex first. This also shows that in OpenGL programming, the actual transformation order is the reverse of the specified order.

[0140] OpenG is used to display 3D models on a 2D screen. This is done in three main parts:

[0141] 1) Coordinate Transformation: Coordinate transformation is described using transformation matrices. This transformation process in 3D drawing involves knowledge of spatial geometry and matrix operations. 3D coordinates are typically defined using homogeneous coordinates. Transformation matrix operations can be categorized into viewing, modeling, and projection operations. These operations can include selection, translation, scaling, orthographic projection, perspective projection, etc.

[0142] 2) Since the final 3D model needs to be displayed in a rectangular window, the parts outside this window need to be clipped to improve drawing efficiency. In 3D graphics, clipping means discarding the parts outside the clipping plane.

[0143] 3) When finally rendering to the display (2D screen), it is necessary to establish the correspondence between the transformed coordinates and the screen pixels. This is usually called "viewport transformation". Different model transformation projections are generated along the x, y, z axes to produce the model transformation projection image and projection information corresponding to the 3D model.

[0144] Module M2: Through model transformation, it maintains the camera's fixed position and moves objects away from the camera to place them appropriately in the scene. Setting the object's position involves rotating, translating, and scaling the model. By default, the object model (i.e., the drawing coordinate system) is centered at the world coordinate system. It implements different model transformation projections along the x, y, and z axes, generating the corresponding model transformation projection image and projection information for the 3D model.

[0145] Module M3: Implemented using an image hash algorithm, it performs hash calculations on the 3D model transformation projection image generated in Module M2 and the 2D model drawing, and finds the image with the highest hash similarity, which is the location of the 3D model cross-section projection image corresponding to the 2D drawing.

[0146] Module M3 specifically includes:

[0147] Module M3.1: Image scaling, uniformly scaling the image size to 32*32, resulting in a total of 1024 pixels;

[0148] Module M3.2: Convert to grayscale, standardize the input for the next step, and convert all non-single-channel images to single-channel grayscale images;

[0149] Module M3.3: Calculate the DCT, select the upper left 8*8 matrix, and calculate the corresponding 32x32 data matrix after the discrete cosine transform of the 32x32 data matrix;

[0150] Module M3.4: Shrink the DCT and take the upper left 8x8 sub-region of the 32x32 data matrix obtained in the previous step;

[0151] Module M3.5: Calculate the average value. Obtain an 8x8 integer matrix G from the previous step, calculate the average value of all elements in this matrix, and assume its value is a;

[0152] Module M3.6: Calculate the fingerprint and initialize the phash of the input image;

[0153] Traverse each pixel of matrix G row by row from left to right;

[0154] If the element G(i,j) in the i-th row and j-th column >= a, then phash += "1";

[0155] If the element G(i,j) in the i-th row and j-th column < a, then phash += "0".

[0156] Module M3.7: After obtaining the phash value of the picture, compare the Hamming distance of the phash values of two pictures. A group of pictures with a Hamming distance less than 10 are similar pictures.

[0157] Module M4: Integrate the drawing feature information, annotation information, two-dimensional drawing data, and module M3 to determine the three-dimensional model projection image and information. According to the recognition feature position information of the projection image and the position information marked on the two-dimensional drawing, determine which part of the three-dimensional drawing the extracted annotation information corresponds to, and map the annotation information recognized from the two-dimensional drawing to the three-dimensional feature list;

[0158] Module M5: Output the three-dimensional model with annotations and the details of the feature annotation data.

[0159] The embodiment of the present invention provides a method, system, device, and medium for matching drawing features and numbers. By adopting the three-dimensional model projection technology plus the image hashing algorithm, the spatial correspondence between the three-dimensional drawing and the two-dimensional view is realized, so as to realize the matching between the feature information in the three-dimensional drawing and the annotation data in the two-dimensional view, solve the problem of data information isolation between the two-dimensional drawing annotation and the three-dimensional model features, and the problem of inability to map automatically. The present invention greatly improves the recognition efficiency and accuracy of the two-dimensional drawing information in production processing, replaces the traditional method of manually identifying drawings, reduces the design cost, and accelerates the automation process of process design.

[0160] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0161] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for matching drawing features with numbers, characterized in that, The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision.

2. The method of matching features of a drawing sheet to a digital model of claim 1, wherein, The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision.

3. The method of matching features of a drawing sheet with a digital model according to claim 1, wherein, The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision.

4. The method of matching features of a drawing sheet to a digital model according to claim 3, wherein, The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically labeling a three-dimensional model based on a two-dimensional drawing, and belongs to the technical field of computer vision. The application relates to a method for automatically 3) After cutting, finally draw to the two-dimensional screen, establish the corresponding relationship between the transformed coordinates and the screen pixels, that is, the window coordinate transformation, slice different model transformation projection along the x, y, z axis, generate the model transformation projection image and projection information corresponding to the three-dimensional model.

5. The method of matching features of a technical drawing with a digital model according to claim 1, wherein, The step S3 comprises: Step S3.1: scaling the picture, uniformly scaling the picture size into a data matrix of N*N, a total of N*N pixel points are obtained. 2 ​ Step S3.2: Convert to grayscale image, unify the next step input standard, and convert all non-single-channel pictures to single-channel grayscale image; Step S3.3: Calculate DCT, and select the data matrix of the upper left corner M*M, calculate the corresponding N*N data matrix after the discrete cosine transformation of the N*N data matrix; Step S3.4: Reduce DCT, and take the M*M sub-region of the N*N data matrix obtained in the last step; Step S3.5: Calculate the average value, obtain an M*M integer matrix G through the last step, and calculate the average value of all elements in the matrix, which is a; Step S3.6: Calculate the fingerprint, initialize the phash value of the input picture; Step S3.7: After obtaining the phash value of the picture, compare the Hamming distance of the phash values of the two pictures, and the group of pictures with a Hamming distance less than 10 is similar pictures.

6. The method of matching features of a drawing sheet to a digital model of claim 5, wherein, The step S3.6 specifically comprises: calculating the fingerprint, initializing the phash of the input picture; Traverse each pixel of the matrix G from left to right row by row; If the element G(i,j) of the i-th row and j-th column is greater than or equal to a, then phash += "1"; If the element G(i,j) of the i-th row and j-th column is less than a, then phash += "0".

7. A system for matching drawing features with numbers, characterized in that, Comprise: Module M1: input three-dimensional model and automatically extract all related feature information; Input two-dimensional drawing and automatically extract all annotation information; Module M2: overall projection of the three-dimensional model, generate the model transformation projection image and projection information corresponding to the three-dimensional model; Module M3: hash calculation of the model transformation projection image and the two-dimensional drawing, and the model transformation projection image with the highest hash similarity is the section projection picture of the three-dimensional model corresponding to the two-dimensional drawing; Module M4: according to the feature information, annotation information, two-dimensional drawing and module M3, the model transformation projection image with the highest hash similarity is the section projection picture of the three-dimensional model corresponding to the two-dimensional drawing, determine which part of the feature of the three-dimensional model the annotation information corresponds to, and correspond the annotation information to the three-dimensional feature list; Module M5: output the three-dimensional model with annotation and feature annotation data details; The module M1 comprises: input three-dimensional model and automatically extract all related feature information, input two-dimensional drawing and automatically extract all annotation information; Extract all related feature information of three-dimensional model by customizing Siemens NX software components; Extract the annotation information in the two-dimensional drawing by combining AI computer vision technology and image algorithm.

8. The system for digital matching of drawing features according to claim 7, characterized in that, The AI computer vision calculation process is as follows: 1) Collect multiple processing process drawings, set the scale of the drawings into training set and test set; 2) Based on the paddle framework, the DBNet and RCNN neural network models are combined to construct a deep learning algorithm based on two-dimensional drawing annotation information recognition, and the recognition algorithm extracts the size, text annotation, shape and position tolerance, surface roughness and its position in the two-dimensional drawing picture; 3) Based on the two-dimensional drawing and the annotation information, the neural network model is trained, and the accuracy is taken as the main evaluation index, and the recall rate is taken as the auxiliary optimization index; 4) Based on the image algorithm, the two-dimensional drawing arrow shape and the annotation line shape are matched to recognize all the arrows and annotation lines and their positions in the two-dimensional drawing; The module M2 includes: generating projection information of the three-dimensional model in a manner of combining OpenGL and OpenCV, and the process of projecting the three-dimensional model into a two-dimensional target scene by OpenGL: 1) Fix the camera on the tripod, and observe the scene from different positions by aiming the camera at the scene; 2) Arrange the scene so that the positions of each object in the photo are moved, rotated or enlarged or reduced according to rules; 3) Select the camera lens and adjust the magnification, and when displaying the object, select how the object is projected onto the screen; 4) Determine the size of the photo, enlarge or reduce the photo, and draw the figure, which occupies the entire screen or part of the screen; The display of OpenGL from the 3D model to the two-dimensional screen specifically includes: 1) Coordinate transformation, which is described by using a transformation matrix; 2) The final 3D model will be displayed in a rectangular window, and the part outside the window will be cut off; 3) After cutting, finally draw to the two-dimensional screen, establish the corresponding relationship between the transformed coordinates and the screen pixels, that is, the view window coordinate transformation, slice different model transformations along the x, y and z axes, generate model transformation projection images and projection information corresponding to the three-dimensional model; The module M3 includes: Module M3.1: Scaling picture, uniformly scaling picture size into a data matrix of N*N, totally N*N pixel points. 2 Module M3.1: Scaling picture, uniformly scaling picture size into a data matrix of N*N, totally N*N pixel points. Module M3.2: Convert to grayscale image, input standard at a time, and convert all non-single-channel pictures to single-channel grayscale images; Module M3.3: Calculate DCT, and select the data matrix of the upper left corner M*M, and calculate the corresponding N*N data matrix after the discrete cosine transformation of the N*N data matrix; Module M3.4: Reduce DCT, and take the M*M sub-region of the N*N data matrix obtained in the last step; Module M3.5: Calculate the average value, obtain an integer matrix G of M*M through the last step, and calculate the average value of all elements in the matrix, which is a; Module M3.6: Calculate the fingerprint, and initialize the phash value of the input picture; Module M3.7: After obtaining the phash value of the picture, compare the Hamming distance of the phash values of the two pictures, and the group of pictures with a Hamming distance less than 10 is similar pictures; The module M3.6 specifically includes: calculating the fingerprint, initializing the phash of the input picture; Traverse the matrix G one row at a time from left to right; If the element G(i,j) of the i-th row and j-th column is greater than or equal to a, then phash += "1"; If the element G(i,j) of the i-th row and j-th column is less than a, then phash += "0".

9. An apparatus, comprising: The device includes: One or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, cause the one or more processors to carry out the steps of the method of any one of claims 1 to 6.

10. A computer readable storage medium storing a computer program, characterized in that, the computer program which, when executed by the processor, carries out the steps of the method of any one of claims 1 to 6.

Images