Method and device for similar search of engineering drawings

Abstract

The invention provides an engineering drawing similarity search method and device, which can accurately search and return similar engineering drawings according to a target drawing. The engineering drawing comprises a target drawing and candidate drawings, the target drawing comprises a plurality of views corresponding to one industrial product, each candidate drawing comprises a plurality of views corresponding to one industrial product, each view corresponds to a vector, and the method comprises the steps that labels in the engineering drawing are removed, and structural lines in the engineering drawing are reserved; a plurality of views in the engineering drawing are extracted respectively, and a sub-image group is obtained; based on a first target vector group, performing mixed search with each second target vector group to obtain a similarity value, obtaining candidate drawings in a preset similarity value range and returning the candidate drawings, the first target vector group comprising vectors obtained by training sub-image groups of the target drawings by using an image processing model, and the second target vector group comprising vectors obtained by training sub-image groups of the target drawings by using an image processing model; the second target vector group comprises vectors obtained by training the sub-image groups of the corresponding candidate drawings by using the image processing model.

Description

Technical Field

[0001] The present invention relates to the field of similarity search, and particularly to a method and device for similarity search of engineering drawings. Background Art

[0002] In today's engineering design field, engineering drawings mainly use two-dimensional Computer Aided Design (CAD) drawings (hereinafter referred to as CAD drawings) as key information carriers and are widely used. In addition to presenting technical contents such as precise dimensions and technical specifications of parts, they also contain note information such as tolerance comparison and drawing titles, with detailed and comprehensive content.

[0003] In manufacturing enterprises, with product iteration, a large number of CAD drawings have been accumulated. It is necessary to effectively manage the drawings, especially to add the function of similarity search for CAD drawings. When designers need to refer to the design ideas of similar old products to optimize new products, it is time-consuming and laborious to manually search for useful drawings from the complex drawing library and it is easy to miss. Another example is that component suppliers need to quickly respond according to the provided drawings and previous similar drawings in the face of diverse non-standard customization requirements of customers.

[0004] Currently, there are deficiencies in the analysis technology of drawing structured information, which cannot meet the demands of the professional field for accurate and efficient search of CAD drawings, resulting in low accuracy and low search efficiency of two-dimensional drawing search results. Summary of the Invention

[0005] In view of the above problems, the present invention provides a method and device for similarity search of engineering drawings, which can solve the problems of low accuracy and low search efficiency when searching for similar engineering drawings.

[0006] The method for similarity search of engineering drawings provided in the first aspect of the present invention, where the engineering drawings include a target drawing and candidate drawings. The target drawing includes a plurality of mutually corresponding views corresponding to an industrial product, and each candidate drawing includes a plurality of mutually corresponding views corresponding to an industrial product. Each view corresponds to a vector. The method for similarity search of engineering drawings includes a marking removal step, a collection step, and a search step.

[0007] Marking removal step: Remove the markings in the engineering drawing and retain the structure lines in the engineering drawing.

[0008] Collection step: Extract a plurality of views in the engineering drawing obtained in the marking removal step respectively to obtain a sub-image group.

[0009] Search step: Based on the first target vector group of the target drawing, perform a hybrid search with the second target vector group of each candidate drawing to obtain a similarity value, and obtain and return the candidate drawings within the preset similarity value range. Among them, the first target vector group includes vectors obtained by training sub-image groups of the target drawing using an image processing model, and the second target vector group includes vectors obtained by training sub-image groups of the corresponding candidate drawing using an image processing model.

[0010] In the above way, after deleting the annotations and corresponding lines in the engineering drawing and then collecting the sub-image groups, the visual complexity of the engineering drawing can be simplified, the interference of the annotations on the search of the engineering drawing can be avoided, which is convenient for the image processing model to train the view after removing the annotations and obtain the corresponding vectors, so as to quickly find the similar part drawings of the drawing to be queried according to the vectors, improve the accuracy of the engineering drawing search results, and greatly improve the search efficiency. At the same time, it avoids omission and repetitive labor when manually searching for similar engineering drawings, reduces the labor intensity, and realizes the effective management of the drawings.

[0011] Optionally, the annotation removal step includes: removing the annotations using layers according to the type of the engineering drawing, or removing the annotations after parsing the engineering drawing.

[0012] In the above way, for the engineering drawings designed in a standardized manner according to layers, the process of parsing the engineering drawings can be omitted, and the efficiency of removing annotations can be improved.

[0013] Optionally, the collection step includes a segmentation step, a cropping step, and a step of obtaining sub-image groups.

[0014] Segmentation step: Use a segmentation model to segment the engineering drawing to obtain masks corresponding to multiple views, and use the masks to process the engineering drawing to obtain local sub-image groups corresponding to each view.

[0015] Cropping step: Based on the local sub-image groups, obtain the central pixel points, and perform image cropping according to the central pixel points to obtain the sub-image groups.

[0016] Step of obtaining sub-image groups: Return the corresponding sub-image groups according to the preset number of views.

[0017] In the above way, obtain the masks according to the segmentation model, and use the masks to process the local parts of the views to obtain local sub-image groups, which increases the processing steps for the local parts of the views. And crop the local sub-image groups based on the central pixel points to standardize the local sub-image groups.

[0018] Optionally, the cropping step includes:

[0019] For each local sub-image in the local sub-image group, obtain the central pixel point according to the circumscribed contour of the local sub-image;

[0020] Obtain the distance from the central pixel point to the end of the circumscribed contour, and perform cropping according to the square contour based on the end corresponding to the maximum value of the distance.

[0021] In the above way, according to the distance between the central pixel point and the end of the circumscribed contour, the local sub-image is cropped into a square shape, which can avoid missing local features of the view and ensure the correspondence between the sub-image group and the view.

[0022] Optionally, the step of obtaining the sub-image group includes: when the number of views of the engineering drawing is greater than or equal to the preset number, filter the sub-image group so that the number of sub-image groups is equal to the preset number; when the number of views of the engineering drawing is less than the preset number, return the sub-image group.

[0023] In the above way, the views can be filtered to avoid too many views, thereby optimizing the correspondence between the views and the sub-image group.

[0024] Optionally, the engineering drawing is provided with extractable text.

[0025] Before the search step, it further includes the step of obtaining extractable text:

[0026] Remove the annotations of the engineering drawing according to the mask to obtain the technical requirement text and the drawing annotation text of the engineering drawing;

[0027] Use the named entity recognition model to extract the extractable text from the technical requirement text and the drawing annotation text.

[0028] In the above way, the extractable text in the engineering drawing is extracted, which is convenient for enhancing the understanding of the text and multiple views of the engineering drawing.

[0029] Optionally, before the search step, it further includes:

[0030] For the sub-image group, perform vectorization based on the image processing model to obtain a view vector group;

[0031] For the extractable text, perform vectorization based on the image processing model to obtain an extractable text vector;

[0032] Merge the view vector group and the extractable text vector to obtain a target vector group, where the target vector group includes a first target vector group corresponding to the target drawing and a second target vector group corresponding to each candidate drawing.

[0033] In the above way, the sub-image group and the extractable text are vectorized, which is convenient for unified analysis and processing of the engineering drawing according to the sub-image group and the extractable text.

[0034] Optionally, before the search step, it further includes a library building step:

[0035] Store the candidate drawings in the database;

[0036] Establish a hash index for each candidate drawing in the database, and establish a one-to-one correspondence between the candidate drawings and the corresponding second target vector groups;

[0037] Store the view vector groups and extractable text vectors in the second target vector groups by columns.

[0038] In the above way, establish the correspondence between the candidate drawings and the second target vector groups, prevent the loss of candidate drawings and related data, and facilitate the expansion of candidate drawings. At the same time, storing by columns can optimize the data compression effect, improve the query efficiency and the utilization rate of the cache.

[0039] Optionally, the similarity value includes a view similarity value and a text similarity value.

[0040] The search steps include:

[0041] Traverse the view vector group of the first target vector group by columns, match it one by one with the view vector group of each second target vector group and calculate the similarity to obtain the view similarity value;

[0042] Calculate the text similarity value between the extractable text vector of the first target vector group and the extractable text vector of the second target vector group;

[0043] Sort the candidate drawings according to the view similarity value and the text similarity value, and obtain and return the candidate drawings whose sorting is within the preset similarity range.

[0044] In the above way, compare the similarity between the candidate drawings and the target drawings according to the two dimensions of the view similarity value and the text similarity value, which is beneficial to improving the accuracy of the similarity search results of engineering drawings.

[0045] The second aspect of the present invention also provides a device for similar search of engineering drawings, including a preprocessing module, a collection module, a training module and a search module.

[0046] The preprocessing module is used to remove the annotations in the engineering drawings and retain the structure lines in the engineering drawings. Among them, the engineering drawings include target drawings and candidate drawings. The target drawings include multiple mutually corresponding views corresponding to an industrial product, and each candidate drawing includes multiple mutually corresponding views corresponding to an industrial product, and each view corresponds to a vector.

[0047] The collection module is used to respectively extract multiple views in the engineering drawings obtained by the preprocessing module to obtain a sub-image group.

[0048] A training module, which is used to train a sub-image group of engineering drawings by using an image processing model to obtain a first target vector group corresponding to a target drawing and a second target vector group corresponding to each candidate drawing.

[0049] A search module, which is used to perform a hybrid search on the first target vector group of the target drawing and the second target vector group of each candidate drawing to obtain a similarity value, and obtain and return candidate drawings within a preset similarity value range.

[0050] In the above manner, the device for similar search of engineering drawings can return candidate drawings with high similarity according to the target drawing, improving the application efficiency of engineering drawings in practice. Description of the Drawings

[0051] Figure 1 It is a flowchart of the method for similar search of engineering drawings in the embodiment of the present invention.

[0052] Figure 2 It is a flowchart of the annotation removal step in the embodiment of the present invention.

[0053] Figure 3 It is a schematic diagram of the target drawing in the embodiment of the present invention.

[0054] Figure 4 It is a schematic diagram of the target drawing after annotation removal in the embodiment of the present invention.

[0055] Figure 5 It is a flowchart of the acquisition step in the embodiment of the present invention.

[0056] Figure 6 It is a schematic diagram of the mask in the embodiment of the present invention.

[0057] Figure 7 It is a schematic diagram of the local sub-image in the embodiment of the present invention.

[0058] Figure 8 It is a schematic diagram of the sub-image after cropping in the embodiment of the present invention.

[0059] Figure 9 It is a flowchart of the cropping step in the embodiment of the present invention.

[0060] Figure 10 It is another flowchart of the method for similar search of engineering drawings in the embodiment of the present invention.

[0061] Figure 11 It is a schematic diagram of the technical requirement text of the target drawing in the embodiment of the present invention.

[0062] Figure 12 It is another schematic diagram of the technical requirement text of the target drawing in the embodiment of the present invention.

[0063] Figure 13 This is a flowchart of the search step in the embodiment of the present invention.

[0064] Figure 14 This is a schematic diagram of the search step in the embodiment of the present invention.

[0065] Figure 15 This is another schematic diagram of the search step in the embodiment of the present invention. Detailed implementation manners

[0066] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0067] First of all, it should be noted that the engineering drawings in the embodiments of the present invention are CAD drawings. In some other embodiments of the present invention, the engineering drawings can also be other types of two-dimensional drawings, and the present invention does not limit this.

[0068] For the convenience of understanding the technical solutions of the present invention, the technical problems of the present invention will be described in detail below.

[0069] Compared with the image search in the prior art, there are significant differences in the two-dimensional CAD drawing search. The existing image search focuses on the overall visual features of the picture, and can extract features such as color, texture, and shape contour to obtain a hash value, or obtain a vector by passing the entire picture through an image classification network framework for retrieving daily images such as landscape and portrait photos. However, CAD drawings are highly professional. Their content may include multiple views, and the view lines show geometric relationships and precise dimensions, rather than simple visual elements. In addition, CAD drawings cover part processing technologies, such as text information on material, surface treatment, heat treatment hardness, etc., resulting in the inapplicability of existing image search algorithms. Considering the specific design and processing quotation of parts, in addition to using the features of vision, shape, and size, the text entities in the drawings should also be combined, and diversified data should be fused to perform similarity search of the drawings, so as to improve the accuracy.

[0070] That is to say, in the image search of the prior art, there is a lack of relevant research on the layout analysis of drawings, and it is difficult to further extract image and text information in terms of content. There are a large number of dimensioning texts and auxiliary lines on the main body of the drawing view for explanation, and the lines of these annotation information will greatly increase the visual complexity. Moreover, there are various ways of dimensioning that meet the specifications, which cause great interference to image matching. In addition, the drawing text formats are diverse and the annotation positions are arbitrary, making it difficult to understand the context of the drawing and accurately extract the key text entity information.

[0071] <Method for Similarity Search of Engineering Drawings>

[0072] The present invention provides a method for similarity search of engineering drawings, which can simplify the visual complexity of engineering drawings and improve the accuracy of similarity search of engineering drawings.

[0073] Reference Figure 1 , the method for similarity search of engineering drawings includes a marking removal step S1, a collection step S2, and a search step S3. The engineering drawings include a target drawing and a plurality of candidate drawings. The target drawing includes a plurality of mutually corresponding views corresponding to an industrial product, and each candidate drawing includes a plurality of mutually corresponding views corresponding to an industrial product. Each view corresponds to a vector representing the drawing features. In the present invention, the view includes a standard view and an axonometric drawing.

[0074] Marking removal step S1: Remove the markings on the multiple views of the engineering drawing and retain the structure lines in the multiple views. The marking is specifically the dimensioning line.

[0075] Collection step S2: Extract the multiple views in the engineering drawing obtained in the marking removal step S1 respectively to obtain a sub-image group.

[0076] Search step S3: Based on the first target vector group of the target drawing, perform a hybrid search with the second target vector group of each candidate drawing to obtain a similarity value, and obtain and return the candidate drawings within the preset similarity value range.

[0077] In the search step S3, the first target vector group includes the vectors obtained by training the sub-image group of the target drawing using an image processing model, and the second target vector group includes the vectors obtained by training the sub-image group of the corresponding candidate drawing using an image processing model. That is to say, the first target vector group and the second target vector group are trained before the search step S3.

[0078] By the above method, after removing the annotations and the corresponding lines in the engineering drawings and then collecting the sub-image groups, the visual complexity of the engineering drawings can be simplified, the interference of the annotations on the search of the engineering drawings can be avoided, and it is convenient for the image processing model to train the views after removing the annotations and obtain the corresponding vectors, so as to quickly find the similar part drawings of the drawing to be queried according to the vectors, improve the accuracy of the search results of the engineering drawings, and greatly improve the search efficiency. At the same time, it avoids omission and repetitive labor when manually searching for similar engineering drawings, reduces the labor intensity, and realizes the effective management of the drawings.

[0079] The steps in the embodiments of the present invention will be described in detail below.

[0080] <Annotation removal step S1>

[0081] Combined with Figure 3 and Figure 4 、Reference Figure 2 ,The annotation removal step S1 includes the following steps S11-S13. Through the annotation removal step S1, the Figure 3 engineering drawing is removed of annotations to obtain the Figure 4 engineering drawing without annotations.

[0082] Step S11: Determine the type of the engineering drawing. When the engineering drawing is in DWG or DXF format, execute step S12. When the engineering drawing is in PDF format, execute step S13.

[0083] Step S12: Use the layer to remove the annotations.

[0084] Specifically, for DWG and DXF formats, if there are set layers and the layers are designed according to the standardization, the CAD editor can be used to directly remove the dimension auxiliary line layer and the drawing frame title bar layer, and only retain the contour lines of the standard view and the axonometric drawing, and finally export a file in picture format. If the layers are not set or the layers are not designed according to the standardization, the drawings in the above formats are converted into PDF format and returned to the previous step S11.

[0085] Step S13: Remove the annotations after parsing the engineering drawing.

[0086] For PDF format, parse to obtain all line elements such as straight lines, curves, polylines and circles, and distinguish the contour lines of the standard view and the axonometric drawing, the dimension annotation lines and the drawing frame lines according to the line width or color, and obtain the drawing after removing the text and the dimension auxiliary lines, which specifically includes the following two methods.

[0087] The first method is to obtain all the medium line widths in the file according to the line width. Since the line width of the dimension annotation is smaller than that of the part line, the dimension annotation lines can be removed by screening out the lines with smaller line widths according to the line width.

[0088] The second method is based on the line color. Since the view dimension lines will be highlighted in color, directly retain the black and gray lines and remove the colored lines, then the outline lines of the view or axonometric drawing can be retained.

[0089] Through the above method, for engineering drawings that are standardized according to layers, the process of parsing the engineering drawings can be omitted, improving the efficiency of removing annotations.

[0090] <Collection step S2>

[0091] Combined with Figures 6 - 8 and referring to Figure 5 , the collection step S2 includes a segmentation step S21, a clipping step S22, and a step S23 of obtaining a sub-image group.

[0092] Segmentation step S21: The segmentation model includes an instance segmentation model. Use the trained instance segmentation model to segment the engineering drawing to obtain masks of modules such as standard views, axonometric drawings, technical requirements, and drawing annotations (as shown in Figure 6 ). Then perform content extraction: Generate a set of masks based on the segmentation results of a single standard view or a single axonometric drawing, and use each mask to process the multiple views obtained in step S1 to generate a set of local sub-image groups with local image elements (as shown in Figure 7 ).

[0093] Clipping step S22: Based on the local sub-image group, obtain the central pixel points, and perform image clipping according to the central pixel points to obtain the sub-image group (as shown in Figure 8 ).

[0094] Step S23 of obtaining the sub-image group: According to a preset number of views, return the corresponding sub-image group.

[0095] Through the above method, obtain masks according to the segmentation model, and use the masks to process the local parts of the views to obtain local sub-image groups, which increases the processing steps for the local parts of the views. And clip the local sub-image groups based on the central pixel points to standardize the local sub-image groups.

[0096] Combined with Figure 7 and Figure 8 and referring to Figure 9 , the clipping step S22 includes steps S221 - S222.

[0097] Step S221: For each local sub-image in the local sub-image group obtained in step S21 (as shown in Figure 7 ), calculate the circumscribed contour of the local sub-image. The circumscribed contour is rectangular. Obtain the central pixel points according to the rectangular circumscribed contour.

[0098] Step S222: Obtain the distances from the central pixel point to the four sides of the circumscribed contour of the rectangle. Cut according to the side corresponding to the maximum distance in accordance with the square contour (as Figure 8 shown), to obtain a sub-image.

[0099] In the above manner, according to the distances between the central pixel point and the ends of the circumscribed contour, the local sub-image is cut into a square shape, which can avoid missing local features of the view and ensure the correspondence between the sub-image group and the view.

[0100] Specifically, the step S23 of obtaining the sub-image group includes: when the number of views of the engineering drawing is greater than or equal to the preset number, filter the sub-image group so that the number of sub-image groups is equal to the preset number; when the number of views of the engineering drawing is less than the preset number, return the sub-image group.

[0101] The views in step S23 include standard views and axonometric views. Set the required number of standard views as m and the number of axonometric views as n. For example, if in addition to the three-view drawing, there are other standard views such as partial views in the engineering drawing and the number exceeds m, filter some smaller sub-images from the sub-image group of the standard views according to the image size, and organize m sub-images to form the sub-image group of the standard views. If the actual number of standard views in the engineering drawing is less than m, only the actual sub-image group of the standard views needs to be returned.

[0102] In the above manner, the views can be filtered to avoid excessive number of views, thereby optimizing the correspondence between the views and the sub-image group.

[0103] Specifically, the engineering drawing is provided with extractable text.

[0104] Refer to Figures 10 - 12 , before the search step S3, it may also include the step of obtaining extractable text, and the step of obtaining extractable text includes the following two steps.

[0105] The first step: According to the segmentation result of step S21, use the masks of the standard view and the axonometric view to remove the annotations on the engineering drawing, to obtain the technical requirement text and the drawing annotation text of the engineering drawing.

[0106] The second step: Use the trained Named Entity Recognition (NER) model to extract the text information of processing technologies such as material, surface treatment, heat treatment, and hardness from the technical requirement text and the drawing annotation text, and return the result in the form of a dictionary, so as to obtain the extractable text.

[0107] For example, Figure 11 the extractable text is: {Material: "40Gr"; Surface treatment: "Nickel plating"; Heat treatment: "Quenching and tempering"; Hardness: "HRC60 - 62"};

[0108] Figure 12 The extractable text is: {Material: "6061"; Surface treatment: "Natural anodizing"; Heat treatment: ""; Hardness: ""}.

[0109] Specifically, the training method of the named entity model for the processing technology text is as follows: Some existing drawing text data are labeled as a data set, and the BIOES labeling method is adopted for the short sentences in the technical requirements and the short sentences or words obtained from the figure notes. Here, the named entity recognition model uses a convolutional neural network (CNN) based on a dictionary plus a conditional random field (CRF). The processing technology text belongs to a specific field, and constructing a dictionary helps with data learning. Incorporating the dictionary data into the CNN network structure of named entity recognition accelerates the training of the model and the accuracy of the results.

[0110] By the above method, the extractable text in the engineering drawings is extracted, which is convenient for enhancing the understanding of the text and multiple views of the engineering drawings.

[0111] Reference Figure 10 , before the search step S3 and after the step of obtaining the extractable text, there is also a training step, that is, using the trained Visual Transformer model. For example, the CLIP (Contrastive Language-Image Pre-training) model network can be used for feature extraction and vectorization. The model converts the sub-image group and the extractable text into vector sets respectively.

[0112] Specifically, the training step includes the following three steps.

[0113] The first step: For the sub-image group obtained in step S23, vectorization is performed based on the CLIP model to obtain a view vector group. The view vector group includes the vector group E v of the standard view and the vector group E c of the axonometric drawing, specifically as follows:

[0114]

[0115]

[0116] The second step: For the extractable text obtained in the step of obtaining the extractable text, vectorization is performed based on the CLIP model to obtain the extractable text vector e t .

[0117] Step 3: Combine the view vector group and the extractable text vector to obtain the target vector group E. The target vector group includes a first target vector group corresponding to the target drawing and a second target vector group corresponding to each candidate drawing. The target vector group E is specifically as follows:

[0118] E = {E v , E c , e t}.

[0119] In the above way, the sub-image group and the extractable text are vectorized to obtain the structured information about a view, which is convenient for unified analysis and processing of engineering drawings based on the sub-image group and the extractable text.

[0120] Specifically, before the search step S3 and after the training step, there is also an indexing step, which includes the following three steps.

[0121] Step 1: Store the candidate drawings in the database. For example, if there are k candidate drawings, train the dataset {I i , 0 < i ≤ k} of k candidate drawings to obtain the vector set {E i , 0 < i ≤ k} of each candidate drawing. The vector combinations of the k candidate drawings form the second target vector group. The second target vector group can be expressed as:

[0122] E candidate = {E1, E2, … E k}

[0123] Step 2: Establish a hash index for each candidate drawing in the database, and establish a one-to-one correspondence between the k candidate drawings and the vector sets in the corresponding second target vector group. Use the one-to-one mapping relationship to improve the query speed.

[0124] Step 3: Store the view vector group and the extractable text vector in the second target vector group by columns. That is, for each vector set E i , 0 < i ≤ k in the second target vector group, store the individual vectors by columns to obtain m standard view vector columns, n axonometric drawing vector columns, and one extractable text vector column.

[0125] In the above way, the correspondence between the candidate drawings and the second target vector group is established, preventing the loss of candidate drawings and related data, and facilitating the expansion of candidate drawings. At the same time, storing by columns can optimize the data compression effect, improve the query efficiency and the utilization rate of the cache.

[0126] <Search step S3>

[0127] Further, the similarity value includes a view similarity value and a text similarity value, and the view similarity value includes a standard view similarity value and an axonometric view similarity value. Combining Figure 14 and Figure 15 、referring to Figure 13 , the corresponding search step S3 may include the following three steps S31 - S33.

[0128] Step S31: Traverse the view vector group of the first target vector group column by column, match it one by one with the view vector group of each second target vector group, and calculate the similarity to obtain the view similarity value.

[0129] Specifically, traverse the m q (m q (m q ≤m) standard view vector columns of the first target vector group E i , and match them one by one with the m i (m i ≤m) standard view vector columns of each vector set E q of the second target vector group, and calculate the similarity. The number of calculations is m i *m q . Take the top - ranked similarity values obtained in the number of calculations to obtain m i similarity values S1 = {s q , 0 < i ≤ m q}}. Then average the m

[0130]

[0131] similarity values to obtain the standard view similarity value as follows: q Traverse the n q (n q ≤n) axonometric view vector columns of the first target vector group E i , and match them one by one with the n i (n i ≤n) axonometric view vector columns of each vector set E q *n i . Take the top - ranked similarity values obtained in the number of calculations to obtain n q similarity values S2 = {s j , 0 < j ≤ n q}}. Then average the n q similarity values to obtain the axonometric view similarity value as follows:

[0132]

[0133] Step S32: For the first target vector group E qThe extractable text vector and each vector set E of the second target vector group i calculate the similarity with the extractable text vector to obtain a text similarity value

[0135] Here, the similarity metric can use cosine similarity, and the calculation method is as follows: Given two vectors A and B, according to the dot product formula a·b = ‖a‖‖b‖cos(θ), the cosine similarity calculation formula is as follows:

[0136]

[0137] The similarity result is between [0, 1]. The closer the value is to 0, the less similar the two vectors are. The closer the value is to 1, the more similar the two vectors are.

[0138] It should be noted that in steps S31 and S32, for the data of the second target vector group of small-scale candidate drawings, full-scale search can be adopted. However, for the data of large-scale candidate drawings, the time complexity of the full-scale search method is very large. Therefore, an approximate search method is adopted. The vector data set of the second target vector group of each view in the database is first divided into n list subspaces through clustering algorithm training, and the subspace to which each vector set belongs is determined. The centers of these subspaces are called clustering centers c i , where i = 1, 2,... n list . For the vector set of the first target vector group, calculate the distance from each clustering center c i . Determine the n probe subspaces with the closest distance for search. That is, after screening the candidate drawings corresponding to these n probe subspaces, calculate the similarity of the candidate drawings again.

[0139] Step S33: Sort the candidate drawings according to the view similarity value and the text similarity value, and obtain and return the candidate drawings ranked within the preset similarity range.

[0140] Specifically, use the improved Ranked Retrieval Fusion (RRF) algorithm to sort the candidate drawing indexes in descending order according to the three similarity scores respectively. Sum the reciprocals of the three rankings of each drawing as the fused similarity score, sort all candidate drawing indexes in descending order according to the fused score, return the top K indexes, and map the candidate drawings to obtain the final similar drawing results.

[0141] Ranked Retrieval Fusion (RRF) is a fusion algorithm used to fuse multiple query results to improve the accuracy of search results. The basic principle of RRF is to rank each query result, assign weights according to the ranking, and finally accumulate the weights of each query result to generate the fused result.

[0142] The mathematical formula of RRF is as follows:

[0143]

[0144] where R represents the number of columns in the sorted result column, rank i (d) represents the ranking of query d in the i-th sorted result, and k is a constant used to smooth the ranking.

[0145] According to the consideration of the importance of views, axonometric drawings, and text in the drawings, RRF is improved. The specific calculation formula is as follows:

[0146]

[0147] where k v 、k c 、k t are constants, and k v < k c < k t .

[0148] In the above way, by comparing the similarity between the candidate drawing and the target drawing in terms of the view similarity value and the text similarity value, it is beneficial to improve the accuracy of the similar search results of engineering drawings, facilitate quickly finding the similar part drawings of the drawing to be queried, greatly improve the design efficiency, and avoid repetitive labor.

[0149] The present invention also provides an image search system. The database builds an index and stores the structured information of candidate image data. After converting the data of the target image into the same type of structured information, similarity calculation is performed, and the top K index results are sorted and returned in descending order of the similarity value. Finally, the images mapped according to the index are returned. After the search is completed, the structured information of the target image is stored in the database again for subsequent searches.

[0150] <Device for Similar Search of Engineering Drawings>

[0151] The second aspect of the present invention also provides a device for similar search of engineering drawings, including a preprocessing module, a collection module, a training module, and a search module.

[0152] The preprocessing module is used to remove the annotations in the engineering drawings and retain the structural lines in the engineering drawings. Among them, the engineering drawings include the target drawing and the candidate drawings. The target drawing includes multiple corresponding views corresponding to an industrial product, and each candidate drawing includes multiple corresponding views corresponding to an industrial product. Each view corresponds to a vector.

[0153] The acquisition module is used to respectively extract multiple views in the engineering drawings obtained by the preprocessing module and obtain a sub-image group.

[0154] The training module is used to train the sub-image group of the engineering drawings by using an image processing model to obtain a first target vector group corresponding to the target drawing and a second target vector group corresponding to each candidate drawing.

[0155] The search module is used to perform a hybrid search based on the first target vector group of the target drawing and the second target vector group of each candidate drawing to obtain a similarity value, and obtain and return the candidate drawings within a preset similarity value range.

[0156] In the above manner, the device for similar search of engineering drawings can return candidate drawings with high similarity according to the target drawing, improving the application efficiency of engineering drawings in practice.

[0157] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for similarity search of engineering drawings, wherein the engineering drawings include a target drawing and a candidate drawing, the target drawing includes a plurality of mutually corresponding views corresponding to an industrial product, each of the candidate drawings includes a plurality of mutually corresponding views corresponding to an industrial product, each of the views corresponds to a vector, characterized in that: The method for similarity search of engineering drawings comprises: a marking removal step, removing the markings in the engineering drawing and retaining the structural lines in the engineering drawing; an acquisition step of respectively extracting a plurality of views in the engineering drawing obtained in the annotation removal step to obtain a sub-image group; A search step is performed based on the first target vector group of the target drawing and the second target vector group of each candidate drawing to obtain a similarity value, obtain the candidate drawings within a preset similarity value range and return them, wherein the first target vector group includes vectors trained on the sub-image group of the target drawing using an image processing model, and the second target vector group includes vectors trained on the corresponding sub-image group of the candidate drawings using the image processing model.

2. The method for similarity search of engineering drawings according to claim 1, characterized in that: The label removal step comprises: According to the type of the engineering drawing, the marking is removed by using a layer, or the marking is removed after parsing the engineering drawing.

3. The method for similarity search of engineering drawings according to claim 1, characterized in that: The collection step comprises: a segmentation step, using a segmentation model to segment the engineering drawing to obtain masks corresponding to a plurality of the views, and using the masks to process the engineering drawing to obtain a local sub-image group corresponding to each of the views; A cropping step, based on the local sub-image group, obtaining a central pixel point, and performing image cropping according to the central pixel point to obtain the sub-image group; The step of obtaining a sub-image group returns the corresponding sub-image group according to a preset number of views.

4. The method for similarity search of engineering drawings according to claim 3, characterized in that: The cutting step comprises: For each local sub-image in the local sub-image group, acquiring the central pixel point according to the circumscribed contour of the local sub-image; The distance from the central pixel point to the end of the circumscribed contour is obtained, and the contour is cut according to the end corresponding to the maximum value of the distance and in accordance with the square contour.

5. The method for similarity search of engineering drawings according to claim 3 or 4, characterized in that: The step of obtaining a sub-image group comprises: When the number of the views of the engineering drawing is greater than or equal to the preset number, filtering the sub-image groups so that the number of the sub-image groups is equal to the preset number; When the number of the views of the engineering drawing is less than the preset number, the sub-image group is returned.

6. The method for similarity search of engineering drawings according to claim 3, characterized in that: The engineering drawing is provided with extractable text, and before the searching step, the step of obtaining the extractable text is also included: Removing the annotation of the engineering drawing according to the mask to obtain the technical requirement text and the drawing annotation text of the engineering drawing; The extractable text in the technical requirement text and the drawing annotation text is extracted by using a named entity recognition model.

7. The method for similarity search of engineering drawings according to claim 6, characterized in that: Before the searching step, the method further includes: For the sub-image group, vectorization is performed based on the image processing model to obtain a view vector group; For the extractable text, vectorization is performed based on the image processing model to obtain an extractable text vector; The view vector group and the extractable text vector are merged to obtain a target vector group, where the target vector group includes the first target vector group corresponding to the target drawing and the second target vector group corresponding to each of the candidate drawings.

8. The method for similarity search of engineering drawings according to claim 7, characterized in that: Before the search step, the method also includes a library building step: Storing the candidate drawings in a database; Creating a hash index for each of the candidate drawings in the database, and making a one-to-one correspondence between the candidate drawings and the corresponding second target vector groups; The view vector group and the extractable text vectors in the second target vector group are stored in columns.

9. The method for similarity search of engineering drawings according to claim 8, characterized in that: The similarity value includes a view similarity value and a text similarity value, The searching step comprises: Traversing the view vector group of the first target vector group by column, matching it with each view vector group of the second target vector group one by one and calculating similarity, so as to obtain the view similarity value; Calculating the text similarity value between the extractable text vectors of the first target vector group and the extractable text vectors of the second target vector group; The candidate drawings are sorted according to the view similarity value and the text similarity value, and the candidate drawings sorted within a preset similarity range are obtained and returned.

10. A device for similarity search of engineering drawings, characterized in that: include: a preprocessing module, configured to remove annotations in the engineering drawings and retain structural lines in the engineering drawings, wherein the engineering drawings include target drawings and candidate drawings, the target drawings include a plurality of mutually corresponding views corresponding to an industrial product, each of the candidate drawings includes a plurality of mutually corresponding views corresponding to an industrial product, and each of the views corresponds to a vector; An acquisition module, used for respectively extracting the plurality of views in the engineering drawing obtained by the preprocessing module to obtain a sub-image group; A training module, configured to train the sub-image group of the engineering drawing using an image processing model to obtain a first target vector group corresponding to the target drawing and a second target vector group corresponding to each of the candidate drawings; A search module is used to perform a mixed search based on the first target vector group of the target drawing and the second target vector group of each candidate drawing to obtain a similarity value, obtain the candidate drawings within a preset similarity value range, and return them.

Images