Primitive identification method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of computer-aided design, in particular to a graph element recognition method and device, electronic equipment and a storage medium, the method comprising the following steps: acquiring a to-be-recognized drawing to determine attribute information of graph elements in the to-be-recognized drawing; determining node features of the graph elements based on the attribute information of the graph elements; determining edge features between the graph elements based on position relationships between any two graph elements with a connection relationship; constructing graph structure data of the to-be-recognized drawing based on the node features and the edge features; and determining a graph element category of the graph elements in the to-be-recognized drawing according to the graph structure data and a graph neural network used for graph element category recognition. The graph elements in the to-be-recognized drawing are taken as nodes, and the connection between two graph elements is taken as an edge to construct the graph structure data, the graph element category of the graph elements is determined from the overall features of the to-be-recognized drawing without grouping, and the accuracy of the graph element category recognition is improved.

Description

Technical Field

[0001] This invention relates to the field of computer-aided design technology, and more specifically to methods, devices, electronic devices, and storage media for primitive recognition. Background Technology

[0002] For the identification of graphic elements in CAD drawings, some methods group the graphic element information, classifying a complete drawing into many groups and then identifying each group; others are based on object detection methods, cutting the drawing into segments. Both of these methods, due to the grouping, only recognize elements within a group, which only considers local features of the drawing and makes it difficult to guarantee the accuracy of the recognition results. Furthermore, object detection methods require a preset detection bounding box size; for drawings exceeding this size, the accuracy drops significantly. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method, apparatus, electronic device, and storage medium for primitive recognition, in order to solve the problem of low accuracy in primitive recognition.

[0004] According to a first aspect, embodiments of the present invention provide a primitive recognition method, including:

[0005] Obtain the drawing to be identified in order to determine the attribute information of the graphic elements in the drawing;

[0006] The node features of the graphic element are determined based on its attribute information;

[0007] Determine the edge features between any two connected primitives based on their positional relationship.

[0008] Based on the node features and the edge features, the graph structure data of the drawing to be identified is constructed;

[0009] Based on the graph structure data and the graph neural network used for primitive category recognition, the primitive category of the primitive in the drawing to be recognized is determined.

[0010] The graphic element recognition method provided in this embodiment of the invention uses graphic elements in the drawing to be recognized as nodes and the connection between any two graphic elements with a connection relationship as edges to construct graphic structure data. Starting from the overall features of the drawing to be recognized, the graphic element category can be determined without grouping, thus improving the accuracy of graphic element category recognition.

[0011] In some implementations, determining the primitive category of the primitive in the drawing to be identified based on the graph structure data and a graph neural network for primitive category recognition includes:

[0012] The graph structure data is input into the graph neural network to determine the first predicted category of the graph primitives, and the second predicted category of the edges between the graph primitives and the second probability of the second predicted category;

[0013] Based on the second predicted category and the second probability of the edge, the first predicted category of the two primitives connected by the edge is corrected to determine the primitive category of the primitive.

[0014] The graph element recognition method provided in this embodiment of the invention combines edge category with edge category when predicting graph element category of graph element node. By using edge classification to assist in graph node recognition, the method makes full use of point-edge interaction features and ensures the accuracy of graph element category recognition results.

[0015] In some implementations, the graph neural network is further used to determine a first probability of the first predicted category, and the step of correcting the first predicted category of two primitives connected by the edge based on the second predicted category of the edge and the second probability to determine the primitive category of the primitive includes:

[0016] Based on the magnitude of the first probability of the two primitives connected by the edge and the magnitude of the second probability of the edge, determine the first target category corresponding to the maximum first probability of the two primitives and the second target category corresponding to the maximum second probability of the edge.

[0017] When the first target category is the same as the second target category, and there is a target graph element with a maximum first probability less than the probability threshold and a maximum second probability greater than the probability threshold among the two graph elements, the first target category of the target graph element is corrected based on the second target category of the edge to determine the graph element category.

[0018] The primitive recognition method provided in this invention uses the second target category of the edge to correct the first target category of the target primitive, thereby reducing the number of cases where the primitive category of the target primitive cannot be identified while ensuring the accuracy of the recognition results, thus improving the accuracy and efficiency of primitive recognition.

[0019] In some implementations, the determination of the graph neural network includes:

[0020] Obtain sample graph structure data, which includes a first category label for sample graph nodes and a second category label for sample edges. A sample edge is a connection edge between any two sample graph elements that have a connection relationship. When the first category labels of the two sample graph elements connected by the sample edge are the same, the second category label of the sample edge is a first preset value. When the first category labels of the two sample graph elements connected by the sample edge are different, the second category label of the sample edge is a second preset value.

[0021] A preset graph neural network is trained based on the sample graph structure data, and the graph neural network is determined.

[0022] The primitive recognition method provided in this invention can enhance the cohesion of labels for the same primitive category by annotating sample edges, and improve the accuracy of node classification by training an auxiliary task for edge label classification. Furthermore, since graph neural networks learn and capture the geometric and topological relationships between primitives, they can effectively recognize primitives even after rotating, mirroring, stretching, scaling, or otherwise altering the shapes in the drawing, and even for designers with different styles.

[0023] In some implementations, determining the edge features between primitives based on the positional relationship between any two primitives with a connection relationship includes:

[0024] For each graphic element, the retrieval area for that graphic element is determined based on its position in the drawing to be identified;

[0025] Within the search area, the positional relationship between two primitives is calculated based on the coordinates of any two primitives that are connected, in order to determine the edge features between the primitives.

[0026] The primitive recognition method provided in this invention defines a retrieval area for each primitive and calculates edge features within the retrieval area, which facilitates the management of primitives whose edge features have been calculated, simplifies data processing, and improves the efficiency of edge feature calculation.

[0027] In some implementations, the coordinates of the primitives include start-point coordinates and end-point coordinates. Within the retrieval area, calculating the positional relationship between two primitives based on their coordinates, to determine the edge features between the primitives, includes:

[0028] Based on the starting coordinates and ending coordinates of the graphic element, determine the equation of the straight line connecting the starting coordinates and the ending coordinates;

[0029] By utilizing the positional relationship of the linear equations of any two connected primitives, the edge characteristics between the primitives are determined, including parallel, connected, intersecting, collinear, or coincident relationships.

[0030] The primitive recognition method provided in this invention represents the connecting line between any two primitives that are connected. It uses the starting coordinates and ending coordinates of the primitive itself to form a straight line equation, and uses the positional relationship of the straight line equation to represent the edge features, which simplifies the processing of edge features and improves the efficiency of primitive recognition.

[0031] In some implementations, determining the node features of the primitive based on its attribute information includes:

[0032] Information about preset attributes in the primitives is obtained to form feature vectors of the preset attributes;

[0033] The feature vectors of all the preset attributes are concatenated to determine the node features of the primitive.

[0034] The graphic element recognition method provided in this embodiment of the invention can reduce the determination of node features and improve the efficiency of graphic element category recognition because the information of the preset attributes can be directly parsed from the drawing to be recognized.

[0035] According to a second aspect, embodiments of the present invention provide a graphic element recognition device, comprising:

[0036] The acquisition module is used to acquire the drawing to be identified in order to determine the attribute information of the graphic elements in the drawing to be identified;

[0037] The first determining module is used to determine the node features of the graphic element based on the attribute information of the graphic element;

[0038] The second determining module is used to determine the edge features between graphic elements based on the positional relationship between any two graphic elements with a connection relationship;

[0039] A construction module is used to construct the graph structure data of the drawing to be identified based on the node features and the edge features;

[0040] The identification module is used to determine the primitive category of the primitive in the drawing to be identified based on the graph structure data and the graph neural network used for primitive category identification.

[0041] According to a third aspect, embodiments of the present invention provide an electronic device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the primitive recognition method described in the first aspect or any embodiment of the first aspect.

[0042] According to a fourth aspect, embodiments of the present invention provide a computer-readable storage medium storing computer instructions for causing the computer to perform the primitive recognition method described in the first aspect or any embodiment of the first aspect.

[0043] It should be noted that the corresponding beneficial effects of the graphic element recognition device, electronic device, and computer-readable storage medium provided in the embodiments of the present invention can be found in the description of the corresponding beneficial effects of the graphic element recognition method above, and will not be repeated here. Attached Figure Description

[0044] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is a flowchart of a primitive recognition method according to an embodiment of the present invention;

[0046] Figure 2 This is a flowchart of a primitive recognition method according to an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram illustrating the determination of element categories according to an embodiment of the present invention;

[0048] Figure 4 This is a flowchart of a primitive recognition method according to an embodiment of the present invention;

[0049] Figure 5 This is a flowchart of a primitive recognition method according to an embodiment of the present invention;

[0050] Figure 6 This is a structural block diagram of a graphic element recognition device according to an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] The primitive recognition method provided in this invention is based on the geometric topological relationships in the drawing to be recognized. By abstracting the primitives in the drawing as nodes, the connection relationships between primitives are calculated based on the attribute information of each primitive. These connection relationships are used to represent edge features, thereby constructing graph structure data. Then, by utilizing the interaction of point and edge features in a graph neural network, the geometric topological features between primitives are automatically learned. Based on the graph neural network, the primitive category is predicted in the graph structure data, thereby identifying the primitive category of each primitive in the drawing to be recognized.

[0054] This graphic element recognition method eliminates the need for grouping the drawing to be recognized; instead, it identifies the entire drawing. Whether it's point-based elements like doors and beds or line-based elements like walls, it can directly recognize the graphic elements and effectively extract their geometric and topological features. It boasts high recognition accuracy for drawings of different types, designers, and sizes.

[0055] The graphic element recognition method provided in this embodiment of the invention can be applied to design terminals, such as computers. Users design drawings to be recognized through interactive operations on the design terminal, or import the drawings to be recognized into the design terminal through other means. The design terminal then executes the graphic element recognition method of this embodiment to identify the graphic element categories in the drawings, obtaining the graphic element category of each element. This method can also be applied to cloud collaboration platforms. After users complete the design of the drawings to be recognized on their respective design terminals, they upload the drawings to the cloud collaboration platform. Accordingly, the cloud collaboration platform determines the graphic element category of each graphic element in the drawings by executing the graphic element recognition method of this embodiment. Since the drawings to be recognized are uploaded to the cloud collaboration platform, other users can also share the drawings. That is, when other users have a need for graphic element recognition, they can trigger the cloud collaboration platform to perform graphic element recognition on the drawings to obtain the graphic element categories. Of course, the above are only some optional application scenarios of this embodiment of the invention, and the scope of protection of this invention is not limited thereto. Specific settings can be made according to actual needs.

[0056] According to an embodiment of the present invention, a method for recognizing graphic elements is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0057] This embodiment provides a primitive recognition method that can be used in electronic devices such as computers, mobile terminals, and servers. Figure 1 This is a flowchart of a primitive recognition method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0058] S11, Obtain the drawing to be identified in order to determine the attribute information of the graphic elements in the drawing to be identified.

[0059] When modeling the drawing to be identified, the drawing is abstracted into graph feature data. That is, each graphic element in the drawing is abstracted into a graph node, and the relationships between graphic elements are parallelism, intersection, connection, collinearity, overlap, etc. The relationships between graphic elements are represented by the edge features connecting the graphic elements. Based on this, graphic elements are represented by graphic element nodes, and the relationships between graphic elements are represented by edge features, thus obtaining the graph feature data of the drawing to be identified.

[0060] The drawing to be identified can be a CAD drawing, a BIM drawing, etc., and there are no restrictions on its type. After obtaining the drawing, the electronic device parses it to obtain the attribute information of each element in the drawing. This attribute information includes, but is not limited to, element type, node degree, node layer degree, sine or cosine value of the angle between the node and the X-axis of the drawing, etc.

[0061] Specifically, primitive types include line segments, arcs, ellipses, text, etc., with a vector type of One-Hot; node degree: the number of primitives connected to each node, measuring the richness of surrounding primitives, with a numerical vector type; node layer self-degree: the number of primitives of the same type as itself among the primitives connected to each node, measuring the similarity between surrounding primitives and itself, with a numerical vector type; sine and cosine values ​​of the angle between the node and the X-axis: measuring the position and direction of the primitive relative to the X-axis, with a numerical vector type.

[0062] If the drawing to be identified is a CAD drawing, the electronic device can input the CAD drawing into the dwg2json tool and output the attribute information of each element in the CAD drawing.

[0063] S12, Determine the node features of the graphic elements based on their attribute information.

[0064] As mentioned above, primitives are represented through primitive nodes, and the features of primitive nodes are represented by the attribute information of the primitives. Based on this, for each primitive, the node features of the primitive are represented using the primitive attribute information.

[0065] Since a graphic element has multiple attribute information, it's possible to predetermine which attribute(s) will be used to represent its node features. For example, if a graphic element has attributes 1-6, and attributes 1-3 are chosen to represent its node features, then the node features of graphic element 1 are represented by attribute information for attribute 1, attribute information for attribute 2, and attribute information for attribute 3. To facilitate data processing, attribute information for attributes 1-3 is represented as vectors. Therefore, the node features of graphic element 1 are the concatenation of vector 1 for attribute 1, vector 2 for attribute 2, and vector 3 for attribute 3. Similarly, through this process, the node features of each graphic element in the drawing to be identified can be obtained.

[0066] S13. Determine the edge features between any two primitives that are connected, based on their positional relationship.

[0067] The positions of graphic elements in the drawing to be identified are fixed, and the positional relationship between any two connected graphic elements is also fixed. Whether any two connected graphic elements are connected can be distinguished by the connecting lines between them. This positional relationship can be calculated using the coordinates of the graphic elements in the drawing, or by performing bounding box calculations on the graphic elements and then comparing the positional relationships of any two bounding boxes to determine the edge features between the graphic elements. Edge features include, but are not limited to, parallelism, intersection, connection, and coincidence.

[0068] Taking bounding box calculation as an example, since the position and boundary of the primitive are determined, the bounding box is calculated for the boundary of the primitive to obtain the bounding box of the primitive; then, by calculating the intersection-union ratio of two bounding boxes, it is determined whether the two bounding boxes intersect or coincide; then, the edges of the two bounding boxes are compared to determine whether the two bounding boxes are parallel; and so on.

[0069] Of course, other methods can also be used to determine the edge features between primitives, and no restrictions are imposed on them here.

[0070] S14. Based on node features and edge features, construct the graph structure data of the drawing to be identified.

[0071] After obtaining the node features and edge features, graph structure data can be constructed. For example, the number of graphic elements in the drawing to be identified is obtained, and a corresponding number of graphic element nodes are generated; then, based on the connection relationship of the graphic elements, the graphic element nodes with connection relationship are connected using edges; finally, the node features of each graphic element and the edge features between graphic elements are associated with the corresponding graphic element nodes and edges, thereby generating graph structure data.

[0072] The connections between graph primitives are represented by an adjacency matrix, the order of which is equal to the number of nodes. If an edge exists between two nodes, the element at that position is 1; otherwise, it is 0. The adjacency matrix measures the connectivity between graph nodes. The graph structure data of the drawing to be identified is then constructed based on the adjacency matrix between the primitives.

[0073] S15. Based on the graph structure data and the graph neural network used for primitive category recognition, determine the primitive category of the primitive in the drawing to be recognized.

[0074] The input to a graph neural network is the graph structure data of a drawing, and the output is the primitive category of each primitive in the drawing, or the probability of each primitive belonging to each primitive category, etc. There are no specific limitations on its exact form; it can be set according to actual needs. During inference, the graph neural network performs joint prediction of node and edge classification by capturing the interaction of node and edge information and the topological features of the graph network data through graph network layers.

[0075] The choice of graph neural network allows for the selection of different graph network layers for feature extraction, such as Graph Convolutional Neural Networks (GCN), Graph Isomorphic Networks (GIN), and Graph Transform Networks (GTN). No specific limitations are placed on the specific structure of the graph neural network; it can be configured according to actual needs.

[0076] The graphic element recognition method provided in this embodiment uses graphic elements in the drawing to be recognized as nodes and the connection between any two graphic elements with a connection relationship as edges to construct graphic structure data. Starting from the overall features of the drawing to be recognized, the graphic element category can be determined without grouping, thus improving the accuracy of graphic element category recognition.

[0077] This embodiment provides a primitive recognition method that can be used in electronic devices such as computers, mobile terminals, and servers. Figure 2 This is a flowchart of a primitive recognition method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0078] S21, Obtain the drawing to be identified in order to determine the attribute information of the graphic elements in the drawing to be identified.

[0079] Please see details Figure 1 S11 of the illustrated embodiment will not be described again here.

[0080] S22, Determine the node features of the primitives based on their attribute information.

[0081] Please see details Figure 1 S12 of the illustrated embodiment will not be described again here.

[0082] S23. Determine the edge features between any two primitives that are connected, based on their positional relationship.

[0083] Please see details Figure 1 S13 of the illustrated embodiment will not be described again here.

[0084] S24. Based on node features and edge features, construct the graph structure data of the drawing to be identified.

[0085] Please see details Figure 1 S14 of the illustrated embodiment will not be described again here.

[0086] S25. Based on the graph structure data and the graph neural network used for primitive category recognition, determine the primitive category of the primitive in the drawing to be recognized.

[0087] Specifically, S25 includes:

[0088] S251, Input the graph structure data into the graph neural network to determine the first predicted category of the graph primitives, and the second predicted category and the second probability of the edge between the graph primitives.

[0089] The graphic element categories include, but are not limited to, ordinary door elements, ordinary wall elements, louver elements, wall elements, sliding door elements, etc.; of course, graphic element categories can also be added as needed, such as stairs, elevators, kitchenware, etc.

[0090] After the graph structure data is input into the graph neural network, the graph neural network processes it to obtain the first predicted category of the graph primitives, and the second predicted category and probability of the edges between the graph primitives.

[0091] In some implementations, the graph neural network is determined in the following ways:

[0092] (1) Obtain sample graph structure data, which includes a first category label for sample graph nodes and a second category label for sample edges. A sample edge is a connection edge between any two sample graph elements that have a connection relationship. When the first category labels of the two sample graph elements connected by the sample edge are the same, the second category label of the sample edge is a first preset value; when the first category labels of the two sample graph elements connected by the sample edge are different, the second category label of the sample edge is a second preset value.

[0093] (2) Train a preset graph neural network based on sample graph structure data to determine the graph neural network.

[0094] After training the pre-defined graph neural network, sample data needs to be collected and labeled. Specifically, sample graph structure data is collected, and the first category label of the sample graph nodes and the second category label of the sample edges in the sample graph structure data are labeled. The first category label includes, but is not limited to, ordinary door primitives, ordinary window primitives, louver primitives, wall primitives, sliding door primitives, etc. The description of the second category label is as follows: if the first category labels of the sample primitives at both ends of the sample edge are the same, then the second category label of the sample edge is the same as the first category label of the sample primitive; if the first category labels of the sample primitives at both ends are different, then the second category label of the sample edge is 0.

[0095] The preset graph neural network is then trained using sample graph structure data, and its parameters are updated. After multiple iterations, the final graph neural network is determined. This graph neural network is used to identify the primitive categories of the drawing paper.

[0096] Labeling sample edges can strengthen the cohesion of labels for the same primitive category, and can also improve node classification accuracy by training an auxiliary task for edge label classification. Furthermore, since graph neural networks learn and capture the geometric and topological relationships between primitives, they can effectively identify graphics in drawings that have undergone different processing such as rotation, mirroring, stretching, and scaling, and even those designed by designers of different styles.

[0097] S252, based on the second predicted category and second probability of the edge, correct the first predicted category of the two primitives connected by the edge to determine the primitive category of the primitive.

[0098] Graph-structured networks can output not only the first predicted category of a primitive but also the second predicted category and second probability of the edges connecting the primitives. As mentioned above, the second category label of a sample edge is related to the first category labels of the two primitives connected to the sample edge. Based on this, during the inference phase, the second predicted category of the edge is also related to the first predicted categories of the two connected primitives. Therefore, the second predicted category and second probability of the edge can be used to correct the first predicted category of the two primitives connected to the edge, thereby improving the accuracy of primitive category recognition.

[0099] In some implementations, the graph neural network is further used to determine a first probability of the first predicted category. Based on this, S252 above includes:

[0100] (1) Based on the magnitude of the first probability of the two graph elements connected by the edge and the magnitude of the second probability of the edge, determine the first target category corresponding to the maximum first probability of the two graph elements and the second target category corresponding to the maximum second probability of the edge.

[0101] (2) When the first target category and the second target category are the same, and there is a target element with a maximum first probability less than the probability threshold and a maximum second probability greater than the probability threshold, the first target category of the target element is corrected based on the second target category of the edge to determine the element category of the element.

[0102] The electronic device is equipped with a logic for determining the category of a graphic element. If the first probability of the first predicted category is greater than the probability threshold, then the first predicted category is determined as the first target category of the graphic element. If the first probability of the first predicted category is less than the probability threshold, then it is necessary to combine the second predicted category of the connected edge for correction in order to further determine the first target category of the graphic element.

[0103] For graph structure models, if multiple first predicted categories and their first probabilities are output, the first predicted category with the highest first probability is determined as the first target category by comparing the magnitudes of the first probabilities; similarly, the second predicted category with the highest second probability is determined as the second target category.

[0104] like Figure 3 As shown, with a probability threshold of 0.6, the probabilities of adjacent graph element nodes A and B being gate elements are predicted to be 0.9 and 0.5, respectively. However, the probability of the edge between A and B being a gate label is 0.8. Since the probability of graph element node B being a gate label is 0.5, which is less than 0.6, it is necessary to correct for the error by considering the edge between A and B. At this point, by combining the edge label probabilities with the probability of node A, the final classification of node B is determined to be a gate. Therefore, the accuracy of the node classification label is corrected by the numerical value of the edge classification probability.

[0105] By using the second target category of the edge to correct the first target category of the target primitive, the number of primitive categories that cannot be identified is reduced while ensuring the accuracy of the recognition results, thereby improving the accuracy and efficiency of primitive recognition.

[0106] The graph element recognition method provided in this embodiment combines edge categories when predicting the graph element category of a graph element node. By using edge classification to assist in the recognition of graph nodes, it makes full use of the point-edge interaction features and ensures the accuracy of the graph element category recognition results.

[0107] This embodiment provides a primitive recognition method that can be used in electronic devices such as computers, mobile terminals, and servers. Figure 4 This is a flowchart of a primitive recognition method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0108] S31, Obtain the drawing to be identified in order to determine the attribute information of the graphic elements in the drawing to be identified.

[0109] Please see details Figure 1 S11 of the illustrated embodiment will not be described again here.

[0110] S32, Determine the node features of the graphic elements based on their attribute information.

[0111] Specifically, S32 includes:

[0112] S321, Obtain information about preset attributes in the primitives to form feature vectors of the preset attributes.

[0113] As mentioned above, a primitive may include multiple attributes. By defining which attribute information is used to generate the node features of the primitive, these attributes are called preset attributes. The feature vector of the preset attributes is the vector representation of the preset attribute information.

[0114] S322: Concatenate the feature vectors of all preset attributes to determine the node features of the primitive.

[0115] For the same primitive, there may be multiple preset attributes. The feature vectors of these multiple preset attributes are concatenated to obtain the node features of the primitive.

[0116] S33, determine the edge features between any two connected primitives based on their positional relationship.

[0117] Specifically, S33 includes:

[0118] S331, for each graphic element, determine the retrieval area of ​​the graphic element based on its position in the drawing to be identified.

[0119] The search area for each graphic element is its bounding box. The length and width of the bounding box are the length of the graphic element plus a fixed length. The fixed lengths of the long and short sides of the bounding box are set based on empirical values, or other methods can be used; no restrictions are placed on them here. When selecting the fixed lengths, it is necessary to consider saving time and computational complexity in the next step of calculating the positional relationships of the graphic elements.

[0120] S332, within the retrieval area, calculate the positional relationship between two primitives based on the coordinates of any two primitives with a connection relationship, so as to determine the edge features between the primitives.

[0121] For a search area, there are multiple graphic elements, some of which may be connected, while others may not. In this case, the positional relationship between the connected graphic elements is calculated based on their coordinates. The coordinates of the graphic elements include start and end coordinates. Therefore, step S332 includes:

[0122] (1) Based on the starting coordinates and ending coordinates of the primitive, determine the equation of the straight line connecting the starting coordinates and the ending coordinates.

[0123] (2) Using the positional relationship of the linear equations of any two primitives that are connected, determine the edge features between the primitives. The edge features include parallel, connected, intersecting, collinear or coincident relationships.

[0124] Calculate the equations of the lines connecting the origin and destination points of the primitives. Then, using the relationship between any two line equations in the neighborhood of the primitives, determine the edge characteristics between them. These edge characteristics include, but are not limited to, parallelism, connection, intersection, collinearity, and coincidence. Finally, define these calculated edge characteristics as edge features. For ease of calculation, represent the edge features in vector form to obtain the edge feature vector.

[0125] Since an edge represents the connecting line between any two primitives that are connected, the edge features are represented by using the starting coordinates and ending coordinates of the primitives to form a straight line equation. The edge features are represented by the positional relationship of the straight line equation, which simplifies the processing of edge features and improves the efficiency of primitive recognition.

[0126] S34. Based on node features and edge features, construct the graph structure data of the drawing to be identified.

[0127] Please see details Figure 1 S14 of the illustrated embodiment will not be described again here.

[0128] S35, Based on the graph structure data and the graph neural network used for primitive category recognition, determine the primitive category of the primitive in the drawing to be recognized.

[0129] Please see details Figure 2 S25 of the illustrated embodiment will not be described again here.

[0130] The graphic element recognition method provided in this embodiment can directly parse the preset attribute information from the drawing to be recognized. Therefore, by using the preset attribute information to form a feature vector of preset attributes, the determination of node features can be reduced, thereby improving the efficiency of graphic element category recognition. Defining a retrieval area for each graphic element and calculating edge features within the retrieval area facilitates the management of graphic elements whose edge features have already been calculated, simplifies data processing, and improves the efficiency of edge feature calculation.

[0131] As an optional implementation of the present invention, such as Figure 5As shown, the method includes inputting graph structure data (referred to as graph data) into a graph neural network, which includes a preprocessing network layer, a graph neural network layer, a node classification task, and an edge classification task. Specifically, after the graph data is input, the node feature vectors are processed using a fully connected node layer, and the edge feature vectors are processed using a fully connected edge layer. The results of these two processing steps are input into a feature aggregation network layer, and after processing by the feature aggregation network layer, they are input into the graph neural network layer. The output of the graph neural network layer is input into the node classification network layer and the edge classification network layer, respectively. The node classification network layer is used to predict node labels, and the edge classification network layer is used to predict edge labels. The electronic device then trains and aggregates the output of the graph neural network to determine the primitive category of each primitive.

[0132] This primitive recognition method is WYSIWYG: it directly models the drawing based on predefined point and edge relationships, resulting in strong interpretability; it provides global learning of drawing information: by modeling and learning all the information in the drawing, it can obtain the mutual influence between overall features, resulting in more comprehensive feature learning and the ability to predict the primitive category of the entire drawing; it automatically extracts features: based on the modeled graph data, it performs graph neural network node classification and edge classification training, automatically learning point, edge interaction features, and geometric topological shape features, thus building a powerful primitive feature extractor without the need for further manual rule formulation; and it has low annotation costs. For the graphic element categories to be identified, when labeling data, different graphic element categories only need to be placed in different layers (door graphic elements are placed in the door layer, window graphic elements are placed in the window layer). Each layer is independent of the others and does not affect each other, making labeling simple and efficient; it has high accuracy, strong reusability, and strong generalization: it only requires predefining point and edge relationships, constructing graph network data, and learning and training through graph neural networks, so that graphic element classification has high accuracy; because the model learns to capture the geometric topological relationships between graphic elements, it can effectively identify even some graphics in the drawing that have been rotated, mirrored, stretched, scaled, etc., and even for designers with different styles.

[0133] This embodiment also provides a primitive recognition device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0134] This embodiment provides a graphic element recognition device, such as... Figure 6 As shown, it includes:

[0135] The acquisition module 41 is used to acquire the drawing to be identified in order to determine the attribute information of the graphic elements in the drawing to be identified;

[0136] The first determining module 42 is used to determine the node features of the graphic element based on the attribute information of the graphic element;

[0137] The second determining module 43 is used to determine the edge features between graphic elements based on the positional relationship between any two graphic elements with a connection relationship;

[0138] Construction module 44 is used to construct the graph structure data of the drawing to be identified based on the node features and the edge features;

[0139] The identification module 45 is used to determine the primitive category of the primitive in the drawing to be identified based on the graph structure data and the graph neural network used for primitive category identification.

[0140] In some implementations, the identification module 45 includes:

[0141] An input unit is configured to input the graph structure data into the graph neural network to determine a first predicted category of the graph primitives, and a second predicted category of the edges between the graph primitives and a second probability of the second predicted category;

[0142] A correction unit is used to correct the first prediction category of the two primitives connected by the edge based on the second prediction category and the second probability of the edge, and to determine the primitive category of the primitive.

[0143] In some implementations, the graph neural network is further used to determine a first probability of the first predicted category, and the correction unit includes:

[0144] The first determining subunit is used to determine the first target category corresponding to the maximum first probability of the two graph elements connected by the edge and the second target category corresponding to the maximum second probability of the edge, based on the magnitude of the first probability of the two graph elements connected by the edge and the magnitude of the second probability of the edge.

[0145] The second determining subunit is used to correct the first target category of the target element based on the second target category of the edge when the first target category is the same as the second target category, and there is a target element with a maximum first probability less than a probability threshold and a maximum second probability greater than the probability threshold among the two elements, so as to determine the element category of the element.

[0146] In some implementations, the determination module of the graph neural network includes:

[0147] The first acquisition unit is used to acquire sample graph structure data, which includes a first category label of sample graph nodes and a second category label of sample edges. The sample edge is the connection edge between any two sample graph elements that have a connection relationship. When the first category labels of the two sample graph elements connected by the sample edge are the same, the second category label of the sample edge is a first preset value. When the first category labels of the two sample graph elements connected by the sample edge are different, the second category label of the sample edge is a second preset value.

[0148] The training unit is used to train a preset graph neural network based on the sample graph structure data and to determine the graph neural network.

[0149] In some implementations, the second determining module 43 includes:

[0150] The third determining subunit is used to determine the retrieval area of ​​each graphic element based on its position in the drawing to be identified.

[0151] The fourth determining subunit is used to calculate the positional relationship between two graphic elements based on the coordinates of any two graphic elements with a connection relationship within the retrieval area, so as to determine the edge features between the graphic elements.

[0152] In some implementations, the coordinates of the graphic element include start-point coordinates and end-point coordinates, and the fourth determining subunit includes:

[0153] The fifth determining subunit is used to determine the equation of the straight line connecting the starting coordinates and the ending coordinates of the graphic element based on the starting coordinates and the ending coordinates of the graphic element.

[0154] The sixth determining subunit is used to determine the edge features between the graphic elements by utilizing the positional relationship of the linear equations of any two graphic elements with a connection relationship. The edge features include parallel, connected, intersecting, collinear or coincident relationships.

[0155] In some implementations, the first determining module 42 includes:

[0156] The second acquisition unit is used to acquire information about preset attributes in the primitive to form a feature vector of the preset attributes;

[0157] The splicing unit is used to splice the feature vectors of all the preset attributes to determine the node features of the primitive.

[0158] In this embodiment, the graphic element recognition device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0159] Further functional descriptions of the above modules are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0160] This invention also provides an electronic device having the above-described features. Figure 6 The graphic element recognition device shown.

[0161] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an optional embodiment of the present invention, such as... Figure 7 As shown, the electronic device may include: at least one processor 51, such as a CPU (Central Processing Unit), at least one communication interface 53, memory 54, and at least one communication bus 52. The communication bus 52 is used to enable communication between these components. The communication interface 53 may include a display screen or a keyboard; optionally, the communication interface 53 may also include a standard wired interface or a wireless interface. The memory 54 may be high-speed RAM (Random Access Memory) or non-volatile memory, such as at least one disk storage device. Optionally, the memory 54 may also be at least one storage device located remotely from the aforementioned processor 51. The processor 51 may be combined with... Figure 6 The described apparatus has an application program stored in memory 54, and the processor 51 calls the program code stored in memory 54 to perform any of the above method steps.

[0162] The communication bus 52 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 52 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0163] The memory 54 may include volatile memory, such as random-access memory (RAM); the memory may also include non-volatile memory, such as flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory 54 may also include a combination of the above types of memory.

[0164] The processor 51 can be a central processing unit (CPU), a network processor (NP), or a combination of CPU and NP.

[0165] The processor 51 may further include a hardware chip. This hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0166] Optionally, the memory 54 is also used to store program instructions. The processor 51 can invoke the program instructions to implement the primitive recognition method as shown in any embodiment of this application.

[0167] This invention also provides a non-transitory computer storage medium storing computer-executable instructions that can execute the primitive recognition method in any of the above method embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0168] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A primitive recognition method, characterized in that, include: Obtain the drawing to be identified in order to determine the attribute information of the graphic elements in the drawing; The node features of the graphic element are determined based on its attribute information; Determine the edge features between any two connected primitives based on their positional relationship. Based on the node features and the edge features, the graph structure data of the drawing to be identified is constructed; Based on the graph structure data and the graph neural network used for primitive category recognition, the primitive category of the primitive in the drawing to be identified is determined; The step of determining the primitive category of the primitive in the drawing to be identified based on the graph structure data and the graph neural network used for primitive category recognition includes: The graph structure data is input into the graph neural network to determine the first predicted category and the first probability of the first predicted category of the graph primitives, and the second predicted category and the second probability of the edge between the graph primitives. Based on the magnitude of the first probability of the two primitives connected by the edge and the magnitude of the second probability of the edge, determine the first target category corresponding to the maximum first probability of the two primitives and the second target category corresponding to the maximum second probability of the edge. When the first target category is the same as the second target category, and there is a target graph element with a maximum first probability less than the probability threshold and a maximum second probability greater than the probability threshold among the two graph elements, the first target category of the target graph element is corrected based on the second target category of the edge to determine the graph element category.

2. The method according to claim 1, characterized in that, The method for determining the graph neural network includes: Obtain sample graph structure data, which includes a first category label for sample graph nodes and a second category label for sample edges. A sample edge is a connection edge between any two sample graph elements that have a connection relationship. When the first category labels of the two sample graph elements connected by the sample edge are the same, the second category label of the sample edge is a first preset value. When the first category labels of the two sample graph elements connected by the sample edge are different, the second category label of the sample edge is a second preset value. A preset graph neural network is trained based on the sample graph structure data, and the graph neural network is determined.

3. The method according to claim 1, characterized in that, The determination of edge features between primitives based on their positional relationship includes: For each graphic element, the retrieval area for that graphic element is determined based on its position in the drawing to be identified; Within the search area, the positional relationship between two primitives is calculated based on the coordinates of any two primitives that are connected, in order to determine the edge features between the primitives.

4. The method according to claim 3, characterized in that, The coordinates of the graphic element include start coordinates and end coordinates. Within the search area, the positional relationship between two graphic elements is calculated based on their coordinates to determine the edge features between them, including: Based on the starting coordinates and ending coordinates of the graphic element, determine the equation of the straight line connecting the starting coordinates and the ending coordinates; By utilizing the positional relationship of the linear equations of any two connected primitives, the edge characteristics between the primitives are determined, including parallel, connected, intersecting, collinear, or coincident relationships.

5. The method according to claim 1, characterized in that, Determining the node features of a graph element based on its attribute information includes: Information about preset attributes in the primitives is obtained to form feature vectors of the preset attributes; The feature vectors of all the preset attributes are concatenated to determine the node features of the primitive.

6. A graphic element recognition device, characterized in that, include: The acquisition module is used to acquire the drawing to be identified in order to determine the attribute information of the graphic elements in the drawing to be identified; The first determining module is used to determine the node features of the graphic element based on the attribute information of the graphic element; The second determining module is used to determine the edge features between graphic elements based on the positional relationship between any two graphic elements with a connection relationship; A construction module is used to construct the graph structure data of the drawing to be identified based on the node features and the edge features; The identification module is used to determine the primitive category of the primitive in the drawing to be identified based on the graph structure data and the graph neural network used for primitive category identification; The step of determining the primitive category of the primitive in the drawing to be identified based on the graph structure data and the graph neural network used for primitive category recognition includes: The graph structure data is input into the graph neural network to determine the first predicted category and the first probability of the first predicted category of the graph primitives, and the second predicted category and the second probability of the edge between the graph primitives. Based on the magnitude of the first probability of the two primitives connected by the edge and the magnitude of the second probability of the edge, determine the first target category corresponding to the maximum first probability of the two primitives and the second target category corresponding to the maximum second probability of the edge. When the first target category is the same as the second target category, and there is a target graph element with a maximum first probability less than the probability threshold and a maximum second probability greater than the probability threshold among the two graph elements, the first target category of the target graph element is corrected based on the second target category of the edge to determine the graph element category.

7. An electronic device, characterized in that, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the primitive recognition method according to any one of claims 1-5 by executing the computer instructions.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the primitive recognition method according to any one of claims 1-5.

Images