Graph model identification method based on multiple modes

Abstract

The invention discloses a multi-modal-based graph model identification method, which comprises the following stages: (1) a preprocessing stage: preprocessing a training set; (2) a model training stage: inputting a training set to train an existing network model to obtain a common primitive recognition model, a character recognition model, a module division model and an anomaly analysis model; (3) an image recognition stage: completing primitive recognition, character recognition and wiring relation recognition for the to-be-recognized plant station wiring diagram; (4) a module division stage: obtaining a plurality of functional modules through a module division model; (5) a post-processing stage: completing a series of downstream tasks; and (6) an output stage: through analysis of the anomaly analysis model, an output module can output the CIM / SVG file meeting the standard. According to the method, accurate identification can be realized, obvious problems can be found and marked in time when the circuit of the plant station wiring diagram has the obvious problems, and the standard CIM / SVG file can be rapidly and accurately output.

Description

Technical Field

[0001] The present invention relates to the technical field of pattern recognition, and particularly to a multi-modal based pattern recognition method. Background Art

[0002] The visual display of an electrical system is usually realized by using SVG technology. Through SVG technology, each electrical device in the electrical system can be represented by graphical electrical primitives. In an electrical system, an electrical primitive represents the graph of an electrical device. Since there are a large number of electrical devices in the electrical system, the overall graph of the electrical system contains a large number of electrical primitives. Therefore, the design of electrical primitives is very important. When designing electrical primitives, the basic characteristics of the electrical device graphs should be analyzed and summarized, and a general graph display model should be designed to cooperate with the visual display of the electrical system. This model can also be called a primitive model.

[0003] The primitive model of an electrical device should include three parts: graphic attributes, application attributes, and interaction information. Graphic attributes are specifically composed of three parts: style, geometric information, and animation effects. Geometric information mainly reflects the shape, size, and position of the primitive. The style is mainly used to make the primitive display in a unified style. Animation effects mainly reflect the changes in the shape of the primitive, such as the opening and closing animations of switches. Through these three attributes, the graph of an electrical device can be accurately defined. Application attributes are mainly used to store CIM model information associated with the electrical device. In addition to the basic CIM model information, the primitive model should also include other additional information such as pointer information of the parent primitive, mapping relationship information of color and shape, and information on the way of scaling and rotation. Interaction events are mainly realized by defining a large number of interaction event scripts. The basis is that SVG technology supports embedding scripting languages. Scripting languages are mainly used to define event functions and triggers. Through triggers, event functions are triggered, and then the event functions change the SVG graph, thus realizing interaction events.

[0004] The graph model analysis is generally based on the CIM graph model. The overall solution of this analysis is divided into two major steps: the construction of document objects based on events and the encapsulation of data by operating documents. The method constructs an object of a custom document type (The E_Document) corresponding to the content of the file through the CIM / E file name. The implementation of this method depends on the method of the SAXReader type object. This class has a reference of the XMLReader type, and the method of the object pointed to by this reference completes the specific analysis operation of the underlying file. At the same time, the callback function in the callback processor is used to construct the custom document object. Existing graph model recognition methods can only parse the substation wiring diagram graph model as it is, but they cannot accurately recognize when graphic elements are tangent or overlapping, and cannot timely detect and warn when there are simple and obvious problems in the circuit of the substation wiring diagram. Therefore, there is an urgent need for a graph model recognition method that can accurately recognize when graphic elements are tangent or overlapping, timely detect and mark when there are obvious problems in the circuit of the substation wiring diagram, and can quickly and accurately output a CIM / SVG file that meets the standards. Summary of the Invention

[0005] To overcome the above-mentioned drawbacks of the prior art, the present invention proposes a multi-modal based graph model recognition method, which can accurately recognize when graphic elements are tangent or overlapping, timely detect and mark when there are obvious problems in the circuit of the substation wiring diagram, and can quickly and accurately output a CIM / SVG file that meets the standards.

[0006] The technical solution adopted by the present invention to solve its technical problems is: a multi-modal based graph model recognition method, including the following stages:

[0007] (1) Preprocessing stage, preprocess the training set;

[0008] (2) Model training stage, input the training set to train the existing network models respectively to obtain a general graphic element recognition model, a text recognition model, a module division model, and an anomaly analysis model; the general graphic element recognition module is used to recognize the types, rotation angles, and graphic element coordinates of the graphic elements in the substation wiring diagram; the text recognition model is used to extract all the text pictures in the substation wiring diagram and recognize the content of the text pictures; the module division model is used to divide the substation wiring diagram into several functional modules according to the graphic elements and wiring relationships extracted from the substation wiring diagram and combined with the wiring functions; the anomaly analysis model is used to perform anomaly analysis on the functional modules and store correction methods.

[0009] (3) Image recognition stage: For the substation wiring diagram to be recognized, complete graphic element recognition, text recognition, and wiring relationship recognition. Graphic element recognition: Complete graphic element recognition according to the general graphic element recognition model. Text recognition: Complete text recognition according to the text recognition model. Wiring relationship recognition: Combine the business rules of the power system and the text around the straight line to complete the recognition of buses and conventional lines and the recognition of the connection relationship between graphic elements.

[0010] (4) Module division stage: After completing step (3), obtain several functional modules through the module division model.

[0011] (5) Post-processing stage: On the basis of completing the graphic element recognition, text recognition, and wiring relationship recognition in step (3), complete a series of downstream tasks, including extraction of basic substation information, recognition of the working voltage of graphic elements, graphic element-text association matching, and completion of abbreviated text.

[0012] (6) Output stage: Through abnormal analysis model analysis, if there is no abnormal type in the abnormal analysis, directly pass through the output module; if the abnormal type is analyzed and there is a correction method in the abnormal analysis, correct the error of the substation wiring diagram through the correction method and then pass through the output module; if the abnormal type is analyzed but there is no correction method, then frame the functional module where the abnormality is located and pass through the output module; the output module can output a CIM / SVG file that meets the standard.

[0013] Further, the graphic elements include circuit breakers, isolating switches, earthing switches, two-winding transformers, three-winding transformers, capacitors, reactors, lines, and generators.

[0014] Further, the general graphic element recognition model adopts an overlapping windowing mechanism and the YOLO object detection algorithm, with CSPDarkNet53 as the feature extraction network.

[0015] Further, the text recognition model adopts the CRNN model; the text recognition model includes a convolutional neural network, a bidirectional recurrent neural network, and a temporal connection classification loss function; the convolutional neural network is used for feature extraction of the substation wiring diagram to obtain a feature map; the temporal connection classification loss function is used for prediction of feature vectors to obtain the probability distribution of each feature vector; according to the probability distribution of each feature vector, obtain the corresponding label sequence, and combine the sequence merging mechanism to output the final predicted text.

[0016] Further, in step (3), the wiring relationship recognition specifically includes wiring preprocessing, topological relationship detection, and post-processing.

[0017] Further, the wiring preprocessing includes the following steps:

[0018] (1) Image grayscale conversion processing: Perform grayscale conversion processing on the substation wiring diagram.

[0019] (2) Remove interference items: Whiten the interference items of the primitive, text, and the straight line parts outside the wiring diagram, and set the pixels within the given coordinate frame to zero.

[0020] (3) Straight line detection, classification, and intersection straight line cutting: Classify the detected straight lines and cut the intersection straight lines. The specific steps are as follows:

[0021] Step 1: Straight line detection: Use a straight line detection algorithm to detect straight lines in the processed substation wiring diagram, and store the coordinates of the two endpoints of the detected straight lines in a straight line list.

[0022] Step 2: Straight line classification: Classify the identified straight lines into vertical straight lines and horizontal straight lines according to the coordinate relationship at both ends of the straight line.

[0023] Step 3: Intersection straight line cutting: Cut the intersecting straight lines into independent straight line segments, and then identify the wiring relationship according to domain knowledge.

[0024] Furthermore, the topology relationship detection includes the following steps: Match the endpoints of all straight line segments with the primitives or the endpoints of other straight line segments. The incorporated rule is: Traverse all straight line segments, first try to match one of the endpoints of the straight line segment with the primitive. The matching rule is that if the distance between the endpoint of the straight line segment and the primitive is less than the set value, the matching is successful; if the matching fails, try to match with the endpoints of other straight line segments. The matching rule is that if the distance between a certain endpoint of one straight line segment and a certain endpoint of another straight line segment is less than the set value, the matching is successful; if both endpoints of a straight line segment can be matched with the endpoints of the primitive or other straight line segments, add this straight line segment to the connection line list.

[0025] Furthermore, the post-processing includes the following steps: Re-topologically connect the straight lines crossing the busbars, preset the wiring relationship rules, and the identification and reconnection of the cross-connection lines need to be distinguished in combination with the knowledge of the power field.

[0026] Furthermore, the training method of the module division model is as follows: After dividing the function modules by combining a large number of substation wiring diagrams with the business rules of the power system, frame the range and then mark the function module type. Use the primitives, wiring relationships, and corresponding function module types within the framed range as the training set to train the existing neural network. Finally, obtain a model with the input of the substation wiring diagram and the output of the substation wiring diagram marked with the function module type and framed with the function module range as the module division model.

[0027] Further, the training method of the abnormal analysis model is as follows: After a large number of function modules have been selected and abnormal substation wiring diagrams have occurred, mark the abnormal types according to the business rules of the power system. Use the graphics elements, wiring relationships within the selected range, and corresponding abnormal types as the training set to train the existing neural network. Finally, obtain a model with the substation wiring diagram as the input and the marked abnormal types as the output, which is the abnormal analysis model; the abnormal types include atypical graphics elements and wiring, overlapping graphics elements, intersecting graphics elements, interfering wiring, and abnormal graphics element formats.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] The present invention adopts a general graphics element recognition model and a character recognition model, which can recognize graphics elements and characters, and combines the business rules of the power system with the characters around the straight line to complete the recognition of busbars and conventional lines and the recognition of the connection relationships of graphics elements; the present invention sets a module division model and an abnormal analysis model. The module division model extracts the graphics elements and wiring relationships from the substation wiring diagram and divides the substation wiring diagram into several function modules according to the wiring functions. The abnormal analysis model is used to perform abnormal analysis on the function modules and stores correction methods. If no abnormal types are analyzed by the abnormal analysis, it directly passes through the output module; if abnormal types are analyzed and there are correction methods, correct the errors of the substation wiring diagram through the correction methods and then pass through the output module; if abnormal types are analyzed but there are no correction methods, then select the function module where the abnormality is located and pass through the output module. The output module can output CIM / SVG files that meet the standards. The abnormal analysis model makes judgments based on function modules, has good adaptability and practicability, and the stored correction methods can accurately identify when graphics elements are tangent and overlapping, and can timely discover and mark when obvious problems occur in the circuit of the substation wiring diagram, and can quickly and accurately output CIM / SVG files that meet the standards. Description of the Drawings

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for description in the embodiments.

[0031] Figure 1 It is a flowchart of a multimodal-based graphic and model recognition method according to an embodiment of the present invention. Detailed Embodiments

[0032] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0033] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0034] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0035] As Figure 1 shown, an embodiment of the present invention provides a multimodal-based graph model recognition method, including the following stages:

[0036] (1) Preprocessing stage, preprocess the training set;

[0037] (2) Model training stage, input the training set to train the existing network models respectively to obtain a general graph element recognition model, a text recognition model, a module division model, and an anomaly analysis model; the general graph element recognition module is used to recognize the type, rotation angle, and graph element coordinates of the graph elements in the substation wiring diagram; the text recognition model is used to extract all the text pictures in the substation wiring diagram and recognize the content of the text pictures; the module division model is used to divide the substation wiring diagram into several functional modules according to the graph elements and wiring relationships extracted from the substation wiring diagram and combined with the wiring functions; the anomaly analysis model is used to perform anomaly analysis on the functional modules and store correction methods; the graph elements include circuit breakers, disconnecting switches, earthing switches, two-winding transformers, three-winding transformers, capacitors, reactors, lines, and generators.

[0038] The general graph element recognition model adopts an overlapping window mechanism and the YOLO object detection algorithm, and uses CSPDarkNet53 as the feature extraction network. Regarding the recognition of the rotation direction of the graph elements, rotation direction classifiers are respectively constructed for the various recognized graph elements, and thus the fine classification of the graph elements can be achieved.

[0039] Considering the accuracy and speed of automatic extraction of substation wiring diagram information comprehensively, the single-stage YOLO algorithm is adopted in the graphic element recognition part. CSPDarkNet53 is used as the feature extraction network to obtain multiple feature maps with different scales. Then, the SPP module is connected to comprehensively expand the receptive field by using pooling kernels of multiple sizes. The improved PAN module is used to deeply fuse multi-scale feature maps through upsampling and downsampling to improve the detection effect of targets at different scales. The time and space complexity of the training and inference processes are both low, which is beneficial to actual deployment and application.

[0040] Since the information density of the XX substation wiring diagram is much greater than that of daily pictures, its resolution is usually high, and the width-to-height ratios of various graphic elements such as circuit breakers, isolating switches, and capacitors to the width and height of the wiring diagram are all less than 0.1, which belongs to the "large image, small target" detection problem. If traditional object detection algorithms are used, the image needs to be reduced to the size required by the algorithm, and then after multiple convolutions and poolings in the object detection model, the problem of graphic element information loss is serious, and dense small targets are more likely to gather into one point in the feature map, resulting in a high rate of missed detection and misdetection.

[0041] To solve this problem, the YOLT (you only look twice) algorithm is adopted. The specific approach of the overlapping sliding window mechanism in the YOLT algorithm is as follows: Set the overlapping rate as r and the sliding window size as w window ×w window , as shown by the red box in the figure, slide and cut from left to right and from top to bottom. The gray shadow in the figure is the overlapping part of two adjacent subgraphs cut, and one side length of the overlapping part is w window , and the other side length is r×w window . After using the overlapping sliding window mechanism, there is no need to reduce the subgraphs, which is equivalent to increasing the size of small target graphic elements in the input of the YOLO model and the spacing of dense graphic elements, reducing the information loss in the preprocessing and model inference processes, and effectively improving the graphic element recognition effect. Reasonably setting the overlapping rate can reduce the impact of cutting on the integrity of graphic elements, making the graphic elements remain complete in at least one subgraph, and to a certain extent, it is equivalent to upsampling the samples, which is more prominent for dense small target graphic elements and is beneficial to the training of the model.

[0042] The text recognition model adopts the CRNN model. The text recognition model includes a convolutional neural network, a bidirectional recurrent neural network, and a temporal connection classification loss function; the convolutional neural network is used to extract features from the substation wiring diagram to obtain a feature map. The temporal connection classification loss function is used to predict the feature vectors to obtain the probability distribution of each feature vector; according to the probability distribution of each feature vector, the corresponding label sequence is obtained, and combined with the sequence merging mechanism, the final predicted text is output.

[0043] The training method of the module division model is as follows: After dividing the massive substation wiring diagrams into functional modules in combination with the power system business rules, select the range and then mark the functional module types. Use the graphics elements, wiring relationships, and corresponding functional module types within the selected range as the training set to train the existing neural network. Finally, obtain a model with the substation wiring diagram as the input and the substation wiring diagram marked with functional module types and the selected range of functional modules as the output, which is the module division model.

[0044] The training method of the anomaly analysis model is as follows: After selecting the functional modules and the abnormal substation wiring diagrams, mark the anomaly types according to the power system business rules. Use the graphics elements, wiring relationships, and corresponding anomaly types within the selected range as the training set to train the existing neural network. Finally, obtain a model with the substation wiring diagram as the input and the marked anomaly types as the output, which is the anomaly analysis model. The anomaly types include atypical graphics elements and wiring, overlapping graphics elements, intersecting graphics elements, interfering wiring, and abnormal graphics element formats.

[0045] (3) In the image recognition stage, for the substation wiring diagram to be recognized, complete graphic element recognition, text recognition, and wiring relationship recognition. Graphic element recognition: Complete graphic element recognition according to the ordinary graphic element recognition model. Text recognition: Complete text recognition according to the text recognition model. Wiring relationship recognition: Combine the power system business rules and the text around the straight line to complete the recognition of buses and conventional lines and the recognition of the connection relationship of graphic elements.

[0046] The wiring relationship recognition specifically includes wiring preprocessing, topology relationship detection, and post-processing.

[0047] The wiring preprocessing includes the following steps:

[0048] (1) Image grayscale conversion processing: Perform grayscale conversion processing on the substation wiring diagram.

[0049] (2) Remove interference items: Whiten the interference items of the graphic elements, text, and the straight line parts outside the wiring diagram, and set the pixels within the given coordinate frame to zero.

[0050] (3) Straight line detection, classification, and intersection straight line cutting: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows:

[0051] Step 1: Straight line detection: Use the straight line detection algorithm to detect straight lines in the processed substation wiring diagram, and store the coordinates of the two endpoints of the detected straight lines in the straight line list.

[0052] Step 2: Straight line classification: Classify the recognized straight lines into vertical straight lines and horizontal straight lines according to the coordinate relationship at both ends of the straight line.

[0053] Step 3: Intersection straight line cutting: Cut the intersecting straight lines into independent straight line segments, and then perform wiring relationship recognition according to domain knowledge.

[0054] The topological relationship detection includes the following steps: matching the endpoints of all line segments with graphic elements or other line segments. The incorporated rules are as follows: traverse all line segments, first try to match one of the endpoints of a line segment with a graphic element. The matching rule is that if the distance between the endpoint of the line segment and the graphic element is less than the set value, the matching is successful; if the matching fails, try to match with the endpoints of other line segments. The matching rule is that if the distance between a certain endpoint of a line segment and a certain endpoint of another line segment is less than the set value, the matching is successful; if both endpoints of a line segment can be matched with the endpoints of a graphic element or other line segments, add this line segment to the connection line list.

[0055] The post-processing includes the following steps: re-topologically connecting the line segments crossing the busbar, presetting the wiring relationship rules, and the identification and reconnecting of the cross-connection lines need to be distinguished by combining the knowledge in the power field.

[0056] For example: the relationship between the connection line and the busbar can be divided into the following 3 types (which can also be summarized according to the specific substation wiring guide). When the connection line and the busbar form a "T" cross relationship, the connection line and the busbar are in a connected relationship; when the connection line and the busbar form a "cross" cross relationship and both endpoints of the connection line are connected to graphic elements, the connection line and the busbar are in a connected relationship; when the connection line and the busbar form a "cross" cross relationship and one of the two endpoints of the connection line is connected to a graphic element, the connection line and the busbar are not in a connected relationship, and this connection line is a line segment crossing the busbar.

[0057] (4) In the module division stage, after completing step (3), obtain several functional modules through the module division model;

[0058] (5) In the post-processing stage, on the basis of completing the graphic element recognition, text recognition, and wiring relationship recognition in step (3), complete a series of downstream tasks, including extracting the basic information of the substation, identifying the operating voltage of the graphic elements, associating and matching the graphic elements with the text, and completing the abbreviation of the text.

[0059] (6) In the output stage, through the abnormal analysis model, if there is no abnormal type in the abnormal analysis, directly pass through the output module; if the abnormal analysis reveals an abnormal type and there is a correction method, correct the error of the substation wiring diagram through the correction method and then pass through the output module; if the abnormal analysis reveals an abnormal type but there is no correction method, then box the functional module where the abnormality is located and pass through the output module; the output module can output a CIM / SVG file that meets the standards.

[0060] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims described above.

Claims

1. A multi-modal based graph pattern recognition method, characterized in that, It includes the following stages: (1) Preprocessing stage, preprocess the training set; (2) Model training stage, input the training set to train the existing network models respectively to obtain a general graphic element recognition model, a text recognition model, a module division model, and an anomaly analysis model; the general graphic element recognition module is used to recognize the types, rotation angles, and graphic element coordinates of the substation wiring diagram; the text recognition model is used to extract all text pictures in the substation wiring diagram and recognize the content of the text pictures; the module division model is used to divide the substation wiring diagram into several functional modules according to the graphic elements and wiring relationships extracted from the substation wiring diagram and combined with the wiring functions; the anomaly analysis model is used to perform anomaly analysis on the functional modules and store correction methods; (3) Image recognition stage, for the substation wiring diagram to be recognized, complete graphic element recognition, text recognition, and wiring relationship recognition; Graphic element recognition: complete graphic element recognition according to the general graphic element recognition model; Text recognition: complete text recognition according to the text recognition model; Wiring relationship recognition: combine the power system business rules and the text around the straight line to complete the recognition of the bus and conventional lines and the recognition of the graphic element connection relationship; (4) Module division stage, after completing step (3), obtain several functional modules through the module division model; (5) Post-processing stage, on the basis of completing the graphic element recognition, text recognition, and wiring relationship recognition in step (3), complete a series of downstream tasks, including substation basic information extraction, graphic element working voltage recognition, graphic element-text association matching, and text abbreviation completion; (6) Output stage, through the analysis of the anomaly analysis model, if there is no anomaly type in the anomaly analysis, it will directly pass through the output module; if an anomaly type is detected in the anomaly analysis and there is a correction method, correct the error of the substation wiring diagram through the correction method and pass through the output module; if an anomaly type is detected in the anomaly analysis but there is no correction method, the functional module where the anomaly is located will be boxed and passed through the output module; the output module can output a CIM / SVG file that meets the standards.

2. The multimodal-based graph pattern recognition method according to claim 1, wherein The graphic elements include circuit breakers, isolating switches, earthing switches, two-winding transformers, three-winding transformers, capacitors, reactors, lines, and generators.

3. A multimodal-based graph model recognition method according to claim 1, characterized in that, The general graphic element recognition model uses an overlapping windowing mechanism and the YOLO object detection algorithm, with CSPDarkNet53 as the feature extraction network.

4. A multimodal-based graph pattern recognition method according to claim 1, wherein The text recognition model uses a CRNN model; the text recognition model includes a convolutional neural network, a bidirectional recurrent neural network, and a temporal connection classification loss function; the convolutional neural network is used to extract features from the substation wiring diagram to obtain a feature map; the temporal connection classification loss function is used to predict the feature vectors to obtain the probability distribution of each feature vector; according to the probability distribution of each feature vector, obtain the corresponding label sequence, and combine the sequence merging mechanism to output the final predicted text.

5. A multimodal-based graph pattern recognition method according to claim 1, wherein In step (3), the wiring relationship recognition specifically includes wiring preprocessing, topological relationship detection, and post-processing.

6. A multimodal-based graph pattern recognition method according to claim 5, characterized in that The wiring preprocessing includes the following steps: (1) Image grayscale conversion processing: perform grayscale conversion processing on the substation wiring diagram; (2) Remove interference items: Paint the interference items of the graphic elements, text, and the straight-line parts outside the wiring diagram white, and set the pixels within the given coordinate frame to zero. (3) Straight-line detection, classification, and cutting of intersecting straight lines: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows: (3) Straight-line detection, classification, and cutting of intersecting straight lines: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows: (3) Straight-line detection, classification, and cutting of intersecting straight lines: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows: (3) Straight-line detection, classification, and cutting of intersecting straight lines: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows:

7. A multimodal-based graph pattern recognition method according to claim 5, characterized in that (3) Straight-line detection, classification, and cutting of intersecting straight lines: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows: (3) Straight-line detection, classification, and cutting of intersecting straight lines: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows:

8. A multimodal-based graph pattern recognition method according to claim 5, characterized in that, (3) Straight-line detection, classification, and cutting of intersecting straight lines: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows:

9. A multimodal-based graph model recognition method according to claim 1, characterized in that (3) Straight-line detection, classification, and cutting of intersecting straight lines: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows:

10. A multimodal-based graph pattern recognition method according to claim 1, characterized in that, (3) Straight-line detection, classification, and cutting of intersecting straight lines: Classify the detected straight lines and cut the intersecting straight lines. The specific steps are as follows:

Images