A topological graph structure information processing system based on image recognition

Abstract

The present invention relates to the fields of industrial design, construction engineering and energy, and discloses a topological graph structure information processing system based on image recognition, which overcomes the problems of insufficient topological relationship restoration, low recognition accuracy, lack of optimization and correction mechanism, difficult editing operation, poor data reusability and the like in the prior art, and provides an efficient, accurate and easy-to-edit solution, which provides strong support for the digital processing, information management and data accumulation of drawings, including an image recognition result acquisition module, a preliminary topological information analysis module, a line segment detection and deduplication module, a graph structure information construction module, an element fitting module, a bending fitting module, which is used to merge two identified points of different axes into one point; and a final graph structure generation module, which is used to integrate the connection relationship between the points formed by the element fitting module and the bending fitting module into a complete graph structure.

Description

Technical Field

[0001] The present invention relates to the fields of industrial design, construction engineering and energy, and in particular to a topological graph structure information processing system based on image recognition. Background Art

[0002] In the fields of industrial design, construction engineering, energy, etc., a large number of drawings still exist in the form of non-editable pictures. These drawings usually need to be further digitized by extracting point and line information through image recognition technology (such as OCR text recognition and object detection). However, the existing technology has the following technical problems in the process of generating structured topological information from image recognition results:

[0003] 1. Insufficient topological relationship restoration:

[0004] Existing image recognition systems can only extract basic geometric information (such as point coordinates, line start and end positions, text content, etc.), but have limited processing capabilities for complex topological relationships between points and lines (such as connections, intersections, mergers, etc.) and cannot directly generate complete graph structure information.

[0005] 2. Insufficient accuracy of recognition results:

[0006] Image recognition results often contain redundancy or noise, such as extra points, misconnected lines, or incorrect text locations. In the absence of a post-processing mechanism, these problems can lead to incorrect topological information, seriously affecting the usability of the data.

[0007] 3. Lack of optimization and correction mechanism:

[0008] In traditional technologies, the processing of boundary conditions mostly relies on manual intervention, with low automation and insufficient efficiency. At the same time, existing methods are difficult to globally optimize the recognition results, making it difficult to directly apply the final output information.

[0009] 4. Output data is difficult to edit directly:

[0010] After image recognition, existing technologies usually only output geometric information such as points, lines, and text, lacking a complete topological structure description. This data format has the following problems in subsequent use:

[0011] Users cannot directly edit the recognition results, such as adjusting node positions, modifying line segment directions, redefining text annotations, etc.

[0012] The lack of an editor-friendly data interface results in poor compatibility when data is transferred between different software, increasing the complexity of secondary processing.

[0013] 5. Low data reusability:

[0014] Existing technologies usually output one-time image recognition results and lack a unified topological structure model to support long-term data accumulation and reuse: data is difficult to store in a unified format, resulting in duplication of work. The lack of ability to accumulate and analyze historical data is not conducive to the needs of intelligent development.

[0015] 6. Low editing efficiency and much manual intervention:

[0016] In the existing technology, topological relationship restoration and recognition result optimization require a lot of manual participation, which is manifested in: users need to manually merge redundant nodes, correct incorrect connections or adjust text positions, which takes a lot of time. The system cannot automatically optimize point-line relationships (such as curved fitting, node merging, etc.) according to context rules, making the overall processing efficiency low. Summary of the invention

[0017] The purpose of the present invention is to provide a topological graph structure information processing system based on image recognition, which overcomes the problems existing in the prior art, such as insufficient topological relationship restoration, low recognition accuracy, lack of optimization and correction mechanism, difficult editing operation, poor data reusability, etc., and provides an efficient, accurate and easy-to-edit solution, which provides strong support for the digital processing, information management and data accumulation of drawings.

[0018] The present invention is implemented by the following technical solution: a topological graph structure information processing system based on image recognition, comprising:

[0019] The image recognition result acquisition module extracts the image content in the drawing through image recognition technology (such as OCR, object detection, etc.) to obtain preliminary image recognition result data. The result usually includes geometric elements such as points, lines, and text;

[0020] The preliminary topological information analysis module converts the image recognition result data into preliminary topological information after preliminary analysis. This module converts the point and line information in the image into a data structure with topological relationships;

[0021] Line segment detection and deduplication module, based on the preliminary topological information, this module performs line segment detection and removes duplicate line segments. This module optimizes the recognition results, reduces redundant information, and lays the foundation for subsequent processing;

[0022] The graph structure information module is constructed. After removing duplicate line segments and other identification information (such as text, nodes, etc.), it is passed to the graph analysis tool of the graph structure information module to construct a complete graph structure information. This module is the core of the entire processing process, and the topological relationship between each node is represented by the graph structure;

[0023] The element fitting module is used to aggregate the recognized rectangular boxes or recognized text information into a single node and enter the topological graph structure; it helps to simplify the geometric structure and reduce the amount of data processing; the text information, as the key annotation content, can provide support for subsequent editing and information management;

[0024] The curved fitting module is used to merge two points on different axes into one point. This module effectively integrates the information of the curved part and improves the accuracy of the graph;

[0025] The final graph structure generation module is used to integrate the nodes formed by the element fitting module, the points formed by the bending fitting module, and the connection relationship between the nodes and the points into a complete graph structure. This graph structure provides a basis for subsequent operations (such as data editing, optimization and information processing).

[0026] In order to better implement the topological graph structure information processing system based on image recognition described in the present invention, the following setting is particularly adopted: when the line segment detection and deduplication module removes duplicate line segments, it specifically includes the following steps:

[0027] 1.1) Line segment endpoint traversal: Hough Transform is used to detect straight lines in the image content. By traversing each detected straight line, its endpoint coordinates are extracted to form the basic data for subsequent calculations. Among them, the endpoint extraction of the line segment is through the functions provided by the image processing library OpenCV, such as cv2.HoughLinesP, to detect straight lines and obtain the starting point and end point of each line;

[0028] 1.2) Calculation of line segment angle: Calculate the angle of the line segment by applying the inverse tangent function (atan2) to the endpoints obtained by traversing the line segment endpoints in step 1.1);

[0029] 1.3) Preliminary grouping based on axonometric reference angles: Compare the angle calculated in step 1.2) with the preset reference angles (such as 30 degrees, 90 degrees, and 150 degrees), and determine which angle range each line segment belongs to based on the angle tolerance (such as ±4.5 degrees); if the line segment angle is close to the reference angle, it is classified into the corresponding group. This process can use angle difference calculation to determine whether it belongs to the same category;

[0030] 1.4) Secondary grouping within the group: After step 1.3), the line segments in the same group are secondary grouped using the proximity judgment method; specifically, the shortest distance between every two line segments is calculated to determine whether they are close; if they are close, they are merged into one group; this process uses the Euclidean distance calculation to determine the spatial proximity between the line segments.

[0031] 1.5) De-duplication and merging of farthest endpoints: After step 1.4), the line segments in each group are de-duplicated, redundant line segments are removed, and the graph structure is optimized; this is mainly achieved by calculating the endpoint distance of the line segments in the group. The specific method is to calculate the Euclidean distance between every two endpoints and select the two farthest endpoints to merge, thereby removing redundant line segments and optimizing the graph structure.

[0032] 1.6) Endpoint accuracy adjustment: In order to improve recognition accuracy, the minimum angle difference method is used to adjust the endpoint coordinates. Then, by calculating the difference between the endpoint angle and the target reference angle, the position of the endpoint is adjusted within a certain radius so that the angle of the line segment is close to the standard angle. This operation is completed through coordinate fine-tuning and radius optimization.

[0033] The present invention effectively improves the image recognition accuracy by performing operations such as endpoint traversal, angle calculation, grouping, de-duplication merging and precision adjustment on line segments in drawings. The algorithm can generate a more accurate graph structure and provide reliable data support for subsequent digitalization and information management.

[0034] To further better implement the topological graph structure information processing system based on image recognition described in the present invention, the following setting is particularly adopted: the step 1.2) is specifically as follows: first, extract the endpoint coordinates (x1, y1) and (x2, y2) of each line segment, then calculate the slope between the two endpoint coordinates, then use np.arctan2 to calculate the radian, then convert the radian to an angle, and convert the angle to a range of 0 to 180 degrees through degree normalization. The calculation results can be used to distinguish layouts in different directions.

[0035] In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting is particularly adopted: the line segments in the same group are secondary grouped by using the proximity judgment method, specifically: the shortest distance between each two line segments is calculated to determine whether they are close; if they are close, they are merged into one group; this process is based on the limit of image pixels, and the Euclidean distance calculation is used point by point for two line segments, and the mean value between points is calculated to determine the spatial proximity;

[0036] The deduplication process is performed on the line segments in each group, specifically by calculating the Euclidean distance between every two endpoints and selecting the two farthest endpoints for merging, thereby removing redundant line segments.

[0037] In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting is particularly adopted: the element fitting module aggregates the recognized rectangular frame or the recognized text information into a single node, including the following steps:

[0038] 2.1) Receive graph structure information: Receive the graph structure information constructed by the graph structure information construction module, use image recognition technology (such as target detection model and OCR text recognition) to obtain the four vertex coordinates of the elements (text, etc.) in the image, and then use Shapely or similar geometric tools to extract the vertex coordinates of the elements (text, etc.) to provide basic data for subsequent geometric analysis. To obtain accurate geometric coordinates, lay the foundation for topological relationship reconstruction and graphic fitting.

[0039] 2.2) Generate an extended bounding box: After step 2.1), use shapely's box function to generate an enclosing rectangle to construct a bounding box using the vertex coordinates of the element (text, etc.); use a geometric algorithm to dynamically adjust the size of the bounding box (for example, increase a certain pixel threshold), and adaptively expand it according to the actual size of the element to ensure that all relevant points are covered, and finally generate an extended bounding box; after the bounding box is expanded, provide area restrictions for subsequent neighboring point analysis to reduce missed detection or false detection.

[0040] 2.3) Neighboring point analysis and extension line generation: After step 2.2), use the spatial geometry analysis method to determine the number of neighboring points of the extended bounding box by calculating the distance between the points of the extended bounding box. According to the number of neighboring points, it will be divided into three categories: single neighboring point, two neighboring points, and three neighboring points. For the above three categories, use the intersection function provided by Shapely to generate one or more extension lines, and use the intersection points of these extension lines as nodes of elements (text, etc.) to enter step 2.4). Among them, the specific operations of generating extension lines for each category and using the intersection points of the extension lines as nodes of elements (text, etc.) are as follows:

[0041] Single neighboring point: Based on the neighboring point, the midpoint of the intersection of the extended line and the bounding box is used as the node;

[0042] Two adjacent points: Generate an extension line based on the two points, and use the intersection of the two extension lines as a node;

[0043] Three adjacent points: Generate extension lines based on the three points, and use the intersection of the three extension lines as nodes;

[0044] This step can accurately determine the relationship between points and provide a basis for the next step of extension line generation and fitting.

[0045] 2.4) Node dynamic adjustment optimization: Generate a circular area of ​​a specified size near the node obtained in step 2.3), and further optimize the node accuracy by dynamically calculating the node that best fits the topological graph structure in the circular area;

[0046] 2.5) Fit elements (text, etc.) to nodes: The nodes obtained in step 2.4) are added to the topological graph structure, and their connection relationships in the topological graph structure are updated. This process is called fitting. It is worth noting that in the fitting of text elements, the axis and the height of the text box are combined to correct the annotation position and angle of the text.

[0047] The purpose is to accurately represent the elements (text, etc.) in the image in the form of points, forming a clear graph structure to provide support for subsequent topological analysis and information operations.

[0048] In order to better implement the topological graph structure information processing system based on image recognition described in the present invention, the following setting is particularly adopted: when the curved fitting module merges two identified points on different axes into one point, the specific steps are as follows:

[0049] 3.1) Find single connection point:

[0050] First, we use topology analysis technology to traverse all nodes in the graph and check the connection relationship of each node. Then, we use the neighbors method in the NetworkX library to quickly identify single connection points that are only connected to one other node. Finally, we combine the properties of the nodes to further screen valid single connection points.

[0051] The purpose is to automatically detect single connection points in the system, especially the end or joint position, to lay the foundation for subsequent merging operations.

[0052] 3.2) After the singly connected nodes are identified in step 3.1), the coordinates of these singly connected nodes are calculated pairwise by using the geometric calculation method, and then the calculation results are compared with the set distance threshold, and the point pairs that exceed the range are filtered out to obtain the point pairs that meet the distance threshold;

[0053] The purpose is to achieve accurate distance calculation to avoid the incorrect merging of distant points, thereby ensuring the accuracy of the graph structure.

[0054] 3.3) Merge points and update connection relationships:

[0055] For the point pairs that meet the distance threshold, calculate the midpoint of the two points and merge the two points into the midpoint. This midpoint will replace the two points as a new node and update the corresponding connection relationship in the topological graph structure. That is, use the graph operation function in NetworkX to add the merged point as a new node to the graph structure, delete the original node, and reconnect the neighbors of the new node and the original node.

[0056] It reduces redundant connections in the graph, dynamically optimizes the topological structure, makes the graph more concise and accurate, and provides convenience for subsequent editing.

[0057] 3.4) Loop iteration and real-time update:

[0058] In step 3.3), after each operation of merging two points into their midpoints, all single connection points are re-traversed through the loop structure to ensure that newly generated single connection points can also be processed in time; then a Boolean variable is used to mark whether the merge occurs. If so, the iteration continues; if no merge operation occurs, the loop is terminated;

[0059] It can ensure that all eligible point pairs can be processed, and the graph structure is automatically updated after each merge until it reaches the optimal state.

[0060] 3.5) Automatic stop mechanism:

[0061] Using the while loop control structure, after each iteration, by checking whether there are still single connection points or whether there is a merger, when there is no single connection point or no merger, the loop is exited and the bending fitting is ended. At this point, all eligible nodes in the entire graph structure information have undergone the bending fitting operation.

[0062] In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting method is particularly adopted: it also includes a dictionary storage module, which stores the generated graph structure data in a dictionary storage manner, and is provided with a dictionary A and a dictionary B, wherein dictionary A is used to store detailed information of each node including coordinates, attributes, etc.; dictionary B is used to store the connection relationship between nodes, and is used to represent the topological structure.

[0063] In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting method is particularly adopted: it also includes a data docking and output module, which is used to convert the graph structure data into JSON format, dock with other information management platforms (such as professional drawing software, etc.), and support further processing and application of data.

[0064] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0065] The present invention can accurately restore topological relationships and proposes a processing framework for generating a complete topological graph structure from image recognition results, including reconstruction of point-line relationships, boundary expansion, and connection relationship optimization.

[0066] The present invention can optimize the recognition results, and realize automatic optimization through geometric calculation and rule matching for redundant points, erroneous line segments and text offsets generated during the recognition process, thereby improving the accuracy and practicality of the results.

[0067] The present invention can enhance the automatic processing capability and provide a complete set of post-processing mechanisms, including functions such as curve fitting, single connection point processing, text height and angle adjustment, etc., which can significantly reduce manual intervention and improve processing efficiency.

[0068] The present invention can expand application scenarios and design a system architecture with good scalability. It can meet the needs of various drawing scenarios such as building floor plans, energy network diagrams such as electricity, water, natural gas, heating, and chemicals, and provide the industry with a highly versatile solution.

[0069] The present invention can support editing operations, and the output result of the present invention is complete topological information structure data. By generating standardized topological structures (such as point-line associations, connection relationships, and text annotations, etc.), users can directly interactively edit these results in the software, such as modifying the direction of line segments, adjusting text content, and rearranging node positions, etc., thereby greatly improving the user's editing efficiency and flexibility.

[0070] The present invention can promote information management, and the topological information structure of the present invention provides high-quality input data for subsequent information processing. Through standardized structured output, the system can easily connect to the enterprise's digital management platform (such as BIM system, operation and maintenance management system, etc.), and realize functions such as visual display of equipment, operation status analysis, and automatic generation of maintenance plans.

[0071] The present invention can improve the reuse value of data. The topological information data generated by the system can not only be used for digital editing of current drawings, but also for long-term data accumulation and knowledge base construction of the enterprise, providing more reliable data support for subsequent optimization design and intelligent analysis.

[0072] The present invention overcomes the problems existing in the prior art, such as insufficient topological relationship restoration, low recognition accuracy, lack of optimization and correction mechanism, difficult editing operation, poor data reusability, etc., and provides an efficient, accurate and easy-to-edit solution, which provides strong support for the digital processing, information management and data accumulation of drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is the main logic diagram of the present invention.

[0074] Figure 2 Flowchart for deduplication of line segments.

[0075] Figure 3 A flowchart for fitting elements such as text.

[0076] Figure 4 A schematic diagram illustrating the principle of fitting elements such as text.

[0077] Figure 5Flowchart of the curved fitting. DETAILED DESCRIPTION

[0078] The present invention is further described in detail below in conjunction with examples, but the embodiments of the present invention are not limited thereto.

[0079] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0080] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0081] Embodiment 1:

[0082] The present invention designs a topological graph structure information processing system based on image recognition, which overcomes the problems existing in the prior art, such as insufficient topological relationship restoration, low recognition accuracy, lack of optimization and correction mechanism, difficult editing operation, poor data reusability, etc., and provides an efficient, accurate and easy-to-edit solution, which provides strong support for the digital processing, information management and data accumulation of drawings, including

[0083] The image recognition result acquisition module extracts the image content in the drawing through image recognition technology (such as OCR, object detection, etc.) to obtain preliminary image recognition result data. The result usually includes geometric elements such as points, lines, and text;

[0084] The preliminary topological information analysis module converts the image recognition result data into preliminary topological information after preliminary analysis. This module converts the point and line information in the image into a data structure with topological relationships;

[0085] Line segment detection and deduplication module, based on the preliminary topological information, this module performs line segment detection and removes duplicate line segments. This module optimizes the recognition results, reduces redundant information, and lays the foundation for subsequent processing;

[0086] The graph structure information module is constructed. After removing duplicate line segments and other identification information (such as text, nodes, etc.), it is passed to the graph analysis tool of the graph structure information module to construct a complete graph structure information. This module is the core of the entire processing process, and the topological relationship between each node is represented by the graph structure;

[0087] The element fitting module is used to aggregate the recognized rectangular boxes or recognized text information into a single node and enter the topological graph structure; it helps to simplify the geometric structure and reduce the amount of data processing; the text information, as the key annotation content, can provide support for subsequent editing and information management;

[0088] The curved fitting module is used to merge two points on different axes into one point. This module effectively integrates the information of the curved part and improves the accuracy of the graph;

[0089] The final graph structure generation module is used to integrate the nodes formed by the element fitting module, the points formed by the bending fitting module, and the connection relationship between the nodes and the points into a complete graph structure. This graph structure provides a basis for subsequent operations (such as data editing, optimization and information processing).

[0090] Embodiment 2:

[0091] This embodiment is further optimized on the basis of the above embodiment, and the same points as the above technical solution are not repeated here. In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting method is particularly adopted: when the line segment detection and deduplication module removes duplicate line segments, it specifically includes the following steps:

[0092] 1.1) Line segment endpoint traversal: Hough Transform is used to detect straight lines in the image content. By traversing each detected straight line, its endpoint coordinates are extracted to form the basic data for subsequent calculations. Among them, the endpoint extraction of the line segment is through the functions provided by the image processing library OpenCV, such as cv2.HoughLinesP, to detect straight lines and obtain the starting point and end point of each line;

[0093] 1.2) Calculation of line segment angle: Calculate the angle of the line segment by applying the inverse tangent function (atan2) to the endpoints obtained by traversing the line segment endpoints in step 1.1);

[0094] 1.3) Preliminary grouping based on axonometric reference angles: Compare the angle calculated in step 1.2) with the preset reference angles (such as 30 degrees, 90 degrees, and 150 degrees), and determine which angle range each line segment belongs to based on the angle tolerance (such as ±4.5 degrees); if the line segment angle is close to the reference angle, it is classified into the corresponding group. This process can use angle difference calculation to determine whether it belongs to the same category;

[0095] 1.4) Secondary grouping within the group: After step 1.3), the line segments in the same group are secondary grouped using the proximity judgment method; specifically, the shortest distance between every two line segments is calculated to determine whether they are close; if they are close, they are merged into one group; this process uses the Euclidean distance calculation to determine the spatial proximity between the line segments.

[0096] 1.5) De-duplication and merging of farthest endpoints: After step 1.4), the line segments in each group are de-duplicated, redundant line segments are removed, and the graph structure is optimized; this is mainly achieved by calculating the endpoint distance of the line segments in the group. The specific method is to calculate the Euclidean distance between every two endpoints and select the two farthest endpoints to merge, thereby removing redundant line segments and optimizing the graph structure.

[0097] 1.6) Endpoint accuracy adjustment: In order to improve recognition accuracy, the minimum angle difference method is used to adjust the endpoint coordinates. Then, by calculating the difference between the endpoint angle and the target reference angle, the position of the endpoint is adjusted within a certain radius so that the angle of the line segment is close to the standard angle. This operation is completed through coordinate fine-tuning and radius optimization.

[0098] The present invention effectively improves the image recognition accuracy by performing operations such as endpoint traversal, angle calculation, grouping, de-duplication merging and precision adjustment on line segments in drawings. The algorithm can generate a more accurate graph structure and provide reliable data support for subsequent digitalization and information management.

[0099] Embodiment 3:

[0100] This embodiment is further optimized on the basis of any of the above embodiments. The similarities with the above technical solutions are not repeated here. In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting method is particularly adopted: the step 1.2) is specifically as follows: first, extract the endpoint coordinates (x1, y1) and (x2, y2) of each line segment, then calculate the slope between the two endpoint coordinates, then use np.arctan2 to calculate the radian, then convert the radian to an angle, and convert the angle to a range of 0 to 180 degrees through degree normalization. The calculation results can be used to distinguish layouts in different directions.

[0101] Embodiment 4:

[0102] This embodiment is further optimized on the basis of any of the above embodiments, and the similarities with the above technical solutions are not repeated here. In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting is particularly adopted: the line segments in the same group are secondary grouped using the proximity judgment method, specifically: the shortest distance between each two line segments is calculated to determine whether they are close; if they are close, they are merged into one group; this process is based on the limit of image pixels, and the Euclidean distance calculation is used for the two line segments point by point, and the mean value between the points is calculated to determine the spatial proximity;

[0103] The deduplication process is performed on the line segments in each group, specifically by calculating the Euclidean distance between every two endpoints and selecting the two farthest endpoints for merging, thereby removing redundant line segments.

[0104] Embodiment 5:

[0105] This embodiment is further optimized on the basis of any of the above embodiments, and the same points as the above technical solutions are not repeated here. In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting method is particularly adopted: the element fitting module aggregates the recognized rectangular frame or the recognized text information into a single node, including the following steps:

[0106] 2.1) Receive graph structure information: Receive the graph structure information constructed by the graph structure information construction module, use image recognition technology (such as target detection model and OCR text recognition) to obtain the four vertex coordinates of the text and other elements in the image, and then use Shapely or similar geometric tools to extract the vertex coordinates of the text and other elements to provide basic data for subsequent geometric analysis. To obtain accurate geometric coordinates, lay the foundation for topological relationship reconstruction and graphic fitting.

[0107] 2.2) Generate an extended bounding box: After step 2.1), use Shapely's box function to generate an enclosing rectangle to construct a bounding box using the vertex coordinates of elements such as text; use a geometric algorithm to dynamically adjust the size of the bounding box (for example, increase a certain pixel threshold), and adaptively expand it according to the actual size of the element to ensure that all relevant points are covered, and finally generate an extended bounding box; after the bounding box is expanded, provide area restrictions for subsequent neighboring point analysis to reduce missed detection or false detection.

[0108] 2.3) Neighboring point analysis and extension line generation: After step 2.2), use the spatial geometry analysis method to determine the number of neighboring points of the extended bounding box by calculating the distance between the points of the extended bounding box. According to the number of neighboring points, they are divided into three categories: single neighboring point, two neighboring points, and three neighboring points. For the above three categories, use the intersection function provided by Shapely to generate one or more extension lines, and use the intersection points of these extension lines as nodes of elements such as text to enter step 2.4). Among them, the specific operations of generating extension lines for each category and using the intersection points of the extension lines as nodes of elements such as text are as follows:

[0109] Single neighboring point: Based on the neighboring point, the midpoint of the intersection of the extended line and the bounding box is used as the node;

[0110] Two adjacent points: Generate an extension line based on the two points, and use the intersection of the two extension lines as a node;

[0111] Three adjacent points: Generate extension lines based on the three points, and use the intersection of the three extension lines as nodes;

[0112] This step can accurately determine the relationship between points and provide a basis for the next step of extension line generation and fitting.

[0113] 2.4) Node dynamic adjustment optimization: Generate a circular area of ​​a specified size near the node obtained in step 2.3), and further optimize the node accuracy by dynamically calculating the node that best fits the topological graph structure in the circular area;

[0114] 2.5) Fitting text and other elements into nodes: The nodes obtained in step 2.4) are added to the topological graph structure, and their connection relationships in the topological graph structure are updated. This process is called fitting. It is worth noting that in the fitting of text elements, the axis and the height of the text box are combined to correct the annotation position and angle of the text.

[0115] The purpose is to accurately represent the text and other elements in the image in the form of points, forming a clear graph structure to provide support for subsequent topological analysis and information operations.

[0116] Embodiment 6:

[0117] This embodiment is further optimized on the basis of any of the above embodiments, and the similarities with the above technical solutions are not repeated here. In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting is particularly adopted: when the curved fitting module merges two points of different axes identified into one point, the specific steps are as follows:

[0118] 3.1) Find single connection point:

[0119] First, we use topology analysis technology to traverse all nodes in the graph and check the connection relationship of each node. Then, we use the neighbors method in the NetworkX library to quickly identify single connection points that are only connected to one other node. Finally, we combine the properties of the nodes to further screen valid single connection points.

[0120] The purpose is to automatically detect single connection points in the system, especially the end or joint position, to lay the foundation for subsequent merging operations.

[0121] 3.2) After the singly connected nodes are identified in step 3.1), the coordinates of these singly connected nodes are calculated pairwise by using the geometric calculation method, and then the calculation results are compared with the set distance threshold, and the point pairs that exceed the range are filtered out to obtain the point pairs that meet the distance threshold;

[0122] The purpose is to achieve accurate distance calculation to avoid the incorrect merging of distant points, thereby ensuring the accuracy of the graph structure.

[0123] 3.3) Merge points and update connection relationships:

[0124] For the point pairs that meet the distance threshold, calculate the midpoint of the two points and merge the two points into the midpoint. This midpoint will replace the two points as a new node and update the corresponding connection relationship in the topological graph structure. That is, use the graph operation function in NetworkX to add the merged point as a new node to the graph structure, delete the original node, and reconnect the neighbors of the new node and the original node.

[0125] It reduces redundant connections in the graph, dynamically optimizes the topological structure, makes the graph more concise and accurate, and provides convenience for subsequent editing.

[0126] 3.4) Loop iteration and real-time update:

[0127] In step 3.3), after each operation of merging two points into their midpoints, all single connection points are re-traversed through the loop structure to ensure that newly generated single connection points can also be processed in time; then a Boolean variable is used to mark whether a merge occurs. If so, the iteration continues; if no merge operation occurs, the loop is terminated; efficient loop iteration logic is implemented to avoid repeated calculations while keeping the graph structure dynamically updated.

[0128] It can ensure that all eligible point pairs can be processed, and the graph structure is automatically updated after each merge until it reaches the optimal state.

[0129] 3.5) Automatic stop mechanism:

[0130] Using the while loop control structure, after each iteration, by checking whether there are still single connection points or whether there is a merger, when there is no single connection point or no merger, the loop is exited and the bending fitting is ended. At this point, all eligible nodes in the entire graph structure information have undergone the bending fitting operation.

[0131] Embodiment 7:

[0132] This embodiment is further optimized on the basis of any of the above embodiments, and the similarities with the above technical solutions are not repeated here. In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting method is particularly adopted: it also includes a dictionary storage module, which stores the generated graph structure data in a dictionary storage manner, and is provided with a dictionary A and a dictionary B, wherein dictionary A is used to store detailed information of each node including coordinates, attributes, etc.; dictionary B is used to store the connection relationship between nodes, and is used to represent the topological structure.

[0133] Embodiment 8:

[0134] This embodiment is further optimized on the basis of any of the above embodiments, and the similarities with the above technical solutions are not repeated here. In order to better realize the topological graph structure information processing system based on image recognition described in the present invention, the following setting method is particularly adopted: it also includes a data docking and output module, which is used to convert the graph structure data into JSON format, dock with other information management platforms (such as professional drawing software, etc.), and support further processing and application of data.

[0135] Embodiment 9:

[0136] A topological graph structure information processing system based on image recognition, combined with Figure 1 As shown, its main logic includes:

[0137] Image recognition results: The content in the drawing is extracted through image recognition technology (such as OCR, target detection, etc.), including element recognition results: a rectangular box composed of four point coordinates and information composed of element names. Text recognition results: a polygonal box composed of four point coordinates and information composed of text content.

[0138] Parsing into preliminary topological information: The image recognition result data is initially parsed and converted into preliminary topological information. This step converts the point and line information in the image into a data structure with topological relationships.

[0139] Line segment detection and deduplication process: Based on the preliminary topological information, this step performs line segment detection and removes duplicate line segments. This step optimizes the recognition results, reduces redundant information, and lays the foundation for subsequent processing. Figure 2 As shown, the specific steps are:

[0140] Endpoint traversal of line segments: Hough Transform is used to detect straight lines in the image content. Each detected straight line is traversed to extract its endpoint coordinates to form the basic data for subsequent calculations. The endpoint extraction of line segments is done through functions provided by the image processing library OpenCV, such as cv2.HoughLinesP, to detect straight lines and obtain the starting and ending points of each line.

[0141] Segment angle calculation: The endpoints obtained by traversing the segment endpoints are used to calculate the angle of the segment using the inverse tangent function (atan2); specifically, first, the endpoint coordinates (x1, y1) and (x2, y2) of each segment are extracted, and then the slope between the two endpoint coordinates is calculated, and then the radians are calculated using np.arctan2, and then the radians are converted to degrees, and the angles are converted to the range of 0 to 180 degrees through degree normalization. The calculation results can be used to distinguish layouts in different directions.

[0142] Perform a grouping based on the axonometric reference angle: compare the angle calculated in the previous step with the preset reference angle (such as positive axonometric angle: 30 degrees, positive axonometric angle: 90 degrees, positive axonometric angle: 150 degrees), and determine which angle range each line segment belongs to based on the angle tolerance (such as ±4.5 degrees); if the line segment angle is close to the reference angle, it is classified into the corresponding group. This process can use angle difference calculation to determine whether it belongs to the same category;

[0143] Angles other than 30 / 90 / 150: For line segments that do not meet the standard axonometric angle, determine whether they are oblique lines. Compare the line segments that do not meet the standard axonometric angle with the standard reference angles (such as 30 degrees, 90 degrees, and 150 degrees). If the angle difference is greater than the set tolerance range, it is determined to be an oblique line and processed separately;

[0144] Secondary grouping based on line segment proximity:

[0145] After the above step, the line segments in the same group are grouped again using the proximity judgment method. Specifically, the shortest distance between every two line segments is calculated to determine whether they are close. If they are close, they are merged into one group. This process is based on the limit of image pixels, and the Euclidean distance calculation is used point by point for the two line segments, and the mean value between points is calculated to determine the degree of spatial proximity.

[0146] Find the two farthest endpoints in the group and perform de-duplication and merging:

[0147] After the above steps, the line segments in each group are deduplicated, redundant line segments are removed, and the graph structure is optimized; this is mainly achieved by calculating the endpoint distance of the line segments in the group. The specific method is to calculate the Euclidean distance between every two endpoints and select the two farthest endpoints to merge, thereby removing redundant line segments and optimizing the graph structure.

[0148] With the endpoint as the center of the circle within the radius, adjust the endpoint coordinates to match the axonometric angle to improve accuracy:

[0149] In order to improve the recognition accuracy, the minimum angle difference method is used to adjust the endpoint coordinates, and then the difference between the endpoint angle and the target reference angle is calculated to adjust the position of the endpoint within a certain radius so that the angle of the line segment is close to the standard angle. This operation is completed through coordinate fine-tuning and radius optimization.

[0150] Segment merging: This is mainly achieved by calculating the endpoint distance of the segment within the group. The specific method is to calculate the Euclidean distance between every two endpoints and select the two farthest endpoints for merging, thereby removing redundant segments and optimizing the graph structure.

[0151] The present invention effectively improves the image recognition accuracy by performing operations such as endpoint traversal, angle calculation, grouping, de-duplication merging and precision adjustment on line segments in drawings. The algorithm can generate a more accurate graph structure and provide reliable data support for subsequent digitalization and information management.

[0152] All the information of the drawing is constructed into graph structure information using NetworkX: after removing duplicate line segments and other identification information (such as text, nodes, etc.), it is passed to the graph analysis tool of the graph structure information construction module to construct the complete graph structure information. This step is the core of the entire processing process, and the topological relationship between each node is represented by the graph structure.

[0153] Fitting of elements such as text: The recognized rectangular boxes (equipment, etc.) or recognized text information (such as numbers, equipment names, etc.) are aggregated into a single node and entered into the topological graph structure; this helps to simplify the geometric structure and reduce the amount of data processing; text information, as the key annotation content, can provide support for subsequent editing and information management.

[0154] Combination Figure 3 As shown, the process of fitting elements such as text includes the following steps:

[0155] Receive graph structure information: Receive the graph structure information constructed by the graph structure information construction module, use image recognition technology (such as target detection model and OCR text recognition) to obtain the four vertex coordinates of the text and other elements in the image, and then use the shapely library or similar geometric tools to process the vertex coordinates of the extracted text and other elements to provide basic data for subsequent geometric analysis. To obtain accurate geometric coordinates, lay the foundation for topological relationship reconstruction and graphic fitting.

[0156] Generate an expanded bounding box based on the four coordinates of the recognition result: After the previous step, use Shapely's box function to generate an outer rectangle to construct a bounding box through the vertex coordinates of elements such as text; use a geometric algorithm to dynamically adjust the size of the bounding box (for example, increase a certain pixel threshold), and adaptively expand it according to the actual size of the element to ensure that all related points are covered, and finally generate an expanded bounding box; after the bounding box is expanded, provide area restrictions for subsequent neighboring point analysis to reduce missed detection or false detection.

[0157] Calculate the number of neighboring points. After the previous step, use the spatial geometry analysis method to determine the number of neighboring points of the extended bounding box by calculating the distance between the points of the extended bounding box. According to the number of neighboring points, they are divided into three categories: single neighboring point, two neighboring points, and three neighboring points. For the above three categories, use the intersection function provided by Shapely to generate one or more extension lines, and use the intersection points of these extension lines as nodes of elements such as text for the next step. Among them, the specific operations of generating extension lines for each category and using the intersection points of the extension lines as nodes of elements such as text are as follows:

[0158] Single neighboring point: Based on the neighboring point, the midpoint of the intersection of the extended line and the bounding box is used as the node;

[0159] Two adjacent points: Generate an extension line based on the two points, and use the intersection of the two extension lines as a node;

[0160] Three adjacent points: Generate extension lines based on the three points, and use the intersection of the three extension lines as nodes;

[0161] Generate a circular area of ​​a specified size near the node obtained in the previous step, and further optimize the node accuracy by dynamically calculating the node that best fits the topological graph structure in the circular area;

[0162] Finally, with the found point, the text and other elements of the four coordinates are fitted into a point and entered into the graph network:

[0163] The nodes obtained in the previous step are added to the topological structure, and their connection relationships in the topological structure are updated. This process is called fitting. It is worth noting that in the fitting of text elements, the axis and the height of the text box are combined to correct the annotation position and angle of the text.

[0164] Examples of fitting elements such as text are Figure 4 As shown in the figure, the purpose is to accurately represent the text and other elements in the image in the form of points, forming a clear graph structure, and providing support for subsequent topological analysis and information operations.

[0165] Curved fitting is used to merge the two points on different axes into one point. This step effectively integrates the information of the curved part and improves the accuracy of the graph.

[0166] Combination Figure 5 As shown, the two points on different axes identified are merged into one point. The specific steps are as follows:

[0167] To begin, find a single connection point:

[0168] First, we use topological structure analysis technology to traverse all nodes in the graph and check the connection relationship of each node. Then, we use the neighbors method in the NetworkX library to quickly identify single connection points that are only connected to one other node. Finally, we combine the properties of the nodes to further screen valid single connection points.

[0169] The purpose is to automatically detect single connection points in the system, especially the end or joint position, to lay the foundation for subsequent merging operations.

[0170] Is there a single connection point: Search in the topology graph structure whether there is a node with only one connection relationship (called a single connection point). If not, end or determine whether it is within the threshold. If so, proceed to the next step.

[0171] Traverse all single connection points: Traverse all single connection points in the topology structure and proceed to the next step.

[0172] Point type check: Check whether the type of these single connection points belongs to the "point" type. If not, end. If yes, proceed to the next step.

[0173] Calculate the distance between two points: Calculate the distance between these single-connected points pairwise, and then proceed to the next step.

[0174] Is it within the threshold: Determine whether the distance is within the threshold and then proceed to the next step.

[0175] Merge two points and update the connection: If they are within the threshold, use geometric methods to find the midpoint of the two points, replace the two points, and update them as new nodes in the topology structure.

[0176] Loop processing: Use the while loop control structure. After each iteration, check whether there are any single connection points or whether any mergers have occurred. When there are no single connection points or no mergers, exit the loop and end the curved fitting. At this point, all eligible nodes in the entire graph structure information have undergone the curved fitting operation.

[0177] Generate the final graph structure: It is used to integrate the connection relationship between the points and individual points formed by fitting elements such as text and bending fitting into a complete graph structure. This graph structure provides the basis for subsequent operations (such as data editing, optimization and information processing).

[0178] Dictionary A is used to store detailed information of each node including coordinates, attributes, etc.

[0179] Dictionary B is used to store the connection relationship between nodes and to represent the topological structure.

[0180] The docking interface is used to convert graph structure data into JSON format, connect with other information management platforms (such as professional drawing software, etc.), and support further processing and application of data.

[0181] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A topological graph structure information processing system based on image recognition, characterized in that: include The image recognition result acquisition module extracts the image content in the drawing through image recognition technology to obtain preliminary image recognition result data; The preliminary topological information analysis module converts the image recognition result data into preliminary topological information through preliminary analysis; Line segment detection and deduplication module, based on the preliminary topological information, this module performs line segment detection and removes duplicate line segments; The graph structure information module is constructed. After removing duplicate line segments and other identification information, it is passed to the graph analysis tool of the graph structure information module to construct the complete graph structure information. The element fitting module is used to aggregate the recognized rectangular boxes or recognized text information into a single node and enter the topological graph structure; The curved fitting module is used to merge two identified points of different axes into one point; The final graph structure generation module is used to integrate the nodes formed by the element fitting module, the points formed by the bending fitting module, and the connection relationship between the nodes and the points into a complete graph structure.

2. A topological graph structure information processing system based on image recognition according to claim 1, characterized in that: The line segment detection and deduplication module specifically includes the following steps when removing duplicate line segments: 1.1) Line segment endpoint traversal: Hough transform is used to detect straight lines in the image content. By traversing each detected straight line, its endpoint coordinates are extracted to form the basic data for subsequent calculations. Among them, the endpoint extraction of the line segment is to detect the straight line and obtain the starting point and end point of each line through the function provided by the image processing library OpenCV; 1.2) Calculation of line segment angle: Calculate the angle of the line segment by performing the inverse tangent function on the endpoints obtained by traversing the line segment endpoints in step 1.1); 1.3) Preliminary grouping based on axonometric reference angle: Compare the angle calculated in step 1.2) with the preset reference angle, and determine which angle range each line segment belongs to based on the angle tolerance; if the line segment angle is close to the reference angle, it is classified into the corresponding group; 1.4) Secondary grouping within a group: After step 1.3), the line segments within the same group are secondary grouped using the proximity judgment method; 1.5) De-duplication and merging of farthest endpoints: After step 1.4), de-duplication is performed on the line segments in each group, redundant line segments are removed, and the graph structure is optimized; 1.6) Endpoint accuracy adjustment: Use the minimum angle difference method to adjust the endpoint coordinates, and then adjust the position of the endpoint within a certain radius by calculating the difference between the endpoint angle and the target reference angle, so that the angle of the line segment is close to the standard angle.

3. A topological graph structure information processing system based on image recognition according to claim 2, characterized in that: The step 1.2) is specifically as follows: first, extract the endpoint coordinates (x1, y1) and (x2, y2) of each line segment, then calculate the slope between the two endpoint coordinates, then use np.arctan2 to calculate the radian, then convert the radian to an angle, and convert the angle to a range of 0 to 180 degrees through degree normalization.

4. A topological graph structure information processing system based on image recognition according to claim 2, characterized in that: The method of using proximity judgment to perform secondary grouping of line segments in the same group is specifically as follows: the shortest distance between each two line segments is calculated to determine whether they are close; if they are close, they are merged into one group; this process is based on the limit of image pixels, and the Euclidean distance calculation is used point by point for two line segments, and the mean value between points is calculated to determine the degree of spatial proximity; The deduplication process is performed on the line segments in each group, specifically by calculating the Euclidean distance between every two endpoints and selecting the two farthest endpoints for merging, thereby removing redundant line segments.

5. A topological graph structure information processing system based on image recognition according to any one of claims 1 to 4, characterized in that: The element fitting module aggregates the recognized rectangular frame or the recognized text information into a single node, including the following steps: 2.1) Receiving graph structure information: Receive the graph structure information constructed by the graph structure information construction module, use image recognition technology to obtain the four vertex coordinates of the elements in the image, and then use Shapely to extract the vertex coordinates of the elements; 2.2) Generate an extended bounding box: After step 2.1), use shapely's box function to generate an enclosing rectangle to construct a bounding box based on the vertex coordinates of the element; use a geometric algorithm to dynamically adjust the size of the bounding box, adaptively expand it according to the actual size of the element, and finally generate an extended bounding box; 2.3) Neighboring point analysis and extension line generation: After step 2.2), use the spatial geometry analysis method to calculate the distance between the points of the extended bounding box, and determine the number of neighboring points of the extended bounding box. According to the number of neighboring points, it will be divided into three categories: single neighboring point, two neighboring points, and three neighboring points. For the above three categories, use the intersection function provided by Shapely to generate one or more extension lines, and use the intersection points of these extension lines as the nodes of the elements to enter step 2.4). Among them, the specific operations of generating extension lines for each category and using the intersection points of the extension lines as the nodes of the elements are as follows: Single neighboring point: Based on the neighboring point, the midpoint of the intersection of the extended line and the bounding box is used as the node; Two adjacent points: Generate an extension line based on the two points, and use the intersection of the two extension lines as a node; Three adjacent points: Generate extension lines based on the three points, and use the intersection of the three extension lines as nodes; 2.4) Node dynamic adjustment optimization: Generate a circular area of ​​a specified size near the node obtained in step 2.3), and further optimize the node accuracy by dynamically calculating the node that best fits the topological graph structure in the circular area; 2.5) Fit elements to nodes: The nodes obtained in step 2.4) are added to the topological graph structure, and their connection relationships in the topological graph structure are updated. This process is called fitting. In the fitting of text elements, the axis and the height of the text box are combined to correct the annotation position and angle of the text.

6. A topological graph structure information processing system based on image recognition according to any one of claims 1 to 4, characterized in that: When the curved fitting module merges two identified points of different axes into one point, the specific steps are as follows: 3.1) Find single connection point: First, we use topology analysis technology to traverse all nodes in the graph and check the connection relationship of each node. Then, we use the neighbors method in the NetworkX library to quickly identify single connection points that are only connected to one other node. Finally, we combine the properties of the nodes to further screen valid single connection points. 3.2) After the singly connected nodes are identified in step 3.1), the coordinates of these singly connected nodes are calculated pairwise by using the geometric calculation method, and then the calculation results are compared with the set distance threshold, and the point pairs that exceed the range are filtered out to obtain the point pairs that meet the distance threshold; 3.3) Merge points and update connection relationships: For the point pairs that meet the distance threshold, calculate the midpoint of the two points and merge the two points into the midpoint. This midpoint will replace the two points as a new node and update the corresponding connection relationship in the topological graph structure. That is, use the graph operation function in NetworkX to add the merged point as a new node to the graph structure, delete the original node, and reconnect the neighbors of the new node and the original node. 3.4) Loop iteration and real-time update: In step 3.3), after each operation of merging two points into their midpoints, all single connection points are re-traversed through the loop structure to ensure that newly generated single connection points can also be processed in time; then a Boolean variable is used to mark whether the merge occurs. If so, the iteration continues; if no merge operation occurs, the loop is terminated; 3.5) Automatic stop mechanism: Using the while loop control structure, after each iteration, by checking whether there are still single connection points or whether there is a merger, when there is no single connection point or no merger, the loop is exited and the bending fitting is ended. At this point, all eligible nodes in the entire graph structure information have undergone the bending fitting operation.

7. A topological graph structure information processing system based on image recognition according to claim 1, characterized in that: It also includes a dictionary storage module, which stores the generated graph structure data in a dictionary storage manner, and is provided with a dictionary A and a dictionary B, wherein dictionary A is used to store detailed information of each node including coordinates and attributes; dictionary B is used to store the connection relationship between nodes, and is used to represent the topological structure.

8. The topological graph structure information processing system based on image recognition according to claim 1, characterized in that: It also includes a data docking and output module, which is used to convert graph structure data into JSON format and connect with other information management platforms.

Images