An automatic recognition method for switch cabinet terminal diagrams suitable for non-tabular graphics

Abstract

The present invention relates to the field of image recognition technology, and discloses a method for automatic recognition of switch cabinet terminal diagrams applicable to non-tabular graphic elements, comprising the following steps: S1: inputting drawings; S2: preprocessing the terminal diagram, including drawing format conversion, grayscale, binarization, and boundary removal processing, to obtain a preprocessed terminal diagram. The present invention is aimed at the current situation that the operation and maintenance of switch cabinets still rely on manual recognition of paper drawings, and manual recognition of drawings has the problems of taking a long time and having a high error rate. Through the intelligent recognition of terminal diagrams, not only can the time spent on manual recognition of connection relationships be greatly shortened, but the accuracy of recognition can also be improved, thereby ensuring production safety and personal safety. In the development of digital power grids, the digitization of terminal diagrams is achieved by converting the abstract physical information of terminal diagrams into digital information, laying a solid foundation for the successful construction of three-dimensional models.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an automatic recognition method for switch cabinet terminal diagrams applicable to non-table graphic elements. Background Art

[0002] Technologies like digital twins and the metaverse hold immense application value in the power industry, prompting widespread, large-scale 3D modeling of power equipment in recent years. The technology behind building 3D models of power equipment has rapidly evolved from coarse to fine, and from low to high precision. However, to date, most 3D models of power equipment only cover the main structure, such as the equipment's outer casing and large mechanical components. Modeling the tiny components of internal secondary systems remains difficult, particularly secondary cables with small diameters, numerous bends, and complex routing. This is primarily due to the low efficiency, high workload, and poor reusability of cable modeling, resulting in extremely high manual modeling costs and a difficulty meeting the demands of commercial applications.

[0003] To this end, extensive research has been conducted on automated 3D modeling technology based on secondary system wiring diagrams, aiming to enable programmatic generation of secondary cables using wiring diagrams. Wiring diagrams, which are divided into signal loop diagrams and terminal diagrams, serve as reference drawings for secondary system cable wiring during power equipment production. To achieve programmatic generation of 3D models based on wiring diagrams, the problem of automatic drawing recognition must be solved.

[0004] Terminal diagrams primarily consist of tabular and non-tabular terminal elements. Currently, several technical solutions exist for tabular terminal elements, achieving good recognition rates. However, due to issues with non-tabular terminal elements, such as the separation of component numbers from the terminal and circuit components, and the lack of a simple distribution of terminals and circuits as in tabular terminal elements, recognition technology for non-tabular terminal elements is still immature. To address the issue of automatic recognition of non-tabular terminal elements, this patent provides a feasible technical solution. Summary of the Invention

[0005] (1) Technical problems solved

[0006] To address the shortcomings of existing technologies, this invention has devised an automatic recognition technology for secondary drawings of substation switchgear equipment, primarily addressing the problem of extracting terminal connection information from terminal diagrams. This technical solution enables the classification and segmentation of tabular and non-tabular terminal elements, as well as the output of terminal connection relationships. This invention effectively frees up staff productivity and improves the accuracy and efficiency of identifying terminal connection relationships.

[0007] (2) Technical solution

[0008] To achieve the above object, the present invention provides the following technical solution: a method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements, comprising the following steps:

[0009] S1: Input drawings;

[0010] S2: Preprocessing the terminal diagram, including drawing format conversion, grayscale, binarization, and boundary removal, to obtain a preprocessed terminal diagram;

[0011] S3: Classify and segment the pre-processed terminal diagram to obtain a set of table-type terminal graphic primitive images and a set of non-table-type terminal graphic primitive images in the terminal diagram;

[0012] S4: Perform table and text recognition on the table-type terminal graphic element image set, obtain the connection relationship of the table-type terminal graphic elements and fill in the corresponding form;

[0013] S5: Identify the location of each component number in the non-table primitive image set, obtain the text number corresponding to each component through text recognition, and use the text number as the form name of each component;

[0014] S6: Identify the closed box containing the terminals of each component in the non-table graphic element set, and complete the recognition of the terminals inside the closed box and the straight lines connecting the terminals, and use the text recognition method to recognize the terminal number and circuit number; fill the recognized information into the corresponding form;

[0015] S7: Merge the list generated by the table-type graphic element image set and the list generated by the non-table-type graphic element image set to output the overall form of the terminal diagram.

[0016] Preferably, the drawing format conversion, after the format conversion, needs to call the grayscale function of OpenCV to convert the input image into a grayscale image, and use the inverse binary thresholding method to complete the binarization operation. The inverse binary thresholding is to select a specific threshold value thresh, set the pixel value of the point greater than the threshold to 0, and the pixel value less than the threshold to 255. The inverse binary thresholding formula is as follows:

[0017]

[0018] Preferably, a boundary removal module is provided in the preprocessing of the switch cabinet terminal diagram. First, a row structure element of the corrosion operation is designed, and then an opening operation is performed to obtain the horizontal lines in the image. The so-called opening operation is to first perform corrosion to filter out the lines other than the horizontal lines, and then perform expansion, record the position information of the horizontal lines, calculate the length of the horizontal lines, and find the endpoint information of the four horizontal lines with the longest length and the second longest length. The eight endpoint coordinates are the coordinates of the four vertex corners of the two rectangles with the largest boundaries. By traversing the pixel points (i, j) in the image, the pixel values of the pixel points between the two rectangles and on the edge lines of the rectangles are set to 0, thereby removing the interference of the boundary on subsequent recognition.

[0019] Preferably, in said S2, since the characteristics of the two types of graphic elements, tabular and non-tabular, contained in the terminal diagram are quite different, the corresponding recognition methods are also quite different, so the terminal diagram needs to be classified and segmented. First, the pre-processed image is subjected to two corrosion and expansion operations to obtain the horizontal and vertical lines of the entire terminal diagram, and the intersection is obtained according to the intersection of the horizontal and vertical lines. The contour is found by the findContours function of OpenCV. Finally, the area is judged to be a tabular graphic element based on the contour area, the number of intersections and the position characteristics of the intersections. If it is judged to be a table, the approxPolyDP function is used to approximate the area into a shape, and the boundingRect function is used to convert this area into a rectangle. Finally, the coordinate information of the rectangular area is recorded, and the image of each rectangular area is cropped and stored in a set. The other parts of the terminal diagram are non-tabular graphic elements, and the images judged to be non-tabular are stored in another set.

[0020] Preferably, in said S3, the rectangular area of the table-type terminal is first rotated to the positive direction of the text, and then the horizontal lines, vertical lines and intersections of the image are obtained using the same method as in the second step. The positions of the intersections and the positions of the horizontal and vertical lines are then used to identify the cells and the coordinates of the four corners of each cell are recorded. The text image of each cell is used to complete the recognition of the text in the cell using Tesseract to obtain the text content. The so-called Tesseract is an open source framework for text recognition, which has advantages in recognizing English letters and numbers, and meets the requirements for component number, circuit number and terminal number recognition. Finally, the text content is stored in the form in order according to the position information of each cell.

[0021] Preferably, in said S4, since the terminals in some non-tabular graphics exist inside an unclosed graphic, a morphologically based closing operation is performed on the non-tabular terminal area image before terminal recognition, that is, a process of first dilation and then corrosion, to connect the narrow breaks in the non-tabular terminal image set, and then perform Hough circle transform on the image after the closing operation, and obtain the position information (x, y, r) of the circle with the largest radius, where (x, y) is the center of the circle and r is the radius of the circle, and call the minAreaRect function of opencv to obtain the circumscribed rectangle of the extracted largest circle, and use Tesseract to perform text recognition on the image within the circumscribed rectangle, store the text number obtained by each text recognition into an array, and generate multiple blank forms with the text number as the name.

[0022] Preferably, in said S5, the terminal of the non-table-type primitive exists in the closed figure below the largest circle obtained in the fourth step, so it is necessary to extract the closed figure before terminal recognition, and match the largest circle and the closed figure belonging to the same primitive using the characteristic of the shortest distance;

[0023] Use OpenCV's findContours function to identify the outer contours of all closed figures in the image, thereby obtaining the set of all points on the outer contours. Then use the pointPolygonTest function to obtain the distance between the center of the circle obtained in the fourth step and each outer contour. A positive distance value represents the distance between the center of the circle and the outer contour of the largest circle in which the circle is located, and a negative distance value represents the distance to the outer contour of the closed figure other than the circle in which the circle is located. Therefore, it is necessary to delete the positive numbers in the obtained distance values to remove the interference of the outer contour of the circle in which the circle is located.

[0024] Then, a sorting algorithm is used to sort the distance values between each circle center and the outer contour of the closed figure. The outer contour with the largest distance value is found, which is the outer contour of the closed figure closest to each largest circle. The position information of the closed figure is then recorded. Finally, the closed figure needs to be identified to obtain the internal terminal number of the component, the loop number, and the external terminal number of the component connected to the other end of the loop.

[0025] To obtain the terminal numbers inside the component, first identify the small circles and rectangles inside the closed figure and record their location information. If a small circle is identified, take the circumscribed rectangle of the small circle and use Tesseract to perform text recognition on the image inside the circumscribed rectangle. If a small rectangle is identified, directly use Tesseract to perform text recognition on the image inside the small rectangle. Then, fill the number obtained from text recognition, i.e., the terminal number inside the component, into the first column of the corresponding blank form generated in step 4.

[0026] To obtain the circuit number, first remove the shorter straight lines in the drawing. Then, use the position information of each small circle or rectangle to search for the horizontal line closest to it and obtain the position information of the horizontal line. Take a rectangular area above the horizontal line, where the horizontal line is the length of the rectangle and the height of the character is the width of the rectangle. Finally, use Tesseract to perform text recognition on the image of the taken rectangle. The number obtained by text recognition, i.e., the circuit number, is filled in the second column of the corresponding blank form generated in the fourth step.

[0027] To obtain the component external terminal number; use the horizontal line information obtained when obtaining the loop number to take a rectangle containing the component external terminal number, where the horizontal line away from the endpoint of the smallest circle or the smallest rectangle is the position of the rectangle's width, the height of the character in the vertical direction is taken as the rectangle's width, and a certain value in the horizontal direction is taken as the rectangle's length; finally, use the image of the taken rectangle for text recognition; fill in the number obtained by text recognition, i.e., the component external terminal number, into the third column of the corresponding blank form generated in the fourth step.

[0028] Preferably, in said S5, after S1-S5 are completed, a connection relationship table corresponding to the components in the order of "component internal terminal number - loop number - external component serial number: terminal number" can be obtained.

[0029] Preferably, in said S6, since the recording methods of the tabular terminal form obtained in the third step and the non-tabular terminal form obtained in the fifth step are different, some adjustments need to be made to the forms before merging them;

[0030] The form obtained in the third step is simply arranged according to the table content, namely "upper external component serial number: terminal number - disconnector terminal number - loop number - lower external component serial number: terminal number". The corresponding row of content is expected to be arranged in two rows, namely "loop number - disconnector terminal number - 1 - upper external component number - upper external terminal number / loop number - disconnector terminal number - 2 - lower external component number - lower external terminal number". Among them, the loop number and terminal number can be obtained directly in the obtained form, while the upper and lower external component numbers cannot be obtained directly. They need to be retrieved using the component serial number in the array containing all text numbers obtained in the fourth step. After completing the above process, a complete table of terminal to be merged is obtained.

[0031] The target form layout for non-tabular terminal connection relationships is "circuit number - component number - component internal terminal number - external component number - external component terminal number." Therefore, you first need to add a table row before the first row of each form obtained in step 5 and fill in the name of each form in the table. The resulting form layout is "component number - component internal terminal number - circuit number - external component serial number: terminal number." Among them, the component number, component internal terminal number, circuit number, and external component terminal number can all be filled in directly, but the external component number cannot be directly obtained and must be retrieved from the array containing all text numbers obtained in step 4 using the component serial number. After completing the above process, a complete non-tabular terminal form to be merged is obtained.

[0032] Finally, merge the table terminals and non-table terminals to be merged that have the same arrangement order to obtain a complete connection relationship table.

[0033] (3) Beneficial effects

[0034] Compared with the prior art, the present invention provides a method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements, which has the following beneficial effects:

[0035] This method, which is applicable to non-tabular graphics, is designed to automatically identify switchgear terminal diagrams. This method addresses the current situation where switchgear operation and maintenance still rely on manual identification of paper drawings, which is time-consuming and has a high error rate. Intelligent identification of terminal diagrams can not only greatly shorten the time spent on manual identification of connection relationships, but also improve the accuracy of identification, thereby ensuring production safety and personal safety. In the development of digital power grids, the digitization of terminal diagrams is achieved by converting the abstract physical information of terminal diagrams into digital information, laying a solid foundation for the successful construction of three-dimensional models. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a step diagram of the present invention;

[0037] Figure 2 This is the terminal diagram of the present invention;

[0038] Figure 3 It is the boundary removal map of the present invention;

[0039] Figure 4 This is the terminal diagram after preprocessing of the present invention;

[0040] Figure 5 A generated diagram of a blank form of a non-table graphic element of the present invention;

[0041] Figure 6 This is the outer contour identification diagram of the present invention;

[0042] Figure 7 This is an acquisition diagram of the closed figure closest to the element "1A" of the present invention;

[0043] Figure 8 This is a diagram of the "1A" component circuit numbering area selection of the present invention;

[0044] Figure 9 This is a diagram of the external terminal numbering area of the "1A" component of the present invention;

[0045] Figure 10 This is a diagram of the conversion of the "1A" element of the present invention into a table;

[0046] Figure 11 It is a table-type graphic element form adjustment diagram of the present invention;

[0047] Figure 12 This is the adjustment diagram of the "1A" component form of the present invention. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0049] Examples of embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0050] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0051] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0052] like Figure 1-12 As shown, the present invention provides a technical solution: a method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements, comprising the following steps:

[0053] S1: Input drawings;

[0054] S2: Preprocessing the terminal diagram, including drawing format conversion, grayscale, binarization, and boundary removal, to obtain a preprocessed terminal diagram;

[0055] S3: Classify and segment the pre-processed terminal diagram to obtain a set of table-type terminal graphic primitive images and a set of non-table-type terminal graphic primitive images in the terminal diagram;

[0056] S4: Perform table and text recognition on the table-type terminal graphic element image set, obtain the connection relationship of the table-type terminal graphic elements and fill in the corresponding form;

[0057] S5: Identify the location of each component number in the non-table primitive image set, obtain the text number corresponding to each component through text recognition, and use the text number as the form name of each component;

[0058] S6: Identify the closed box containing the terminals of each component in the non-table graphic element set, and complete the recognition of the terminals inside the closed box and the straight lines connecting the terminals, and use the text recognition method to recognize the terminal number and circuit number; fill the recognized information into the corresponding form;

[0059] S7: Merge the list generated by the table-type graphic element image set and the list generated by the non-table-type graphic element image set to output the overall form of the terminal diagram.

[0060] Furthermore, the drawing format is converted. After the format conversion, the terminal diagram needs to call the grayscale function of OpenCV to convert the input image into a grayscale image, and use the inverse binary thresholding method to complete the binarization operation. Inverse binary thresholding is to select a specific threshold value thresh, set the pixel value of the point greater than the threshold to 0, and the pixel value less than the threshold to 255. The inverse binary thresholding formula is as follows:

[0061]

[0062] Furthermore, a boundary removal module is provided in the preprocessing of the switch cabinet terminal diagram. First, the row structure element of the corrosion operation is designed, and then the opening operation is performed to obtain the horizontal lines in the image. The so-called opening operation is to first perform corrosion to filter out the lines other than the horizontal lines, and then perform expansion, record the position information of the horizontal lines, calculate the length of the horizontal lines, and find the endpoint information of the four horizontal lines with the longest and second longest lengths. The eight endpoint coordinates are the coordinates of the four vertex corners of the two rectangles with the largest boundaries. By traversing the pixel points (i, j) in the image, the pixel values of the pixel points between the two rectangles and on the edge lines of the rectangles are set to 0, thereby removing the interference of the boundary on subsequent recognition.

[0063] Furthermore, in S2, since the characteristics of the two types of graphic elements, tabular and non-tabular, contained in the terminal diagram are quite different, the corresponding recognition methods are also very different, so the terminal diagram needs to be classified and segmented. First, the preprocessed image is subjected to two corrosion and expansion operations to obtain the horizontal and vertical lines of the entire terminal diagram. The intersection is obtained based on the intersection of the horizontal and vertical lines, and the contour is found through OpenCV's findContours function. Finally, the area is judged to be a tabular graphic element based on the contour area, the number of intersections, and the position characteristics of the intersections. If it is judged to be a table, the approxPolyDP function is used to approximate the area into a shape, and the boundingRect function is used to convert this area into a rectangle. Finally, the coordinate information of the rectangular area is recorded, and the image of each rectangular area is cropped and stored in a set. The other parts of the terminal diagram are non-tabular graphic elements, and the images judged to be non-tabular are stored in another set.

[0064] Furthermore, in S3, the table-type terminal rectangular area is first rotated to the positive direction of the text, and then the horizontal lines, vertical lines and intersections of the image are obtained using the same method as the second step. The intersection positions and the positions of the horizontal and vertical lines are then used to identify the cells and record the coordinates of the four corners of each cell. The text image of each cell is used to use Tesseract to complete the recognition of the text in the cell and obtain the text content. The so-called Tesseract is an open source framework for text recognition, which has advantages in recognizing English letters and numbers, and meets the needs of component number, circuit number and terminal number recognition. Finally, the text content is stored in the form in order according to the position information of each cell.

[0065] Furthermore, in S4, since the terminals in some non-tabular graphics exist inside unclosed graphics, the non-tabular terminal area image must be subjected to a morphological closing operation before terminal recognition, that is, a process of first dilation and then corrosion, to connect the narrow breaks in the non-tabular terminal image set, and then perform Hough circle transform on the image after the closing operation, and obtain the position information (x, y, r) of the circle with the largest radius, where (x, y) is the center of the circle and r is the radius of the circle. The minAreaRect function of opencv is called to obtain the circumscribed rectangle of the extracted largest circle, and the image within the circumscribed rectangle is used for text recognition using Tesseract. The text number obtained by each text recognition is stored in an array, and multiple blank forms are generated with the text number as the name.

[0066] Furthermore, in S5, the terminals of non-table primitives exist in the closed figure below the largest circle obtained in the fourth step. Therefore, before terminal recognition, the closed figure needs to be extracted and the largest circle and the closed figure belonging to the same primitive are matched using the shortest distance feature.

[0067] Use OpenCV's findContours function to identify the outer contours of all closed figures in the image, thereby obtaining the set of all points on the outer contours. Then use the pointPolygonTest function to obtain the distance between the center of the circle obtained in the fourth step and each outer contour. A positive distance value represents the distance between the center of the circle and the outer contour of the largest circle in which the circle is located, and a negative distance value represents the distance to the outer contour of the closed figure other than the circle in which the circle is located. Therefore, it is necessary to delete the positive numbers in the obtained distance values to remove the interference of the outer contour of the circle in which the circle is located.

[0068] Then, a sorting algorithm is used to sort the distance values between each circle center and the outer contour of the closed figure. The outer contour with the largest distance value is found, which is the outer contour of the closed figure closest to each largest circle. The position information of the closed figure is then recorded. Finally, the closed figure needs to be identified to obtain the internal terminal number of the component, the loop number, and the external terminal number of the component connected to the other end of the loop.

[0069] To obtain the terminal numbers inside the component, first identify the small circles and rectangles inside the closed figure and record their location information. If a small circle is identified, take the circumscribed rectangle of the small circle and use Tesseract to perform text recognition on the image inside the circumscribed rectangle. If a small rectangle is identified, directly use Tesseract to perform text recognition on the image inside the small rectangle. Then, fill the number obtained from text recognition, i.e., the terminal number inside the component, into the first column of the corresponding blank form generated in step 4.

[0070] To obtain the circuit number, first remove the shorter straight lines in the drawing. Then, use the position information of each small circle or rectangle to search for the horizontal line closest to it and obtain the position information of the horizontal line. Take a rectangular area above the horizontal line, where the horizontal line is the length of the rectangle and the height of the character is the width of the rectangle. Finally, use Tesseract to perform text recognition on the image of the taken rectangle. The number obtained by text recognition, i.e., the circuit number, is filled in the second column of the corresponding blank form generated in the fourth step.

[0071] To obtain the component external terminal number; use the horizontal line information obtained when obtaining the loop number to take a rectangle containing the component external terminal number, where the horizontal line away from the endpoint of the smallest circle or the smallest rectangle is the position of the rectangle's width, the height of the character in the vertical direction is taken as the rectangle's width, and a certain value in the horizontal direction is taken as the rectangle's length; finally, use the image of the taken rectangle for text recognition; fill in the number obtained by text recognition, i.e., the component external terminal number, into the third column of the corresponding blank form generated in the fourth step.

[0072] Furthermore, in S5, after S1-S5 are completed, a connection relationship table corresponding to the components in the order of "component internal terminal number-circuit number-external component serial number: terminal number" can be obtained.

[0073] Furthermore, in S6, since the recording methods of the tabular terminal form obtained in the third step and the non-tabular terminal form obtained in the fifth step are different, some adjustments need to be made to the forms before merging them;

[0074] The form obtained in the third step is simply arranged according to the table content, namely "upper external component serial number: terminal number - disconnector terminal number - loop number - lower external component serial number: terminal number". The corresponding row of content is expected to be arranged in two rows, namely "loop number - disconnector terminal number - 1 - upper external component number - upper external terminal number / loop number - disconnector terminal number - 2 - lower external component number - lower external terminal number". Among them, the loop number and terminal number can be obtained directly in the obtained form, while the upper and lower external component numbers cannot be obtained directly. They need to be retrieved using the component serial number in the array containing all text numbers obtained in the fourth step. After completing the above process, a complete table of terminal to be merged is obtained.

[0075] The target form layout for non-tabular terminal connection relationships is "circuit number - component number - component internal terminal number - external component number - external component terminal number." Therefore, you first need to add a table row before the first row of each form obtained in step 5 and fill in the name of each form in the table. The resulting form layout is "component number - component internal terminal number - circuit number - external component serial number: terminal number." Among them, the component number, component internal terminal number, circuit number, and external component terminal number can all be filled in directly, but the external component number cannot be directly obtained and must be retrieved from the array containing all text numbers obtained in step 4 using the component serial number. After completing the above process, a complete non-tabular terminal form to be merged is obtained.

[0076] Finally, merge the table terminals and non-table terminals to be merged that have the same arrangement order to obtain a complete connection relationship table.

[0077] A best embodiment is provided for this technical solution:

[0078] like Figure 2 Taking the terminal diagram shown as an example, it includes a table-type element and three non-table-type elements 1A, 2B, and 3C. The table-type element contains 20 terminals, and the three elements 1A, 2B, and 3C each have n terminals, numbered 1, 2, 3, ..., n.

[0079] The first step of format conversion, grayscale, and binarization can be achieved through existing algorithms or tools. The boundary removal part first identifies the four longest and second longest horizontal lines to obtain the endpoint positions of the two largest rectangles (x1, y1), (x1, y2), (x2, y1), and (x2, y2). The longest horizontal line identified is Figure 3 The second longest horizontal line is marked with a lead line in Figure 2 Then, the pixel values are set to 0 to remove the interference between the boundaries of the two rectangles and the pixels between the rectangles. The area where the pixels are set to 0 is Figure 2 The shaded part in the figure. The terminal diagram after pre-processing is as follows Figure 4 shown.

[0080] The second step uses line recognition, erosion, expansion, and intersection recognition algorithms to identify lines and intersections and obtain their location information. The location information, number of intersections, and cell area are then used to determine the table primitive image unit. This completes the classification and segmentation of both table and non-table primitives.

[0081] The fourth step in this example is to identify the three largest circles that contain the component numbers of non-table graphics elements and take the circumscribed rectangle of each largest circle. Then use Tesseract to identify the component numbers "1A", "2B", and "3C" in the rectangular area image and store them in an array, and generate three blank forms named "1A", "2B", and "3C" respectively. Figure 5 shown.

[0082] The fifth step is to use the findContours function to identify the outer contours of all closed figures in the image. The recognition results are as follows: Figure 6 As shown by the dark lines in the middle, the pointPolygonTest function is then used to obtain the distance between the center of the circle obtained in the fourth step and each outer contour, and the closed figure closest to each large circle is found and the position information is recorded. Taking component 1A as an example, the coordinates of the center of the circle obtained in the fourth step are (xA, yA). The distances between the center of the circle and the six outer contours are calculated respectively, and a positive number (distance1) and five negative numbers (distance2, distance3, distance4, distance5, distance6) are obtained. The five negative numbers are sorted and the largest negative number is found, i.e., distance2, so as to find the outer contour of the closed figure closest to the circle where the number "1A" is located. Figure 7 As shown by lead 1 in the figure, the position information of the closed figure is recorded. Then the small circles and small rectangles inside the closed figure are identified and the position information is recorded. It can be seen that the terminals of the "1A" and "3C" components are inside the small circles, and the terminals of the "2B" component are inside the small rectangles. The circumscribed rectangles are taken for the identified small circles, and the rectangular images of the identified small rectangles are retained. Finally, all the obtained rectangular images are subjected to text recognition using Tesseract, and the internal terminal numbers of the "1A", "2B", and "3C" components are identified, and the internal terminal numbers of the components are filled in. Figure 5 The first column of the corresponding blank form.

[0083] Then, the interference of shorter horizontal lines is eliminated, and the horizontal line closest to the small circle or small rectangle is identified. The horizontal line is taken as the length of the rectangle and the character height is taken as the width. A rectangular area is taken above the horizontal line. Tesseract is used to perform text recognition on the rectangular area. The text result obtained is the circuit number. Taking the "1A" component as an example, the horizontal line closest to the small circle where the terminals "1" and "2" are located is as follows: Figure 8 As shown by the middle lead line 3, a rectangular area with the length of the horizontal line as the length and the height of the character as the width is made on the two horizontal lines. The selection of the rectangular area is as follows: Figure 8As shown by lead 2 in the middle. After identification, it can be concluded that the circuit number corresponding to terminal "1" is "a", and the circuit number corresponding to terminal "2" is "c". Fill in all the identified circuit numbers Figure 5 The second column of the corresponding blank form.

[0084] Finally, a rectangular area is taken at the end of the horizontal line away from the internal terminal and the rectangular area image is used for text recognition using Tesseract. In the vertical direction, the height of the character is the width of the rectangle, and in the horizontal direction, a certain value is taken as the length of the rectangle. Taking the "1A" component as an example, a rectangular area is taken at the left end of the line where the loop "a" is located. Figure 9 As shown in the figure, the external terminal number obtained after identification is "2:B-1". Fill in all the external terminal numbers identified Figure 4 The corresponding third column of the blank form.

[0085] Taking the "1A" component as an example, the form obtained after the fifth step of recognition is as follows Figure 10 shown.

[0086] Step 6: The table terminals to be merged are obtained according to the technical solution. Figure 11 As shown, the adjusted non-table terminal to be merged table takes the "1A" component as an example. Figure 12 As shown in the figure, simply merge all the components' tables to be merged to get the complete connection relationship table.

[0087] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further restrictions, an element defined by the statement "comprising a reference structure" does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

Claims

1. A method for automatically identifying switch cabinet terminal diagrams suitable for non-table graphic elements, characterized in that: The following steps are involved: S1: Input drawings; S2: Preprocessing the terminal diagram, including drawing format conversion, grayscale, binarization, and boundary removal, to obtain a preprocessed terminal diagram; S3: Classify and segment the pre-processed terminal diagram to obtain a set of table-type terminal graphic primitive images and a set of non-table-type terminal graphic primitive images in the terminal diagram; S4: Perform table and text recognition on the table-type terminal graphic element image set, obtain the connection relationship of the table-type terminal graphic elements and fill in the corresponding form; S5: Identify the location of each component number in the non-tabular primitive image set, and obtain the text number corresponding to each component through text recognition. Use the text number as the form name of each component, perform a morphological closing operation on the non-tabular terminal area image, and then perform Hough circle transform on the image after the closing operation to obtain the position information (x, y, r) of the circle with the largest radius, where (x, y) is the center of the circle and r is the radius of the circle. Call the minAreaRect function of OpenCV to obtain the circumscribed rectangle of the extracted largest circle, and use Tesseract to perform text recognition on the image within the circumscribed rectangle; S6: Before terminal recognition, the closed graphics need to be extracted, and the largest circle and the closed graphics belonging to the same graphic element are matched using the shortest distance feature, and the closed frame containing the terminal of each component in the non-tabular graphic element set is identified, and the terminals inside the closed frame and the straight lines connecting the terminals are identified, and the terminal numbers and loop numbers are identified using the text recognition method; the identified information is filled in the corresponding form, and the terminals of the non-tabular graphic element exist in the closed graphics below the largest circle obtained in the fourth step. For the acquisition of the terminal numbers inside the components, first, the small circles and small rectangles inside the closed graphics need to be identified and the position information recorded; if a small circle is identified, the circumscribed rectangle of the small circle is taken, and the image inside the circumscribed rectangle is used for text recognition by Tesseract; if a small rectangle is identified, the image inside the small rectangle is directly used for text recognition. Tesseract performs text recognition; for obtaining the circuit number; first remove the shorter straight lines in the drawing, then use the position information of each small circle or small rectangle to search for the horizontal line closest to it and obtain the position information of the horizontal line; take a rectangular area above the horizontal line, where the horizontal line is the length of the rectangle and the height of the character is the width of the rectangle; finally, use Tesseract to perform text recognition on the image of the taken rectangle; for obtaining the external terminal number of the component; use the horizontal line information obtained when obtaining the circuit number to take the rectangle containing the external terminal number of the component, where the horizontal line away from the end point of the smallest circle or smallest rectangle is the position of the width of the rectangle, take the height of the character in the vertical direction as the width of the rectangle, and take the set value in the horizontal direction as the length of the rectangle; finally, use the image of the taken rectangle to perform text recognition; S7: Merge the list generated by the table-type graphic element image set and the list generated by the non-table-type graphic element image set to output the overall form of the terminal diagram.

2. The method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements according to claim 1, characterized in that: The drawing format conversion, the terminal diagram after format conversion also needs to call OpenCV's grayscale function to convert the input image into a grayscale image, and use the inverse binary thresholding method to complete the binarization operation. Inverse binary thresholding is to select a specific threshold value thresh, set the pixel value of the point greater than the threshold to 0, and the pixel value less than the threshold to 255. The inverse binary thresholding formula is as follows:

3. The method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements according to claim 1, characterized in that: A boundary removal module is provided in the preprocessing of the switch cabinet terminal diagram. First, a row structure element of the corrosion operation is designed, and then an opening operation is performed to obtain the horizontal lines in the image. The so-called opening operation is to first perform corrosion to filter out the lines other than the horizontal lines, and then perform expansion, record the position information of the horizontal lines, calculate the length of the horizontal lines, and find the endpoint information of the four horizontal lines with the longest length and the second longest length. The eight endpoint coordinates are the coordinates of the four vertex corners of the two rectangles with the largest boundaries. By traversing the pixel points (i, j) in the image, the pixel values of the pixel points between the two rectangles and on the edge lines of the rectangles are set to 0, thereby removing the interference of the boundary on subsequent recognition.

4. The method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements according to claim 1, characterized in that: In S3, the terminal diagram is classified and segmented. First, the preprocessed image is subjected to two corrosion and expansion operations to obtain the horizontal and vertical lines of the entire terminal diagram. The intersection is obtained based on the intersection of the horizontal and vertical lines, and the contour is found through OpenCV's findContours function. Finally, the area is judged to be a table-type primitive based on the contour area, the number of intersections, and the position characteristics of the intersections. If it is judged to be a table, the approxPolyDP function is used to approximate the area into a shape, and the boundingRect function is used to convert this area into a rectangle. Finally, the coordinate information of the rectangular area is recorded, and the images of each rectangular area are cropped and stored in a set. The other parts of the terminal diagram are non-table-type primitives, and the images judged to be non-table-type are stored in another set.

5. The method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements according to claim 1, characterized in that: In the above S4, the rectangular area of the table terminal is first rotated to the positive direction of the text, and then the horizontal lines, vertical lines and intersections of the image are obtained using the same method as the second step. The positions of the intersections and the positions of the horizontal and vertical lines are then used to identify the cells and record the coordinates of the four corners of each cell. The text image of each cell is used to use Tesseract to complete the recognition of the text in the cell and obtain the text content. Finally, the text content is stored in the form in order according to the position information of each cell.

6. The method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements according to claim 1, characterized in that: In S5, since some terminals in non-tabular graphics exist inside unclosed graphics, before terminal recognition, the text number obtained by each character recognition must be stored in an array, and multiple blank forms must be generated with the text number as the name.

7. The method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements according to claim 1, characterized in that: In S6, the findContours function of OpenCV is used to identify the outer contours of all closed figures in the image, thereby obtaining the set of all points on the outer contours. Then, the pointPolygonTest function is used to obtain the distance between the center of the circle obtained in the fourth step and each outer contour. A positive distance value represents the distance between the center of the circle and the outer contour of the largest circle in which the circle is located, and a negative distance value represents the distance to the outer contour of the closed figure other than the circle in which the circle is located. Therefore, it is necessary to delete the positive numbers in the obtained distance values to remove the interference of the outer contour of the circle in which the circle is located. Then, a sorting algorithm is used to sort the distance values between each circle center and the outer contour of the closed figure. The outer contour with the largest distance value is found, which is the outer contour of the closed figure closest to each largest circle. The position information of the closed figure is then recorded. Finally, the closed figure needs to be identified to obtain the internal terminal number of the component, the loop number, and the external terminal number of the component connected to the other end of the loop. Then fill the number obtained by text recognition, that is, the internal terminal number of the component, into the first column of the corresponding blank form generated in the fourth step; Fill the number obtained by text recognition, i.e. the circuit number, into the second column of the corresponding blank form generated in step 4; Fill in the number obtained from text recognition, i.e. the component external terminal number, into the third column of the corresponding blank form generated in the fourth step.

8. The method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements according to claim 1, characterized in that: In the above S6, after S1-S5 are completed, a connection relationship table corresponding to the components in the order of "component internal terminal number - circuit number - external component serial number: terminal number" can be obtained.

9. The method for automatically identifying switch cabinet terminal diagrams applicable to non-table graphic elements according to claim 1, characterized in that: In the above S7, since the recording methods of the tabular terminal form obtained in the third step and the non-tabular terminal form obtained in the fifth step are different, some adjustments need to be made to the forms before merging them; The form obtained in the third step is simply arranged according to the table content, namely "upper external component serial number: terminal number - disconnector terminal number - loop number - lower external component serial number: terminal number". The corresponding row of content is expected to be arranged in two rows, namely "loop number - disconnector terminal number - 1 - upper external component number - upper external terminal number / loop number - disconnector terminal number - 2 - lower external component number - lower external terminal number". Among them, the loop number and terminal number are directly obtained from the obtained form, while the upper and lower external component numbers cannot be obtained directly. They need to be retrieved from the array containing all text numbers obtained in the fourth step using the component serial number. After completing the above process, a complete table of terminal to be merged is obtained. The target form layout for non-tabular terminal connection relationships is "circuit number - component number - component internal terminal number - external component number - external component terminal number." Therefore, you first need to add a table row before the first row of each form obtained in step 5 and fill in the name of each form in the table. The resulting form layout is "component number - component internal terminal number - circuit number - external component serial number: terminal number." The component number, component internal terminal number, circuit number, and external component terminal number can all be filled in directly, but the external component number cannot be directly obtained and must be retrieved using the component serial number from the array containing all text numbers obtained in step 4. This process will yield a complete form for non-tabular terminals to be merged. Finally, merge the table terminals and non-table terminals to be merged that have the same arrangement order to obtain a complete connection relationship table.

Images