CNN-based pressure pipeline single line diagram feature recognition method

Abstract

The invention relates to the field of pressure pipeline engineering technology and the like, and discloses a CNN-based pressure pipeline single line diagram feature recognition method, which comprises the following steps: accurately recognizing and extracting the position and type of a pipe fitting through a target detection technology; automatically capturing text information such as labels and annotations by applying an OCR (Optical Character Recognition) technology, wherein the text information comprises coordinates and specific contents; meanwhile, key features such as directions and coordinates of pipelines are accurately detected and extracted by utilizing edge detection and a Hough transform algorithm; comprising the following specific steps: importing a pressure pipeline single line diagram image into an image denoising module and carrying out denoising processing; inputting the de-noised image into a text recognition module to realize extraction processing of text contents to obtain contents and coordinates of all texts in the image; carrying out key information extraction on the output of the previous two steps by utilizing a YOLOX pipe fitting detection module; then, performing data processing through edge detection, Hough transform and expansion operation by using a pipeline identification module; and finally, normalizing the output of different modules.

Description

Technical Field

[0001] The invention relates to the fields of pressure pipeline engineering technology, and in particular to a pressure pipeline single-line diagram feature recognition method based on CNN. Background Art

[0002] At present, the traditional method of making a single-line diagram of a pressure pipeline usually involves hand-drawing on site, and then using software to convert the hand-drawn drawings into digital format, which is inefficient. When these single-line diagrams need to be modified, it is usually necessary to find the corresponding files and then edit or redraw them through software. In addition, the various element features in the single-line diagram of the pressure pipeline still need to be output through manual processing. This process is not only cumbersome, but also has a certain degree of subjectivity. However, with the rapid development of image recognition technology, especially the convolutional neural network-based algorithm in deep learning, various element features in the image can be more accurately identified. For example, optical character recognition (OCR) technology can extract text and its spatial coordinates from an image, while target detection technology can identify the type of pipe fittings in the image and its precise position. Hough transform detection can detect straight lines and circles in the image. These image recognition technologies are widely used in various fields.

[0003] The existing method for making a single-line diagram of a pressure pipeline has the following problems: 1. Low modification efficiency: When modifying the pressure piping single-line diagram, the traditional method requires first locating the corresponding design file, and then manually editing it through professional software, which is inefficient.

[0004] 2. High work redundancy: If the source file is unfortunately lost or damaged, the traditional approach is to re-refer to the existing picture data and use the design software to draw the pressure piping single-line diagram from scratch.

[0005] 3. Information extraction is cumbersome: In the traditional way, it is cumbersome to extract text coordinates, text content, pipe coordinates, pipe type, pipeline direction, pipeline coordinates and other features from the pressure pipeline single-line diagram, which limits further application and analysis. In addition, the output process through manual processing is subjective and prone to errors. Summary of the invention

[0006] The purpose of the present invention is to provide a CNN-based pressure pipeline single-line diagram feature recognition method, which accurately identifies and extracts the location and type of pipe fittings through target detection technology; applies OCR technology to automatically capture text information such as labels and annotations, including their coordinates and specific content; at the same time, uses edge detection and Hough transform algorithms to accurately detect and extract key features such as the direction and coordinates of the pipeline.

[0007] The present invention is implemented by the following technical solution: A method for identifying features of a pressure pipeline single-line diagram based on CNN, comprising the following specific steps: 1) Import the pressure pipeline single-line diagram image into the image denoising module and perform denoising processing to obtain the denoised image; 2) Input the denoised image into the text recognition module, extract the text content through the text detection algorithm and text recognition algorithm, and obtain the content and coordinates of all texts in the image; 3) The output of step 1) and step 2) are used to extract key information using the YOLOX pipe fitting detection module, and the type and coordinates of each pipe fitting in the image are output; 4) After step 3), the pipeline identification module uses one or more of edge detection, Hough transform and dilation operations to process the data and obtain the directions and coordinates of all pipelines in the image; 5) Finally, the outputs of the text recognition module, YOLOX pipe detection module, and pipeline recognition module are normalized.

[0008] In order to better realize the CNN-based pressure pipeline single-line diagram feature recognition method described in the present invention, the following setting method is particularly adopted: the step 1) is specifically: the pressure pipeline single-line diagram to be identified is used as input (the beginning of the whole process), imported into the image denoising module, and the Gaussian filtering method is used to reduce the image noise, that is, some abnormal points that may exist in the image are removed to obtain a denoised image. By image denoising, the real details are highlighted to provide a clearer input for subsequent modules.

[0009] In order to better implement the CNN-based pressure pipeline single-line diagram feature recognition method of the present invention, the following setting is particularly adopted: the step 2) includes the following steps: 2.1) Resize the denoised image and input it into the text detection algorithm DBNet of the text recognition module. The coordinates of the text in the image are output, including the coordinates of the upper left corner and lower right corner of all the text in the image, and the coordinates are stored in a two-dimensional array. That is, the text detection algorithm DBNet will adjust the image size to a fixed size before calculation, and then input the fixed-size image into the algorithm. The algorithm will output the coordinates of the text in the image. The text coordinates are the upper left and lower right coordinates of the text after the rectangular box is selected. The detection algorithm DBNet will output the upper left and lower right coordinates of all the text in the image, and store the coordinates in a two-dimensional array.

[0010] 2.2) After obtaining the coordinates of all the text in the image, crop the original image according to the coordinates of the upper left corner and lower right corner of each text. Each cropped sub-image represents a line of text. 2.3) Since some text sub-images may be tilted, the sub-images with tilt problems are first preset as horizontal sub-images, and a mapping relationship between tilt and horizontal is established. Then, the sub-images with tilt problems are adjusted to a horizontal state using an affine transformation method according to the mapping relationship. 2.4) After all horizontal text sub-images are adjusted to a fixed image size, they are input into the text recognition model CRNN of the text recognition module. The CRNN algorithm will classify each character in the sub-image and output the recognition result of each character. The result of each sub-image is the collection of all the character recognition results it contains. 2.5) The coordinates of the upper left corner and lower right corner of each text in the original image and the corresponding text recognition results are combined and output, thus obtaining the content and coordinates of all the text in the image.

[0011] In order to better realize the CNN-based pressure pipeline single-line diagram feature recognition method of the present invention, the following setting is particularly adopted: the step 3) includes the following specific steps: 3.1) Input the output of step 1) and step 2) into the YOLOX pipe inspection module. First, use the denoised image and the results obtained by the text recognition module to fill each text area (a rectangle formed by the upper left corner coordinates and the lower right corner coordinates) with the background color according to the coordinates of each text obtained by text recognition, so as to obtain an image after removing the text on the basis of the original image. 3.2) Then, the image after removing the text is adjusted to a fixed image size and then input into the object detection algorithm YOLOX to detect the pipes in the image, and output the type and coordinates (upper left and lower right coordinates) of each pipe in the image.

[0012] In order to further better realize the CNN-based pressure pipeline single-line diagram feature recognition method described in the present invention, the following setting method is particularly adopted: in the step 3.1), the text area is a rectangle formed by the upper left corner coordinates and the lower right corner coordinates of each text; the coordinates of each pipe fitting include two coordinates of the upper left corner and the lower right corner.

[0013] In order to better realize the pressure pipeline single-line diagram feature recognition method based on CNN described in the present invention, the following setting is particularly adopted: the step 4) includes the following specific steps: 4.1) First, the coordinates of each pipe are obtained according to the pipe detection, that is, each pipe area in the original image (a rectangle formed by the coordinates of the upper left corner and the lower right corner), and each pipe area is filled with the background color; 4.2) After step 4.1), an image is obtained by first removing the text area and then removing the pipe area based on the original image; 4.3) Then, the image is processed by edge detection Canny operator, and the grayscale image obtained has a black background and white line segments, i.e., convex line segment information; 4.4) After step 4.3), the Hough transform method is used for the first time to detect the line segment, and the Hough transform method is used to fit the straight line in the figure. The fitted straight line is represented by the coordinates of the two ends; 4.5) After step 4.4), fill the line segment with the background color according to the coordinates of both ends of the line segment; 4.6) There may be a small number of undetected line segments in the filled image. Then, the dilation operation is used to enhance the features of the remaining undetected line segments in the image after step 4.5). The contour detection method is then used to obtain the line segment area in the image (the edge coordinates of each area). Some abnormal points (noise) are filtered out by calculating the number of pixel values ​​in each area. 4.7) Using the second Hough transform method to detect the remaining line segments on the image processed in step 4.6), and obtain the coordinates of the two ends of each line segment; 4.8) Then, the coordinate information of the line segments detected by the first and second Hough transform methods are combined to obtain the coordinate information of all line segments; 4.9) For each line segment, we have two end coordinates. We can use the two end coordinates to calculate the slope of the line segment, and then use the inverse tangent function to calculate the angle between the line segment and the horizontal direction, that is, the direction in the figure.

[0014] In order to better implement the CNN-based pressure pipeline single-line diagram feature recognition method described in the present invention, the following setting is particularly adopted: in the step 4.6), the contour detection method is used to obtain the line segment area in the figure, specifically, the contour detection method is used to obtain the edge coordinates of each area of ​​the line segment area.

[0015] In order to better realize the CNN-based pressure pipeline single-line diagram feature recognition method of the present invention, the following setting is particularly adopted: the step 5) is specifically: For the output results of the text recognition module, output the coordinates of the upper left corner and lower right corner of each line of text, as well as the corresponding text content; For the output results of the YOLOX pipe inspection module, the type, upper left corner coordinates and lower right corner coordinates of each pipe are output; For the output results of the pipeline identification module, the coordinates and directions of both ends of each line segment are output; The output results of the text recognition module, YOLOX pipe detection module and pipeline identification module are merged in JSON format, that is, different keys are used to distinguish different modules, and the value corresponding to each key is the output result of each module.

[0016] Compared with the prior art, the present invention has the following advantages and beneficial effects: The present invention uses edge detection, Hough transform, and image algorithms based on convolutional neural networks to identify the features of text, pipe fittings, pipelines and other elements in the pressure pipeline single-line diagram (including text coordinates, text content, pipe fitting coordinates, pipe fitting type, pipeline direction, pipeline coordinates). By solving the problem of being unable to edit directly due to missing source files, and only retaining the original pictures without being able to modify them effectively, the present invention allows users to easily import previous original pictures for feature recognition. This process can extract six major categories of key features in the picture, and users can import these recognition results into professional software for further editing and operation. It not only improves editing efficiency, but also simplifies the operation process, bringing intelligence and convenience to pipeline design and maintenance.

[0017] The present invention can also develop customized image recognition models according to the needs of different industries to adapt to the single-line diagram production requirements in different fields and improve the degree of customization and adaptability.

[0018] The present invention automatically extracts six types of features in the pressure pipeline single-line diagram, including text coordinates, text content, pipe coordinates, pipe type, pipeline direction, and pipeline coordinates, through an AI algorithm, which greatly improves work efficiency and can effectively avoid the risks of manual processing.

[0019] The results of the pressure piping single-line diagram AI algorithm recognition used in this invention are highly flexible and can adjust the output format according to needs, making the information easier to import into other systems or software for visualization and further editing. This capability provides users with more options, making the pressure piping single-line diagram processing process smoother and more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of Cartesian coordinate system and Hough coordinate system.

[0021] Figure 2 Schematic diagram of the conversion between Cartesian coordinate system and Hough coordinate system.

[0022] Figure 3 It is the algorithm flow chart of the present invention.

[0023] Figure 4 This is a diagram showing the effect of the image denoising process described in the present invention.

[0024] Figure 5 This is a diagram showing the effect after processing by the text recognition module of the present invention.

[0025] Figure 6 This is a diagram showing the effect after processing by the YOLOX pipe inspection module of the present invention.

[0026] Figure 7 This is a diagram showing the effect after processing by the pipeline identification module of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described in detail below in conjunction with examples, but the embodiments of the present invention are not limited thereto.

[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0029] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0030] The present invention is obtained based on the following theoretical foundations: 1. Convolutional Neural Networks: Deep learning is a machine learning method whose core idea is to build a multi-layer neural network by simulating the connection between neurons in the human brain, so as to achieve learning and understanding of data. In deep learning, Convolutional Neural Networks (CNN) is a special neural network architecture that is specifically used to process tasks with grid-structured data (such as images).

[0031] Convolutional neural networks are composed of multiple layers, including convolutional layers, pooling layers, activation function layers, and fully connected layers. In CNN, the convolutional layer extracts features from the image through convolution operations, the pooling layer is used to reduce the dimension of the feature map and reduce the amount of calculation, the activation function layer introduces nonlinearity, and the fully connected layer maps the extracted features to the output layer for tasks such as classification or regression.

[0032] The advantage of CNN lies in its ability to extract features from images and other data. Through convolution operations, it can capture the local correlation of spatial information. At the same time, parameter sharing and hierarchical structures enable it to have good generalization capabilities when processing large-scale data. Due to its outstanding performance in the field of image processing, CNN has become one of the mainstream algorithms in computer vision and image recognition tasks, and is widely used in face recognition, object detection, image classification and other fields.

[0033] 2. OCR character recognition: Text recognition refers to the process of converting text content in images or videos into editable and searchable text data through computer vision and natural language processing technology. Text recognition has important applications in many fields, including optical character recognition, natural scene text recognition, handwritten text recognition, etc.

[0034] 3. Target detection - YOLOX algorithm: Object detection is an important task in the field of computer vision, which aims to detect objects in images or videos and determine their locations and categories. YOLO (You Only Look Once) is a popular object detection algorithm series. The YOLO algorithm transforms the object detection task into a regression problem, and implements object detection by dividing the image into grids and predicting the bounding box and category probability of each grid. YOLOX is one of the optimized versions in the YOLO series, focusing on improving detection performance and speed. The YOLOX algorithm has made many improvements and optimizations in structure, including: (1) Backbone network The Backbone structure in YOLOX combines the advantages of PANet and CSPDarknet53 to improve the efficiency and performance of feature extraction. This makes the Backbone in YOLOX have a deeper network structure and higher resolution feature maps, which helps to improve the accuracy of target detection.

[0035] (2) Neck network YOLOX introduced the YOLOXNeck structure and used structures such as PANet to perform multi-scale feature fusion, which enhanced the robustness and generalization ability of target detection. YOLOXNeck can effectively fuse features at different levels and improve the performance and effect of target detection.

[0036] (3) Head Network YOLOX uses the YOLOXHead structure, adopts the design ideas of YOLOv3 and YOLOv4, and introduces the decoupled head module to improve the accuracy and efficiency of target detection. The YOLOX algorithm has multiple scale architecture characteristics at the structural level, covering a variety of different scales such as YOLOX-s, YOLOX-m, YOLOX-x, etc., to meet the needs of different scenarios.

[0037] 4. Hough transform detection: Hough transform is a method for finding straight lines and circles in an image. Hough transform uses a voting-like method to obtain a set of shapes in the current image.

[0038] In the Hough coordinate system, the horizontal coordinate uses the slope k of the straight line in the Cartesian coordinate system, and the vertical coordinate uses the intercept b of the straight line in the Cartesian coordinate system.

[0039] like Figure 1 As shown, A is the Cartesian coordinate system and B is the Hough coordinate system; a straight line in Cartesian space determines a point in Hough space. A point in Hough space determines a straight line in Cartesian space.

[0040] like Figure 2 As shown, A is the Cartesian coordinate system and B is the Hough coordinate system; there are two points in the Cartesian coordinate system, and these two points correspond to two straight lines in the Hough coordinate system. Because the two points are on the same straight line in the Cartesian coordinate system, that is, the slope and intercept are the same, then the two straight lines will intersect at one point in the Hough coordinate system.

[0041] In summary, in Hough space, the more straight lines there are passing through a point, the more points the straight line mapped in Cartesian space is composed of.

[0042] 5. Image edge detection: Image edge detection is an important technology in the field of image processing, which is used to detect the boundaries of objects in images. Edges usually represent significant changes in pixel intensity in an image, which may represent the boundaries between objects or changes in the internal structure of an object.

[0043] The Canny operator is a classic edge detection algorithm proposed by John F. Canny in 1986. It is widely used in the field of computer vision. The main steps include: (1) First, the input image is Gaussian smoothed to reduce the impact of noise. Gaussian filtering can blur the image, making subsequent edge detection more accurate.

[0044] (2) Perform gradient calculation on the smoothed image to obtain the gradient amplitude and direction of each pixel in the image.

[0045] (3) In this step, the algorithm scans the image, retains the pixels with the largest local gradient, and suppresses other pixels, thereby refining the edges.

[0046] (4) Use two thresholds, a high threshold and a low threshold, to binarize the image. Pixel gradients higher than the high threshold are marked as strong edges, those lower than the low threshold are marked as non-edges, and those between the two are marked as weak edges.

[0047] (5) The last step is edge connection, which connects weak edges with strong edges to form complete edges by checking the gradient strength of pixels.

[0048] Embodiment 1: The present invention designs a CNN-based pressure pipeline single-line diagram feature recognition method, which accurately identifies and extracts the location and type of pipe fittings through target detection technology; automatically captures text information such as labels and annotations, including their coordinates and specific content, using OCR technology; and accurately detects and extracts key features such as the direction and coordinates of the pipeline using edge detection and Hough transform algorithms, including the following specific steps: 1) Image denoising: Import the pressure pipeline single-line diagram image into the image denoising module and perform denoising to obtain the denoised image; 2) Text recognition module processing: The denoised image is input into the text recognition module, and the text content is extracted through the text detection algorithm and text recognition algorithm to obtain the content and coordinates of all texts in the image; 3) YOLOX pipe fitting detection module processing: The output of step 1) and step 2) are processed by YOLOX pipe fitting detection module to extract key information, and the type and coordinates of each pipe fitting in the image are output; 4) Processing by pipeline identification module: After step 3), the pipeline identification module uses one or more of edge detection, Hough transform and dilation operations to process the data and obtain the directions and coordinates of all pipelines in the image; 5) Result output processing: Finally, the outputs of the text recognition module, YOLOX pipe fitting detection module and pipeline recognition module are normalized.

[0049] Embodiment 2: This embodiment is further optimized on the basis of the above embodiment, and the similarities with the above technical solution are not repeated here. In order to better realize the pressure pipeline single-line diagram feature recognition method based on CNN described in the present invention, the following setting method is particularly adopted: the step 1) is specifically: the pressure pipeline single-line diagram to be identified is used as input (the beginning of the whole process), imported into the image denoising module, and the Gaussian filtering method is used to reduce the image noise, that is, some abnormal points that may exist in the image are removed to obtain a denoised image. By image denoising, the real details are highlighted, and a clearer input is provided for subsequent modules.

[0050] Embodiment 3: This embodiment is further optimized on the basis of any of the above embodiments, and the same points as the above technical solutions are not repeated here. In order to better realize the pressure pipeline single-line diagram feature recognition method based on CNN described in the present invention, the following setting is particularly adopted: the step 2) includes the following steps: 2.1) Resize the denoised image and input it into the text detection algorithm DBNet of the text recognition module. The coordinates of the text in the image are output, including the coordinates of the upper left corner and lower right corner of all the text in the image, and the coordinates are stored in a two-dimensional array. That is, the text detection algorithm DBNet will adjust the image size to a fixed size before calculation, and then input the fixed-size image into the algorithm. The algorithm will output the coordinates of the text in the image. The text coordinates are the upper left and lower right coordinates of the text after the rectangular box is selected. The detection algorithm DBNet will output the upper left and lower right coordinates of all the text in the image, and store the coordinates in a two-dimensional array.

[0051] 2.2) After obtaining the coordinates of all the text in the image, crop the original image according to the coordinates of the upper left corner and lower right corner of each text. Each cropped sub-image represents a line of text. 2.3) Since some text sub-images may be tilted, the sub-images with tilt problems are first preset as horizontal sub-images, and a mapping relationship between tilt and horizontal is established. Then, the sub-images with tilt problems are adjusted to a horizontal state using an affine transformation method according to the mapping relationship. 2.4) After all horizontal text sub-images are adjusted to a fixed image size, they are input into the text recognition model CRNN of the text recognition module. The CRNN algorithm will classify each character in the sub-image and output the recognition result of each character. The result of each sub-image is the collection of all the character recognition results it contains. 2.5) The coordinates of the upper left corner and lower right corner of each text in the original image and the corresponding text recognition results are combined and output, thus obtaining the content and coordinates of all the text in the image.

[0052] Embodiment 4: This embodiment is further optimized on the basis of any of the above embodiments, and the same points as the above technical solutions are not repeated here. In order to better realize the pressure pipeline single-line diagram feature recognition method based on CNN described in the present invention, the following setting is particularly adopted: the step 3) includes the following specific steps: 3.1) Input the output of step 1) and step 2) into the YOLOX pipe inspection module. First, use the denoised image and the results obtained by the text recognition module to fill each text area (a rectangle formed by the upper left corner coordinates and the lower right corner coordinates) with the background color according to the coordinates of each text obtained by text recognition, so as to obtain an image after removing the text on the basis of the original image. 3.2) Then, the image after removing the text is adjusted to a fixed image size and then input into the object detection algorithm YOLOX to detect the pipes in the image, and output the type and coordinates (upper left and lower right coordinates) of each pipe in the image.

[0053] Embodiment 5: This embodiment is further optimized on the basis of any of the above embodiments, and the similarities with the above technical solutions are not repeated here. In order to better realize the pressure pipeline single-line diagram feature recognition method based on CNN described in the present invention, the following setting is particularly adopted: in the step 3.1), the text area is a rectangle formed by the upper left corner coordinates and the lower right corner coordinates of each text; the coordinates of each pipe fitting include two coordinates of the upper left corner and the lower right corner.

[0054] Embodiment 6: This embodiment is further optimized on the basis of any of the above embodiments, and the same points as the above technical solutions are not repeated here. In order to better realize the pressure pipeline single-line diagram feature recognition method based on CNN described in the present invention, the following setting is particularly adopted: the step 4) includes the following specific steps: 4.1) First, the coordinates of each pipe are obtained according to the pipe detection, that is, each pipe area in the original image (a rectangle formed by the coordinates of the upper left corner and the lower right corner), and each pipe area is filled with the background color; 4.2) After step 4.1), an image is obtained by first removing the text area and then removing the pipe area based on the original image; 4.3) Then, the image is processed by edge detection Canny operator, and the grayscale image obtained has a black background and white line segments, i.e., convex line segment information; 4.4) After step 4.3), the Hough transform method is used for the first time to detect the line segment, and the Hough transform method is used to fit the straight line in the figure. The fitted straight line is represented by the coordinates of the two ends; 4.5) After step 4.4), fill the line segment with the background color according to the coordinates of both ends of the line segment; 4.6) There may be a small number of undetected line segments in the filled image. Then, the dilation operation is used to enhance the features of the remaining undetected line segments in the image after step 4.5). The contour detection method is then used to obtain the line segment area in the image (the edge coordinates of each area). Some abnormal points (noise) are filtered out by calculating the number of pixel values ​​in each area. 4.7) Using the second Hough transform method on the image processed in step 4.6) to detect the remaining line segments, and obtain the coordinates of the two ends of each line segment; 4.8) Then, the line segment coordinate information detected by the first and second Hough transform methods are combined to obtain the coordinate information of all line segments; 4.9) For each line segment, we have two end coordinates. We can use the two end coordinates to calculate the slope of the line segment, and then use the inverse tangent function to calculate the angle between the line segment and the horizontal direction, that is, the direction in the figure.

[0055] Embodiment 7: This embodiment is further optimized on the basis of any of the above embodiments, and the similarities with the above technical solutions are not repeated here. In order to better realize the pressure pipeline single-line diagram feature recognition method based on CNN described in the present invention, the following setting method is particularly adopted: in the step 4.6), the contour detection method is used to obtain the line segment area in the figure, specifically, the edge coordinates of each area of ​​the line segment area are obtained using the contour detection method.

[0056] Embodiment 8: This embodiment is further optimized on the basis of any of the above embodiments, and the same points as the above technical solutions are not repeated here. In order to better realize the pressure pipeline single-line diagram feature recognition method based on CNN described in the present invention, the following setting is particularly adopted: the step 5) is specifically: For the output results of the text recognition module, output the coordinates of the upper left corner and lower right corner of each line of text, as well as the corresponding text content; For the output results of the YOLOX pipe inspection module, the type, upper left corner coordinates and lower right corner coordinates of each pipe are output; For the output results of the pipeline identification module, the coordinates and directions of both ends of each line segment are output; The output results of the text recognition module, YOLOX pipe detection module and pipeline identification module are merged in JSON format, that is, different keys are used to distinguish different modules, and the value corresponding to each key is the output result of each module.

[0057] Embodiment 9: A CNN-based pressure pipeline single-line diagram feature recognition method combined with Figure 3 As shown, the following steps are included: 1. Single-line diagram input: First, import the single-line diagram of the pressure pipeline to be identified; (II) Image denoising processing: The single-line diagram of the pressure pipeline to be identified is taken as input (the beginning of the whole process), imported into the image denoising module, and the Gaussian filtering method is used to reduce the image noise, that is, some abnormal points that may exist in the image are removed to obtain the denoised image. Through image denoising, the real details are highlighted, providing clearer input for subsequent modules.

[0058] (III) Text recognition module processing: The first step is to input the denoised image into the text detection algorithm DBNet of the text recognition module. The text detection algorithm DBNet will adjust the image size to a fixed size before calculation, and then input the fixed-size image into the text detection algorithm DBNet. The text detection algorithm DBNet will output the coordinates of the text in the image. The text coordinates are the upper left and lower right coordinates of the text after the text is selected in the form of a rectangular box. The text detection algorithm DBNet will output the upper left and lower right coordinates of all the text in the image, and store the coordinates in a two-dimensional array.

[0059] Step 2: After obtaining the coordinates of all the text in the image, crop the original image according to the coordinates of the upper left corner and lower right corner of each text. Each cropped sub-image represents a line of text. Step 3: Since some text sub-images may have tilt problems, the sub-images with tilt problems are first preset as horizontal sub-images, and a mapping relationship between tilt and horizontal is established. Then, the sub-images with tilt problems are adjusted to a horizontal state using an affine transformation method according to the mapping relationship. Step 4: After adjusting all horizontal text sub-images to a fixed image size, they are input into the text recognition model CRNN of the text recognition module. The CRNN algorithm will classify each character in the sub-image and output the recognition result of each character. The result of each sub-image is the collection of all the character recognition results it contains. Step 5: Combine the upper left corner and lower right corner coordinates of each text in the original image and the corresponding text recognition results for output, thus obtaining the content and coordinates of all texts in the image.

[0060] (IV) YOLOX pipe fittings inspection module processing: Step 1: Input the output of step (2) and step (3) into the YOLOX pipe inspection module. First, use the denoised image and the results obtained by the text recognition module to fill each text area (a rectangle formed by the upper left corner coordinates and the lower right corner coordinates) with the background color according to the coordinates of each text obtained by text recognition, so as to obtain an image after removing the text on the basis of the original image. Step 2: Then, adjust the image after removing the text to a fixed image size, and then input it into the target detection algorithm YOLOX to detect the pipes in the image, and output the type and coordinates (upper left and lower right coordinates) of each pipe in the image.

[0061] (V) Pipeline identification module processing: Step 1: First, get the coordinates of each pipe according to the pipe detection, that is, each pipe area in the original image (a rectangle formed by the coordinates of the upper left corner and the lower right corner), and fill each pipe area with the background color; Step 2: After that, an image is obtained by removing the text area and then the pipe area based on the original image; Step 3. Then, the image is processed by edge detection Canny operator, and the grayscale image obtained has a black background and white line segments, that is, convex line segment information; Step 4: After that, the first Hough transform method is used to detect the line segment, and the Hough transform method is used to fit the straight line in the figure. The fitted straight line is represented by the coordinates of the two ends; Step 5. Then, fill the line segment with the background color according to the coordinates of the two ends of the line segment; Step 6. There may be a small number of undetected line segments in the filled image. Then, the dilation operation is used to enhance the features of the remaining undetected line segments in the image after step 5. The contour detection method is then used to obtain the line segment area in the image (the edge coordinates of each area). Some abnormal points (noise) are filtered out by calculating the number of pixel values ​​in each area. Step 7: Use the second Hough transform method to detect the remaining line segments on the image processed in step 6 to obtain the coordinates of both ends of each line segment; Step 8: Then, the line segment coordinate information detected by the first and second Hough transform methods are combined to obtain the coordinate information of all line segments; Finally, for each line segment, which has two end coordinates, the slope of the line segment can be calculated through the two end coordinates, and then the inverse tangent function can be used to calculate the angle between the line segment and the horizontal direction, that is, the direction in the figure.

[0062] (VI) Result output processing: The results processed by the text recognition module, the results processed by the YOLOX pipe fitting detection module, and the results processed by the pipeline recognition module are obtained respectively.

[0063] For the output results of the text recognition module, output the coordinates of the upper left corner and lower right corner of each line of text, as well as the corresponding text content; For the output results of the YOLOX pipe inspection module, the type, upper left corner coordinates and lower right corner coordinates of each pipe are output; For the output results of the pipeline identification module, the coordinates and directions of both ends of each line segment are output; The output results of the text recognition module, YOLOX pipe detection module and pipeline identification module are merged in JSON format, that is, different keys are used to distinguish different modules, and the value corresponding to each key is the output result of each module.

[0064] Embodiment 10: A feature recognition method for pressure pipeline single-line diagram based on CNN.

[0065] (1) After inputting the pressure pipeline single-line diagram, denoising is performed. The effect of image denoising is shown in the figure below: Figure 4 as shown. (2) Input the denoised image into the text recognition module. The effect after processing by the text recognition module is shown in the figure below. Figure 5 shown.

[0066] (3) Fill the text area obtained by the text recognition module with the background color, and then input the image into the YOLOX pipe fitting detection module for detection. The effect after processing by the YOLOX pipe fitting detection module is shown in the figure below. Figure 6 shown.

[0067] (4) The result of the YOLOX pipe detection module is filled with background color in the image, and then the image is input into the pipeline recognition module. The effect after processing by the pipeline recognition module is shown in the figure below. Figure 7 shown.

[0068] (5) Finally, the element features obtained by text recognition, pipe fitting detection, and pipeline recognition are adjusted to the specified format for output.

[0069] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for feature recognition of a pressure pipeline single-line diagram based on CNN, characterized in that: The specific steps include: 1) Import the pressure pipeline single-line diagram image into the image denoising module and perform denoising processing to obtain the denoised image; 2) Input the denoised image into the text recognition module, extract the text content through the text detection algorithm and text recognition algorithm, and obtain the content and coordinates of all texts in the image; 3) The output of step 1) and step 2) are used to extract key information using the YOLOX pipe fitting detection module, and the type and coordinates of each pipe fitting in the image are output; 4) After step 3), the pipeline identification module uses one or more of edge detection, Hough transform and dilation operations to process the data and obtain the directions and coordinates of all pipelines in the image; 5) Finally, the outputs of the text recognition module, YOLOX pipe detection module, and pipeline recognition module are normalized.

2. The method for identifying features of a pressure pipeline single-line diagram based on CNN according to claim 1 is characterized in that: The step 1) is specifically as follows: taking the single-line diagram of the pressure pipeline to be identified as input, importing it into the image denoising module, and using the Gaussian filtering method to reduce the image noise to obtain a denoised image.

3. The method for identifying features of a pressure pipeline single-line diagram based on CNN according to claim 1 is characterized in that: The step 2) comprises the following steps: 2.1) Resize the denoised image and input it into the text detection algorithm DBNet of the text recognition module. The coordinates of the text in the image are output, including the coordinates of the upper left corner and lower right corner of all the text in the image, and the coordinates are stored in a two-dimensional array. 2.2) After obtaining the coordinates of all the text in the image, crop the original image according to the coordinates of the upper left corner and lower right corner of each text. Each cropped sub-image represents a line of text. 2.3) Since some text sub-images have tilt problems, the sub-images with tilt problems are first preset as horizontal sub-images, and a mapping relationship between tilt and horizontal is established. Then, the sub-images with tilt problems are adjusted to a horizontal state using an affine transformation method according to the mapping relationship. 2.4) After all horizontal text sub-images are adjusted to a fixed image size, they are input into the text recognition model CRNN of the text recognition module. The CRNN algorithm will classify each character in the sub-image and output the recognition result of each character. The result of each sub-image is the collection of all the character recognition results it contains. 2.5) Combine the upper left corner and lower right corner coordinates of each text in the original image and the corresponding text recognition results and output them to obtain the content and coordinates of all texts in the image.

4. The method for identifying features of a pressure pipeline single-line diagram based on CNN according to claim 1 is characterized in that: The step 3) includes the following specific steps: 3.1) Input the output of step 1) and step 2) into the YOLOX pipe inspection module. First, use the denoised image and the results obtained by the text recognition module to fill each text area with the background color according to the coordinates of each text obtained by text recognition, so as to obtain an image after removing the text on the basis of the original image. 3.2) Then, the image after removing the text is adjusted to a fixed image size and then input into the object detection algorithm YOLOX to detect the pipes in the image and output the type and coordinates of each pipe in the image.

5. The method for identifying features of a pressure pipeline single-line diagram based on CNN according to claim 4 is characterized in that: In the step 3.1), the text area is a rectangle formed by the coordinates of the upper left corner and the lower right corner of each text; the coordinates of each pipe include the coordinates of the upper left corner and the lower right corner.

6. The method for identifying features of a pressure pipeline single-line diagram based on CNN according to claim 1 is characterized in that: The step 4) includes the following specific steps: 4.1) First, the coordinates of each pipe fitting are obtained according to the pipe fitting detection, that is, each pipe fitting area in the original image is filled with the background color; 4.2) After step 4.1), an image is obtained by first removing the text area and then removing the pipe area based on the original image; 4.3) Then, the image is processed by edge detection Canny operator, and the background in the obtained grayscale image is black and the line segment is white; 4.4) After step 4.3), the Hough transform method is used for the first time to detect the line segment, and the Hough transform method is used to fit the straight line in the figure. The fitted straight line is represented by the coordinates of the two ends; 4.5) After step 4.4), fill the line segment with the background color according to the coordinates of both ends of the line segment; 4.6) Then, the dilation operation is used to enhance the features of the remaining undetected line segments in the image after step 4.5), and then the contour detection method is used to obtain the line segment area in the image, and some abnormal points are filtered out by calculating the number of pixel values ​​in each area; 4.7) Using the second Hough transform method to detect the remaining line segments on the image processed in step 4.6), and obtain the coordinates of the two ends of each line segment; 4.8) Then, the coordinate information of the line segments detected by the first and second Hough transform methods are combined to obtain the coordinate information of all line segments; 4.9) Each line segment has two end coordinates. The slope of the line segment is calculated using the two end coordinates. The inverse tangent function is then used to calculate the angle between the line segment and the horizontal direction, which is the direction in the figure.

7. The method for identifying features of a pressure pipeline single-line diagram based on CNN according to claim 6 is characterized in that: In the step 4.6), the line segment region in the image is obtained using a contour detection method, specifically, the edge coordinates of each region of the line segment region are obtained using the contour detection method.

8. The method for identifying features of a pressure pipeline single-line diagram based on CNN according to claim 1 is characterized in that: The step 5) is specifically as follows: For the output results of the text recognition module, output the coordinates of the upper left corner and lower right corner of each line of text, as well as the corresponding text content; For the output results of the YOLOX pipe inspection module, the type, upper left corner coordinates and lower right corner coordinates of each pipe are output; For the output results of the pipeline identification module, the coordinates and directions of both ends of each line segment are output; The output results of the text recognition module, YOLOX pipe detection module and pipeline identification module are merged in JSON format, that is, different keys are used to distinguish different modules, and the value corresponding to each key is the output result of each module.

Images