Pipeline model modeling method and device based on engineering drawing recognition

Abstract

Embodiments of the present disclosure disclose a pipeline model modeling method and device based on engineering drawing recognition. A specific implementation of the method comprises: performing pipeline recognition on a pipeline engineering drawing to obtain a pipeline structure diagram; performing fixed-resolution directional equipment component recognition on the pipeline engineering drawing according to the pipeline structure diagram; performing area masking on the pipeline structure diagram according to a first equipment component information set; performing dynamic-resolution directional equipment component recognition on the pipeline engineering drawing according to the masked pipeline structure diagram; performing global topology verification according to the pipeline structure diagram, the first equipment component information set and a second equipment component information set; and in response to passing the global topology verification, performing three-dimensional pipeline model rendering according to the pipeline structure diagram, the first equipment component information set and the second equipment component information set to obtain a pipeline model corresponding to the pipeline engineering drawing. This implementation enables efficient and accurate automatic modeling of a pipeline model.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more particularly to the field of digital modeling, specifically to a pipeline modeling method and apparatus based on engineering drawing recognition. Background Technology

[0002] In the industrial field, 3D modeling based on drawings is one of the core technologies for realizing digital twins of factories. Through 3D pipeline models, the spatial layout of pipelines and equipment components connected by pipelines can be intuitively displayed, thereby avoiding the blind spots that exist in 2D drawings.

[0003] Currently, the common method for pipeline modeling is to manually create the model using two-dimensional drawings in modeling software (such as CAD software).

[0004] However, this method is extremely inefficient for creating large pipeline models and is prone to error accumulation due to human error, which can affect the accuracy of the pipeline model. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure propose a pipeline modeling method and apparatus based on engineering drawing recognition to solve the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide a pipeline modeling method based on engineering drawing recognition. The method includes: performing pipeline recognition on pipeline engineering drawings to obtain a pipeline structure diagram, wherein the pipeline engineering drawings are electronic drawings for generating a corresponding pipeline model, and the pipeline structure diagram represents the topological structure of the pipeline; based on the pipeline structure diagram, performing fixed-resolution directional equipment component recognition on the pipeline engineering drawings to obtain a first set of equipment component information, wherein the first set of equipment component information includes: component type, component specification parameters, component connection parameters, component location parameters, and component confidence level; based on the above... The first set of equipment component information is used to mask the pipeline structure diagram to obtain a masked pipeline structure diagram. Based on the masked pipeline structure diagram, dynamic resolution directional equipment component identification is performed on the pipeline engineering drawing to obtain a second set of equipment component information. Global topology verification is performed based on the pipeline structure diagram, the first set of equipment component information, and the second set of equipment component information. In response to passing the global topology verification, a three-dimensional pipeline model is rendered based on the pipeline structure diagram, the first set of equipment component information, and the second set of equipment component information to obtain the pipeline model corresponding to the pipeline engineering drawing.

[0008] Secondly, some embodiments of this disclosure provide a pipeline modeling apparatus based on engineering drawing recognition. The apparatus includes: a pipeline recognition unit configured to perform pipeline recognition on pipeline engineering drawings to obtain a pipeline structure diagram, wherein the pipeline engineering drawings are electronic drawings for generating a corresponding pipeline model, and the pipeline structure diagram represents the topological structure of the pipeline; a first recognition unit configured to perform fixed-resolution directional device component recognition on the pipeline engineering drawings based on the pipeline structure diagram to obtain a first device component information set, wherein the first device component information includes: component type, component specification parameters, component connection parameters, component location parameters, and component confidence level; and a region masking unit configured to perform region masking based on the first device... The system comprises: a component information set; a region masking unit for the pipeline structure diagram to obtain a masked pipeline structure diagram; a second identification unit configured to perform dynamic resolution directional device component identification on the pipeline engineering drawing based on the masked pipeline structure diagram to obtain a second device component information set; a global topology verification unit configured to perform global topology verification based on the pipeline structure diagram, the first device component information set, and the second device component information set; and a three-dimensional pipeline model rendering unit configured to render a three-dimensional pipeline model based on the pipeline structure diagram, the first device component information set, and the second device component information set in response to passing the global topology verification, to obtain the pipeline model corresponding to the pipeline engineering drawing.

[0009] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0010] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0011] The various embodiments of this disclosure have the following beneficial effects: The pipeline modeling method based on engineering drawing recognition, as described in some embodiments of this disclosure, achieves efficient and accurate pipeline modeling. Specifically, the pipeline modeling method based on engineering drawing recognition, as described in some embodiments of this disclosure, firstly, performs pipeline recognition on the pipeline engineering drawing to obtain a pipeline structure diagram. The pipeline engineering drawing is an electronic drawing to be used to generate the corresponding pipeline model, and the pipeline structure diagram represents the topological structure of the pipeline. In practice, the pixel proportion of pipelines and the equipment components connected by the pipelines in the pipeline engineering drawing is often small. If the pipeline engineering drawing is used as a whole for recognition, there will be a large number of invalid inference processes, leading to a large consumption of computing resources. This disclosure considers that the equipment pipeline, as a skeleton structure, connects different equipment components; therefore, this disclosure uses pipeline recognition to quickly extract the pipeline topology structure from the pipeline engineering drawing. Secondly, based on the aforementioned pipeline structure diagram, the pipeline engineering drawings are subjected to fixed-resolution directional equipment component identification to obtain a first set of equipment component information. This first set of information includes: component type, component specifications, component connection parameters, component location parameters, and component confidence level. By combining directional and fixed-resolution identification with the pipeline structure diagram, equipment components with distinct characteristics can be identified quickly and effectively. Next, based on the first set of equipment component information, the pipeline structure diagram is masked to obtain a masked pipeline structure diagram. In practice, considering the differences in specifications among different equipment components, and the potential impact of the pipeline engineering drawings' specifications on identification effectiveness, region masking is used to mask equipment components in the pipeline structure diagram that correspond to the first set of equipment component information to avoid redundant identification during subsequent refined identification processes. Further, based on the masked pipeline structure diagram, the pipeline engineering drawings are subjected to dynamic-resolution directional equipment component identification to obtain a second set of equipment component information. In practice, to avoid omissions due to the small size of equipment components at a fixed resolution, this disclosure employs dynamic resolution-based directional identification to supplement the identification of equipment components. Furthermore, a global topology check is performed based on the aforementioned pipeline structure diagram, the first set of equipment component information, and the second set of equipment component information. This global topology check determines the rationality of the topological connections between pipelines and equipment components. Finally, in response to the global topology check, a 3D pipeline model is rendered based on the aforementioned pipeline structure diagram, the first set of equipment component information, and the second set of equipment component information, resulting in the pipeline model corresponding to the aforementioned pipeline engineering drawings. This method achieves efficient and accurate automatic modeling of pipeline models. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a flowchart of some embodiments of the pipeline modeling method based on engineering drawing recognition according to this disclosure; Figure 2 This is a schematic diagram of the drawing block division process; Figure 3 This is a schematic diagram of the pipeline feature map generation process; Figure 4 This is a schematic diagram illustrating the generation process of a local search domain within the first device search domain; Figure 5 This is a visual interface diagram corresponding to the device component model library; Figure 6 These are schematic diagrams of some embodiments of the pipeline modeling apparatus based on engineering drawing recognition according to this disclosure; Figure 7 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] refer to Figure 1 The diagram illustrates a flow 100 of some embodiments of a pipeline modeling method based on engineering drawing recognition according to the present disclosure. This pipeline modeling method based on engineering drawing recognition includes the following steps: Step 101: Identify pipelines in the pipeline engineering drawings to obtain the pipeline structure diagram.

[0021] In some embodiments, the execution subject (e.g., a computing device) of the pipeline modeling method based on engineering drawing recognition can perform pipeline recognition on pipeline engineering drawings to obtain a pipeline structure diagram.

[0022] The pipeline engineering drawings are electronic drawings of the pipeline models to be generated. Specifically, pipeline engineering drawings can be structural drawings containing pipelines and equipment components connected by the pipelines. For example, the file format of pipeline engineering drawings can be, but is not limited to, JPEG (Joint Photographic Experts Group) format and PDF (Portable Document Format). Equipment components represent mechanical equipment in the corresponding engineering field. For example, in the chemical industry, the pipelines in the pipeline engineering drawings can be used for transporting chemical raw materials. The equipment components connected by the pipelines can be chemical equipment. The pipeline structure diagram represents the topology of the pipeline. Specifically, the data structure of the pipeline structure diagram is a graph.

[0023] In practice, the background of pipeline engineering drawings is often relatively simple (e.g., white or light-colored background), and pipelines are often represented by black or colored lines with relatively fixed line widths. However, for pipeline engineering drawings representing complex pipeline structures, the number of pixels of pipelines and equipment components connected by pipelines is relatively small compared to the background. If the pipeline engineering drawing is used as a whole for pipeline and equipment component identification, a large number of invalid pixels need to be processed. Since equipment components are often connected by pipelines, this disclosure first extracts the pipeline structure diagram to assist in the subsequent directional identification of equipment components, thereby improving identification efficiency and avoiding the waste of computing resources.

[0024] In practice, the aforementioned implementing entity identifies pipelines on pipeline engineering drawings to obtain pipeline structure diagrams, which may include the following steps: Step S1: Perform adaptive threshold binarization on the above pipeline engineering drawings to obtain the binarized pipeline engineering drawings.

[0025] Specifically, since the thickness of the corresponding pipeline lines varies in different drawings, and the contrast between the corresponding pipeline lines and the background also varies, using a fixed threshold binarization method may weaken the difference between the pipeline lines and the background. Therefore, this disclosure improves the difference between the corresponding pipeline lines and the background by using an adaptive threshold binarization method.

[0026] Step S2: Perform edge detection on the pipeline engineering drawings after the above binarization process to obtain a set of candidate line segments.

[0027] Among them, candidate line segments represent the line segments corresponding to the complete pipeline or the partial pipeline.

[0028] Specifically, edge detection can be performed on the binarized pipeline engineering drawings using an edge detection algorithm based on the Canny operator. Furthermore, considering that the actual physical edges may be identified as two thin lines (pseudo-edge phenomenon) after processing with first- or second-order differential operators using the Canny operator-based edge detection algorithm, and that it is sensitive to noise, edge detection can also be performed on the binarized pipeline engineering drawings using directional adjustable filters and line detection filters.

[0029] Step S3: Connect the candidate line segments in the candidate line segment set to obtain the above pipeline structure diagram.

[0030] Specifically, during the edge detection process described above, the complete line segment corresponding to the pipeline may break, forming multiple candidate line segments, which need to be reconnected. First, the local direction corresponding to each candidate line segment can be calculated (for example, the local direction can be determined by calculating the direction vector corresponding to the candidate line segment). Next, at the two endpoints of the candidate line segment, the line segment is extended along the local direction, and an endpoint search is performed. If the distance between the searched endpoint and the extension line of the candidate line segment is less than a distance threshold, and the local direction of the candidate line segment corresponding to the searched endpoint is consistent, then the two candidate line segments are connected to obtain the pipeline structure diagram.

[0031] It should be noted that the aforementioned computing devices can be either hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed on the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0032] In some optional implementations of certain embodiments, the aforementioned execution entity identifies pipelines on pipeline engineering drawings to obtain a pipeline structure diagram, including: Step S1: Rotate the above pipeline engineering drawing at a preset angle in a clockwise direction to obtain the rotated pipeline engineering drawing.

[0033] The preset angle can be 5 degrees.

[0034] Step S2: Divide the above rotated pipeline engineering drawings into blocks to obtain a set of drawing image blocks.

[0035] In this set of drawing image blocks, there is an overlapping area between every two adjacent drawing image blocks. Assuming the size of the rotated pipeline engineering drawing is N×M, the size of each drawing image block can be n×m, where n < N and m < M. The overlap ratio between two adjacent drawing image blocks can be a preset overlap ratio. For example, the preset overlap ratio can be 15%. The overlap ratio represents the ratio of the overlapping area of ​​two adjacent drawing image blocks to the size of the drawing image block.

[0036] In practice, pipelines in pipeline engineering drawings are often arranged vertically or horizontally. Therefore, during the process of dividing the drawing into blocks, there may be overlaps between pipelines and the dividing boundaries. This disclosure overcomes the aforementioned overlap problem by rotating the drawing at a slight angle. In addition, by setting the overlap between two connected drawing image blocks, the semantic coherence of the identified pipelines is ensured, facilitating the subsequent reconstruction of the corresponding pipeline topology.

[0037] As an example, see Figure 2 The diagram illustrates the drawing segmentation process. The executing entity can segment the rotated pipeline engineering drawing 201 along both the horizontal and vertical dimensions, using a segmentation size of n×m. The overlap ratio between any two adjacent drawing image blocks obtained from the segmentation is 15%. Figure 2For example, after dividing the rotated pipeline engineering drawing 201 into blocks, 12 drawing image blocks (a set of drawing image blocks) are obtained, namely: drawing image block A11, drawing image block A12, drawing image block A13, drawing image block A14, drawing image block A21, drawing image block A22, drawing image block A23, drawing image block A24, drawing image block A31, drawing image block A32, drawing image block A33, and drawing image block A34. Specifically, taking drawing image block A11 as an example, it overlaps with drawing image blocks A21 and A12 respectively. Taking drawing image block A13 as an example, it overlaps with drawing image blocks A12, A14, and A23 respectively. Taking drawing image block A22 as an example, it overlaps with drawing image blocks A12, A21, A23, and A32 respectively.

[0038] Step S3: For each drawing image block in the above set of drawing image blocks, perform the following pipeline identification steps: Step S31: Extract pipeline features from the above drawing image blocks using a pipeline feature extraction network to obtain a pipeline feature map.

[0039] The pipeline feature extraction network described above consists of a multi-scale feature extraction module and a feature fusion module. The pipeline feature map is a semantic feature map of a drawing image block. The feature map size of the pipeline feature map is consistent with the image block size of the drawing image block. The multi-scale feature extraction module is used to extract semantic features from the drawing image block at multiple scales.

[0040] As an example, see Figure 3 The diagram illustrates the pipeline feature map generation process. The multi-scale feature extraction module consists of three serially connected downsampling networks: a first downsampling network, a second downsampling network, and a third downsampling network. The input to the first downsampling network is a drawing image patch (n×m in size), and its output size is n / 2×m / 2. The input to the second downsampling network is the output of the first downsampling network, and its output size is n / 4×m / 4. The input to the third downsampling network is the output of the second downsampling network, and its output size is n / 16×m / 16. This yields feature representations under two small receptive fields and one large receptive field. The feature fusion module consists of an upsampling network. The upsampling network first upsamples the output of the third downsampling network to a feature dimension of n / 4×m / 4, then superimposes it with the output of the second downsampling network, and then further upsamples it to a feature dimension of n / 2×m / 2. This is then superimposed with the output of the first downsampling network and further upsampled to n×m, thus obtaining the pipeline feature map.

[0041] Step S32: Generate candidate pipeline description information using the pipeline location network and the above pipeline feature map.

[0042] The candidate pipeline description information includes pipeline location vectors and pipeline confidence scores. The pipeline feature extraction network and pipeline localization network are included in the pipeline recognition network. The pipeline location vector represents the two-dimensional coordinate sequence of the pipeline's location. The pipeline confidence score represents the confidence level corresponding to the identified pipeline. The pipeline localization network consists of: transposed convolutional layers, ReLU activation functions, 2D convolutional layers, and Sigmoid activation functions. The 2D convolutional layers output a segmentation map of the identified pipeline region. The execution entity then converts the segmentation map into a two-dimensional coordinate sequence to obtain the pipeline location vectors included in the candidate pipeline description information. The pipeline feature extraction network and the pipeline localization network are trained as a whole using supervised training.

[0043] In practice, the foreground of pipeline engineering drawings consists of pipelines and equipment components connected by the pipelines, while the background is a white or light-colored background. Therefore, the feature extraction of pipeline feature maps can be completed through the lightweight network structure designed in this disclosure. At the same time, combined with a parallel pipeline recognition strategy, pipeline recognition can be performed quickly on the set of image blocks in the drawing.

[0044] Step S33: In response to the fact that the pipeline confidence level included in the above candidate pipeline description information is greater than or equal to the preset pipeline confidence level, the above candidate pipeline description information is determined as the target pipeline description information.

[0045] Step S34: In response to the fact that the pipeline confidence level included in the above candidate pipeline description information is less than the preset pipeline confidence level, the above candidate pipeline description information is updated by voting according to the target drawing image block group to obtain the target pipeline description information.

[0046] Among them, the target drawing image block in the target drawing image block group is the drawing image block that has an overlapping area with the above-mentioned drawing image blocks.

[0047] In practice, because pipeline identification is performed on image blocks in the drawing image block set in parallel, candidate pipeline description information for each image block is obtained almost synchronously. However, while block processing can improve processing efficiency, it loses the complete feature representation from the perspective of pipeline engineering drawings, which may lead to low-confidence identification. Therefore, when low-confidence identification occurs, since the semantic coherence of the identified pipelines is ensured by setting overlap between two connected image blocks, a voting update method can be used to update the two-dimensional coordinates of the pipeline position vectors included in the candidate pipeline description information to obtain the target pipeline description information.

[0048] As an example, see further. Figure 2 Taking drawing image block A11 as an example, the pipeline confidence score included in its corresponding candidate pipeline description information is lower than the preset pipeline confidence score. In this case, the target drawing image block group includes drawing image block A12 and drawing image block A21. Specifically, combining the two candidate pipeline description information corresponding to drawing image blocks A12 and A21, the two-dimensional coordinates of the pipeline position vector included in the candidate pipeline description information corresponding to drawing image block A11 are updated. More specifically, a probability vote is performed based on the pipeline confidence scores included in the three candidate pipeline description information corresponding to drawing image blocks A11, A12, and A21, and the two-dimensional coordinates of the pipeline position vector included in the candidate pipeline description information corresponding to drawing image block A11 are updated according to the candidate pipeline description information corresponding to the highest number of votes. Step S4: Perform topology restoration based on the obtained target pipeline description information set to obtain the candidate pipeline structure diagram.

[0049] In practice, since the target pipeline description information includes pipeline location vectors, i.e., discretized pipeline location representations, the pipeline topology can be fitted and restored using linear fitting to obtain candidate pipeline structure diagrams.

[0050] Step S5: Rotate the candidate pipeline structure diagram counterclockwise by the preset angle to obtain the pipeline structure diagram.

[0051] In practice, since it is necessary to identify oriented equipment in pipeline engineering drawings in conjunction with pipeline structure diagrams, it is necessary to ensure that the pipeline direction is consistent between the pipeline structure diagram and the pipeline engineering drawings. Since the pipeline structure diagram is identified based on the rotated pipeline engineering drawings, it is necessary to rotate the candidate pipeline structure diagram counterclockwise by a preset angle to restore the pipeline direction.

[0052] Step 102: Based on the pipeline structure diagram, perform fixed-resolution directional equipment component identification on the pipeline engineering drawings to obtain the first set of equipment component information.

[0053] In some embodiments, the aforementioned execution entity can identify directional equipment components in pipeline engineering drawings at a fixed resolution based on the pipeline structure diagram to obtain a first set of equipment component information.

[0054] The first equipment component information represents the equipment components connected via pipelines. This information includes: component type, component specifications, component connection parameters, component location parameters, and component confidence level. Component specifications represent the component specifications. Component connection parameters represent the connection direction of the component. Component location parameters represent the component's location on the pipeline engineering drawings. Component confidence level represents the confidence level of the identified equipment component.

[0055] In practice, since equipment components are often connected by pipelines, the search area can be based on the location of the pipeline structure diagram, allowing for fixed-resolution directional equipment component identification on pipeline engineering drawings. For example, object detection methods (such as the YOLO (You Only Look Once) model) can be used to identify equipment components within the search area of ​​the pipeline structure diagram on the pipeline engineering drawings, obtaining the first equipment component information. In particular, because it is necessary to output component type, component specifications, component connection parameters, and component location parameters, when using a model like YOLO, the model needs to be adapted. This could involve adding parameter classifiers to output component type, component specifications, and component connection parameters, as well as a location regressor to output component location parameters.

[0056] In practice, during the drawing of pipeline engineering drawings, equipment components often use similar legendary scales. Therefore, based on the pipeline engineering drawings (drawing size N×M), combined with the pipeline structure diagram, most equipment components connected by pipelines can be identified. By combining the characteristics of pipeline engineering drawings, areas other than pipelines and equipment components connected by pipelines are selectively masked, thereby achieving efficient equipment component identification.

[0057] In some optional implementations of certain embodiments, the execution entity performs fixed-resolution directional equipment component identification on the pipeline engineering drawings based on the pipeline structure diagram to obtain a first set of equipment component information, including: Step S1: Determine the first equipment search domain based on the above pipeline structure diagram.

[0058] The first equipment search domain is a drawing area constructed based on the pipeline structure diagram for identifying equipment components. Specifically, the first equipment search domain is a closed area formed by extending a preset distance to both sides of the pipeline in the pipeline structure diagram as the center line. The preset distance is controlled by the legend scale corresponding to the pipeline engineering drawing to ensure the effective setting of the first equipment search domain.

[0059] In practice, see Figure 4 The diagram illustrates the generation process of a local search domain within the first device search domain. Taking a local pipeline 401 as an example, the execution entity extends a preset distance (e.g., L) to both sides of the local pipeline 401, using the local pipeline 401 as the center line, and then encloses the area as the local search domain 402 corresponding to the local pipeline 401 within the first device search domain. Since the pipeline structure diagram represents the topological structure of the complete pipeline corresponding to the pipeline engineering drawing, the first device search domain can be constructed based on the pipeline structure diagram.

[0060] Step S2: Within the first equipment search domain, extract global drawing features from the pipeline engineering drawings to generate global drawing features.

[0061] Among them, the global drawing features are the semantic features of the drawings within the first device search domain.

[0062] In practice, since the first equipment search domain is generated based on the pipeline structure diagram, and the actual pipeline layout often results in an irregularly shaped area, the first equipment search domain is first divided into image blocks for the local drawing areas within the first equipment search domain and the aforementioned pipeline engineering drawings. Then, image encoding is used (e.g., through the Transformer Encoder module in the VIT (VisionTransformer) model) to generate image codes corresponding to each block area. Finally, the multiple image codes are sequentially concatenated according to the relative positions of the block areas within the first equipment search domain to obtain the global drawing features. Specifically, mainstream feature extraction processes primarily use the rectangle size as the input size. However, since the first equipment search domain is an irregularly shaped area, it is impossible to directly use the local drawing areas within the first equipment search domain and the aforementioned pipeline engineering drawings as input. Although the local drawing areas can be converted to rectangle sizes using zero-padding, this introduces irrelevant features and increases invalid feature calculations. Therefore, this disclosure adopts an image block + block encoding approach to construct global drawing features.

[0063] Step S3: Locate the region of interest based on the global drawing features described above to obtain the first set of regions of interest.

[0064] The first region of interest is the area suspected of containing device components.

[0065] In practice, a 2D convolutional detector head is used to locate the region of interest (ROI) using global drawing features as input, thus generating a first ROI. The 2D convolutional detector head is a position regressor composed of multiple serially connected 2D convolutional layers. The ROI is then determined using maximum suppression, serving as the first ROI.

[0066] Step S4: For each first region of interest in the above first region of interest set, perform the following first device component identification steps: Step S41: Extract the local drawing features from the global drawing features that correspond to the first region of interest.

[0067] In practice, the aforementioned execution entity can extract partial drawing features from the global drawing features at the location of the first region of interest, and reshape them into a 1×S feature dimension as local drawing features. In particular, (feature) reshaping can be achieved through multiple fully connected layers.

[0068] Step S42: Based on the above pipeline structure diagram, generate the local topological features corresponding to the first region of interest, as the first local topological features.

[0069] In practice, since the first region of interest is a sub-region within the first device search domain, the pipeline position vectors corresponding to the parts of the pipelines in the pipeline structure diagram that correspond to the positions of the first region of interest are extracted and shaped into a feature dimension of 1×S, which serves as the first local topological feature. In particular, (feature) shaping can be achieved through multiple fully connected layers.

[0070] Step S43: Perform dynamic feature fusion on the above-mentioned local drawing features and the above-mentioned first local topological features to obtain the first fused feature.

[0071] In practice, since the feature dimensions of the local drawing features and the first local topological features are the same (both 1×S), feature overlay at corresponding locations can be performed directly. To dynamically adjust the feature influence of the local drawing features and the first local topological features, a 1×2 learnable weight matrix is ​​used to control the feature fusion of the local drawing features and the first local topological features. The sum of the two weight values ​​in the weight matrix is ​​1. For example, the weight matrix could be [0.4 0.6], meaning the weight value corresponding to the local drawing features is 0.4, and the weight value corresponding to the first local topological features is 0.6. Therefore, the first fused feature = 0.4 × local drawing feature + 0.6 × first local topological feature.

[0072] Step S44: Based on the first fusion feature and component type classifier mentioned above, generate a first confidence level and the component types included in the first device component information corresponding to the first region of interest mentioned above.

[0073] The component type classifier consists of multiple fully connected layers, with a ReLU activation function between every two fully connected layers, and a Softmax function following the last fully connected layer. The output dimension of the last fully connected layer is 1×C, where C is the total number of different component types.

[0074] In practice, since the first fusion feature is obtained by fusing local drawing features and the aforementioned first local topology features, its corresponding feature dimension is still 1×S. Therefore, the first fusion feature can be mapped to the final component type through only multiple fully connected layers.

[0075] Step S45: Based on the first fusion feature and component specification parameter classifier described above, generate a second confidence level and component specification parameters included in the first device component information.

[0076] The component specification parameter classifier consists of multiple fully connected layers, with a ReLU activation function between every two fully connected layers, and a Softmax function following the last fully connected layer. The output dimension of the last fully connected layer is 1×D, where D is the total number of different component specifications.

[0077] Step S46: Based on the first fusion feature and component connection parameter classifier described above, generate a third confidence level and component connection parameters included in the first device component information.

[0078] The component connection parameter classifier consists of multiple fully connected layers. A ReLU activation function is applied between every two fully connected layers, and a Softmax function is applied after the last fully connected layer. The output dimension of the last fully connected layer is 1×W, where W is the preset total number of connection directions. For example, a value of 6 for W corresponds to 6 directional dimensions.

[0079] Step S47: Generate the fourth confidence level and the component location parameters included in the first device component information based on the region location corresponding to the first region of interest.

[0080] Since the first region of interest is a region suspected of containing device components, the region confidence level corresponding to the first region of interest in the region of interest location step can be directly used as the fourth confidence level, and the region location corresponding to the first region of interest (e.g., including but not limited to: region corner coordinates and region center coordinates) can be used as the component location parameters included in the first device component information.

[0081] Step S48: Perform a confidence-weighted fusion of the first confidence level, the second confidence level, the third confidence level, and the fourth confidence level to obtain the component confidence level included in the first device component information.

[0082] Component confidence score = Weight 1 × First confidence score + Weight 2 × Second confidence score + Weight 3 × Third confidence score + Weight 4 × Fourth confidence score. The sum of the weights of Weight 1, Weight 2, Weight 3, and Weight 4 is 1. This can form a 1×4 weight matrix containing Weight 1, Weight 2, Weight 3, and Weight 4. This 1×4 weight matrix can learn the weights through self-learning or by setting preset weight values.

[0083] Step 103: Based on the first set of equipment component information, perform area masking on the pipeline structure diagram to obtain the masked pipeline structure diagram.

[0084] In some embodiments, the aforementioned execution entity may perform regional masking on the pipeline structure diagram based on the first set of device component information to obtain a masked pipeline structure diagram.

[0085] Among them, the pipeline structure diagram after masking is a structure diagram in which the information of the first equipment component is masked at the corresponding position in the pipeline structure diagram.

[0086] In practice, there is no overlap between equipment components. Therefore, for areas containing equipment components that have been identified (corresponding to the first equipment component information), they can be masked. Specifically, for example, the corresponding position of the first equipment component information in the pipeline structure diagram can be updated to empty (the pipeline position vector is set to 0) to achieve area masking.

[0087] In some optional implementations of certain embodiments, the execution entity performs region masking on the pipeline structure diagram based on the first device component information set to obtain a masked pipeline structure diagram, including: Step S1: For each piece of information about a first device component in the aforementioned first device component information set, determine the masking area based on the component position parameters included in the aforementioned first device component information.

[0088] Since the component position parameter is characterized by the region position of the corresponding first region of interest, and the first region of interest is a sub-region in the first device search domain, which is a closed region formed by extending a preset distance to both sides with the pipeline as the center, the region where the pipeline of the first region of interest corresponding to the component position parameter is located can be determined as the masking region.

[0089] Step S2: Based on the obtained set of masking regions, perform topological masking on the above pipeline structure diagram to obtain the masked pipeline structure diagram.

[0090] In practice, the pipeline position vector of the local pipeline corresponding to the masking area in the pipeline structure diagram can be set to 0 to achieve topology masking.

[0091] Step 104: Based on the pipeline structure diagram after masking, perform dynamic resolution orientation equipment component identification on the pipeline engineering drawings to obtain the second equipment component information set.

[0092] In some embodiments, the aforementioned execution entity can perform dynamic resolution directional equipment component identification on the pipeline engineering drawings based on the masked pipeline structure diagram to obtain a second set of equipment component information. The second set of equipment component information is identical to the first set of equipment component information, both including: component type, component specification parameters, component connection parameters, component location parameters, and component confidence level.

[0093] In practice, the fixed-resolution directional equipment component identification process used in step 102 can identify most equipment components connected by pipelines, but there are still cases where smaller proportions of equipment components are missed due to insufficient feature representation at the fixed resolution. Therefore, by combining the masked pipeline structure diagram, a second equipment component identification is performed on the area other than the area corresponding to the first equipment component information. Since it failed to effectively identify the components at the fixed resolution, a dynamic resolution approach is used for directional equipment component identification in the second equipment component identification process. Specifically, before identification, multi-scale feature extraction is performed using the FPN model, and then equipment component identification is performed on the search area in the pipeline engineering drawing and the location of the pipeline structure diagram using object detection (e.g., YOLO (You Only Look Once) model) to obtain the second equipment component information. In particular, since it is necessary to output component type, component specification parameters, component connection parameters, and component location parameters, when using a model such as YOLO, the model needs to be adapted. For example, a parameter classifier for outputting component type, component specification parameters, and component connection parameters, and a location regressor for outputting component location parameters need to be added. The purpose of this design is to add a multi-scale feature extraction component to the model used in the fixed-resolution recognition process, thereby achieving multi-scale feature extraction under dynamic resolution, reducing model development costs, and ensuring process consistency in the device component recognition process.

[0094] In some optional implementations of certain embodiments, the execution entity performs dynamic resolution directional equipment component identification on the pipeline engineering drawings based on the masked pipeline structure diagram to obtain a second set of equipment component information, including: Step S1: Determine the second device search domain based on the above-mentioned pipeline structure diagram after masking.

[0095] The creation process of the second device search domain is consistent with that of the first device search domain. Specifically, the second device search domain is a closed area formed by extending a preset distance to both sides of the pipeline in the pipeline structure diagram, with the pipeline as the center line. The preset distance is controlled by the legend scale corresponding to the pipeline engineering drawing, thus ensuring the effective setting of the second device search domain. Since the pipeline structure diagram is masked using the information of the first device components, the second device search domain is a sub-region of the first device search domain.

[0096] Step S2: Within the second equipment search domain, perform multi-resolution scale global drawing feature extraction on the pipeline engineering drawings to obtain multi-scale global drawing features.

[0097] Among them, the multi-scale global drawing features are multi-scale semantic representations of drawings within the second device search domain.

[0098] In practice, similar to the first equipment search domain, the second equipment search domain also relies on the (masked) pipeline structure diagram for generation. However, due to the actual pipeline layout, the second equipment search domain is often an irregularly shaped area. Therefore, the local drawing areas within the pipeline engineering drawings of the first equipment search domain are first divided into image blocks. Then, image encoding is generated for each block area using graph encoding (e.g., through the Transformer Encoder module in the VIT (Vision Transformer) model). Next, multi-scale feature extraction is performed on the graph encoding using the FPN model to obtain multi-scale graph encoding. Finally, according to the relative positions of the block areas in the first equipment search domain, the multiple graph encodings of the same scale are sequentially concatenated to obtain multi-scale global drawing features.

[0099] Step S3: Locate the region of interest based on the multi-scale global map features described above to obtain the second set of regions of interest.

[0100] The second region of interest is the area suspected of containing device components.

[0101] In practice, since global drawing features are single-scale features while multi-scale global drawing features are multi-scale features, it is necessary to upsample the drawing features corresponding to different scales in the multi-scale global drawing features, then stack the features, and finally use a 2D convolutional detector head to locate the region of interest (ROI) using the stacked drawing features as input to generate a second ROI. The 2D convolutional detector head is a position regressor composed of multiple 2D convolutional layers connected in series. It uses maximum suppression to determine the ROI, which is then used as the second ROI.

[0102] Step S4: For each second region of interest in the above set of second regions of interest, perform the following second device component identification steps: Step S41: Extract the multi-scale local map features corresponding to the second region of interest from the multi-scale global map features.

[0103] In practice, the aforementioned execution entity can extract some multi-scale map features from the multi-scale global map features at the location of the second region of interest, and shape them into a 1×S feature dimension as multi-scale local map features. Specifically, compared to the generation of local map features, multi-scale local map features involve multi-scale feature representation. Therefore, after extracting some multi-scale map features, it is necessary to first upsample the map features at different scales within the partial multi-scale map features, then perform feature overlay, and finally shape the result of the feature overlay to obtain multi-scale local map features with a feature dimension of 1×S. In particular, (feature) shaping can be achieved through multiple fully connected layers.

[0104] Step S42: Based on the above-mentioned pipeline structure diagram after masking, generate local topological features corresponding to the second region of interest, as the second local topological features.

[0105] In practice, since the second region of interest is a sub-region within the second device search domain, the pipeline position vectors corresponding to the parts of the pipelines in the pipeline structure diagram that correspond to the positions of the second region of interest are extracted and shaped into a 1×S feature dimension, which serves as the first local topological feature. In particular, this (feature) shaping can be achieved through multiple fully connected layers.

[0106] Step S43: Perform dynamic feature fusion on the above-mentioned multi-scale local drawing features and the above-mentioned second local topological features to obtain the second fused feature.

[0107] In practice, since the feature dimensions of the multi-scale local drawing features and the second local topological features are the same (both 1×S), features at corresponding positions can be directly superimposed. To dynamically adjust the feature influence of the multi-scale local drawing features and the second local topological features, a 1×2 learnable weight matrix is ​​used to control the feature fusion of the two features. The sum of the two weight values ​​in the weight matrix is ​​1. For example, the weight matrix could be [0.4 0.6], meaning the weight value corresponding to the multi-scale local drawing features is 0.4, and the weight value corresponding to the second local topological features is 0.6. Therefore, the second fused feature = 0.4 × multi-scale local drawing features + 0.6 × second local topological features. Specifically, the first fused feature and the second fused feature can share the same 1×2 learnable weight matrix.

[0108] Step S44: Generate second device component information based on the second fusion feature, the component type classifier, the component specification parameter classifier, and the component connection parameter classifier.

[0109] In practice, firstly, the aforementioned executing entity can sequentially input the second fusion feature into the component type classifier, the aforementioned component specification parameter classifier, and the aforementioned component connection parameter classifier to obtain the component type, component specification parameters, and component connection parameters included in the second device component information. Secondly, the location of the region corresponding to the second region of interest (e.g., including but not limited to: region corner coordinates and region center coordinates) can be used as the component location parameter included in the second device component information. Furthermore, the component confidence score included in the second device component information is also obtained by weighted fusion of four confidence scores (including: three confidence scores output by the component type classifier, the aforementioned component specification parameter classifier, and the aforementioned component connection parameter classifier, and one region confidence score corresponding to the second region of interest), and the 1×4 weight matrix used is the same as the weight matrix used for the component confidence score included in the first device component information.

[0110] Step 105: Perform global topology verification based on the pipeline structure diagram, the first equipment component information set, and the second equipment component information set.

[0111] In some embodiments, the aforementioned execution entity may perform global topology verification based on the pipeline structure diagram, the first set of device component information, and the second set of device component information.

[0112] In practice, since the pipeline structure diagram (pipeline topology) and the equipment components connected via the pipelines (first equipment component information and second equipment component information) have already been identified, automatic topology verification can be performed by combining multiple preset topology connection rules with the pipeline structure diagram, the first equipment component information set, and the second equipment component information set. The topology connection rules represent pre-set connection rules that meet the requirements of construction and operational safety.

[0113] In some optional implementations of certain embodiments, the execution entity performs a global topology check based on the pipeline structure diagram, the first device component information set, and the second device component information set, including: Step S1: Perform position conflict verification based on the first set of device component information and the second set of device component information.

[0114] In practice, since the information of the second equipment component is obtained based on the pipeline structure diagram after masking, there is no positional conflict between the equipment components corresponding to the first and second equipment component information. Therefore, positional conflict verification is mainly used to verify conflicts in connection sequence and process sequence. Specifically, positional conflict verification can be performed based on multiple pre-set positional conflict verification rules, combined with the aforementioned first and second equipment component information sets. These conflict verification rules may include, but are not limited to: process sequence constraint rules, cooling equipment connection constraint rules, filtration sequence constraint rules, and instrument connection sequence rules. Process sequence constraint rules are connection rules for equipment components set according to specific process requirements. Cooling equipment connection constraint rules are connection rules used to restrict the corresponding installation positions of cooling equipment (equipment components). Filtration sequence constraint rules are connection rules used to restrict the corresponding installation positions of filter equipment (equipment components). Instrument connection sequence rules are connection rules used to restrict the corresponding installation positions of monitoring instruments (equipment components).

[0115] Step S2: In response to passing the location conflict check, generate a topology node set based on the first device component information set and the second device component information set.

[0116] Among them, the topology node represents the device component corresponding to the first device component information or the second device component information.

[0117] In practice, topology nodes can be created sequentially based on the information of the first device component or the information of the second device component to obtain a set of topology nodes.

[0118] Step S3: Based on the above set of topological nodes, perform graph update on the above pipeline structure diagram to obtain the updated pipeline structure diagram.

[0119] In practice, since topology nodes correspond to information about the first or second device components, topology nodes can be added to the pipeline structure diagram based on the component location parameters included in the information about the first or second device components, thereby obtaining an updated pipeline structure diagram.

[0120] Step S4: Perform a connectivity check on the updated pipeline structure diagram.

[0121] The updated pipeline structure diagram is a directed graph structure, so the connectivity of the updated pipeline structure diagram can be checked by graph traversal. When there is a "loop" or isolated topology node, a check result indicating that the connectivity check failed is generated; otherwise, a check result indicating that the connectivity check passed is generated.

[0122] Step S5: In response to passing the connectivity check, generate a check result representing that the global topology check has passed.

[0123] Step S6: In response to failing the location conflict check or failing the connectivity check, generate a check result indicating that the global topology check failed.

[0124] In practice, when a location conflict check fails, a check result representing the failure can be generated based on the unmet constraint rules, and this result can be used as the check result for failing the global topology check. Similarly, when a connectivity check fails, a check result representing the failure can be used as the check result for failing the global topology check.

[0125] Step 106: In response to global topology verification, render the 3D pipeline model based on the pipeline structure diagram, the first equipment component information set, and the second equipment component information set to obtain the pipeline model corresponding to the pipeline engineering drawing.

[0126] In some embodiments, the aforementioned execution entity may, in response to global topology verification, render a three-dimensional pipeline model based on the pipeline structure diagram, the first set of equipment component information, and the second set of equipment component information to obtain the pipeline model corresponding to the pipeline engineering drawings.

[0127] In practice, pipeline structure diagrams, information sets of the first and second equipment components can be imported into a BIM (Building Information Modeling) platform for automatic 3D pipeline model rendering to obtain the pipeline model corresponding to the pipeline engineering drawings.

[0128] In some optional implementations of certain embodiments, in response to global topology verification, the execution entity performs three-dimensional pipeline model rendering based on the pipeline structure diagram, the first equipment component information set, and the second equipment component information set to obtain the pipeline model corresponding to the pipeline engineering drawing, including: Step S1: Render the pipeline according to the above pipeline structure diagram to obtain the initial pipeline model.

[0129] The initial pipeline model is a three-dimensional model with the pipeline as its framework.

[0130] In practice, since the structure of pipelines is relatively simple, the pipeline structure diagram can be combined to perform three-dimensional structural rendering of the pipeline to obtain the initial pipeline model.

[0131] Step S2: Extract the device component model corresponding to the first device component information in the first device component information set and the device component model corresponding to the second device component information in the second device component information set from the device component model library to obtain the device component model set.

[0132] The equipment component model library is a pre-built database that stores standardized equipment component models. It stores component descriptions, simplified diagrams, and 3D models of each component. For example, see... Figure 5 The diagram shows the visualization interface corresponding to the device component model library. The visualization interface includes a navigation bar 501 and a device component diagram visualization window 502. The navigation bar 501 is used to search for device components, and the device component diagram visualization window 502 is used to display the component diagrams corresponding to the device components in a visual format.

[0133] Step S3: Based on the above set of equipment component models, update the above initial pipeline model to obtain the pipeline model corresponding to the above pipeline engineering drawings.

[0134] In practice, the aforementioned execution entity can add equipment component models to their corresponding positions in the initial pipeline model through model replacement, thereby obtaining the pipeline model corresponding to the aforementioned pipeline engineering drawings. This method standardizes the 3D models of different equipment components, ensuring the standardization of the constructed 3D models. Furthermore, by pre-setting the component 3D models, model remodeling is avoided, meaning that only certain rendering resources are needed for pipeline rendering during pipeline model creation, thus reducing hardware dependence during the 3D model rendering process.

[0135] The various embodiments of this disclosure have the following beneficial effects: The pipeline modeling method based on engineering drawing recognition, as described in some embodiments of this disclosure, achieves efficient and accurate pipeline modeling. Specifically, the pipeline modeling method based on engineering drawing recognition, as described in some embodiments of this disclosure, firstly, performs pipeline recognition on the pipeline engineering drawing to obtain a pipeline structure diagram. The pipeline engineering drawing is an electronic drawing to be used to generate the corresponding pipeline model, and the pipeline structure diagram represents the topological structure of the pipeline. In practice, the pixel proportion of pipelines and the equipment components connected by the pipelines in the pipeline engineering drawing is often small. If the pipeline engineering drawing is used as a whole for recognition, there will be a large number of invalid inference processes, leading to a large consumption of computing resources. This disclosure considers that the equipment pipeline, as a skeleton structure, connects different equipment components; therefore, this disclosure uses pipeline recognition to quickly extract the pipeline topology structure from the pipeline engineering drawing. Secondly, based on the aforementioned pipeline structure diagram, the pipeline engineering drawings are subjected to fixed-resolution directional equipment component identification to obtain a first set of equipment component information. This first set of information includes: component type, component specifications, component connection parameters, component location parameters, and component confidence level. By combining directional and fixed-resolution identification with the pipeline structure diagram, equipment components with distinct characteristics can be identified quickly and effectively. Next, based on the first set of equipment component information, the pipeline structure diagram is masked to obtain a masked pipeline structure diagram. In practice, considering the differences in specifications among different equipment components, and the potential impact of the pipeline engineering drawings' specifications on identification effectiveness, region masking is used to mask equipment components in the pipeline structure diagram that correspond to the first set of equipment component information to avoid redundant identification during subsequent refined identification processes. Further, based on the masked pipeline structure diagram, the pipeline engineering drawings are subjected to dynamic-resolution directional equipment component identification to obtain a second set of equipment component information. In practice, to avoid omissions due to the small size of equipment components at a fixed resolution, this disclosure employs dynamic resolution-based directional identification to supplement the identification of equipment components. Furthermore, a global topology check is performed based on the aforementioned pipeline structure diagram, the first set of equipment component information, and the second set of equipment component information. This global topology check determines the rationality of the topological connections between pipelines and equipment components. Finally, in response to the global topology check, a 3D pipeline model is rendered based on the aforementioned pipeline structure diagram, the first set of equipment component information, and the second set of equipment component information, resulting in the pipeline model corresponding to the aforementioned pipeline engineering drawings. This method achieves efficient and accurate automatic modeling of pipeline models.

[0136] Further reference Figure 6As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a pipeline modeling device based on engineering drawing recognition. These device embodiments are similar to... Figure 1 Corresponding to the method embodiments shown, this pipeline modeling device based on engineering drawing recognition can be specifically applied to various electronic devices.

[0137] like Figure 6 As shown, a pipeline modeling device 600 based on engineering drawing recognition in some embodiments includes: a pipeline recognition unit 601, a first recognition unit 602, a region masking unit 603, a second recognition unit 604, a global topology verification unit 605, and a three-dimensional pipeline model rendering unit 606, wherein, Pipeline identification unit 601 is configured to identify pipelines in pipeline engineering drawings to obtain a pipeline structure diagram, wherein the pipeline engineering drawings are electronic drawings for generating a corresponding pipeline model, and the pipeline structure diagram represents the topology of the pipeline; first identification unit 602 is configured to identify directional equipment components in the pipeline engineering drawings at a fixed resolution based on the pipeline structure diagram to obtain a first equipment component information set, wherein the first equipment component information includes: component type, component specification parameters, component connection parameters, component location parameters, and component confidence level; region masking unit 603 is configured to perform region masking on the pipeline structure diagram based on the first equipment component information set. The system obtains a masked pipeline structure diagram; a second identification unit 604 is configured to perform dynamic resolution directional device component identification on the pipeline engineering drawing based on the masked pipeline structure diagram to obtain a second device component information set; a global topology verification unit 605 is configured to perform global topology verification based on the pipeline structure diagram, the first device component information set, and the second device component information set; and a three-dimensional pipeline model rendering unit 606 is configured to perform three-dimensional pipeline model rendering based on the pipeline structure diagram, the first device component information set, and the second device component information set in response to passing the global topology verification to obtain the pipeline model corresponding to the pipeline engineering drawing.

[0138] It is understandable that the units and references recorded in the pipeline modeling device 600 based on engineering drawing recognition are... Figure 1 The steps described in the method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method are also applicable to the pipeline modeling device 600 based on engineering drawing recognition and the units contained therein, and will not be repeated here.

[0139] The following is for reference. Figure 7 It shows a schematic diagram of the structure of an electronic device (e.g., a computing device) 700 suitable for implementing some embodiments of the present disclosure. Figure 7The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0140] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory 702 or a program loaded from a storage device 708 into a random access memory 703. The random access memory 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, the read-only memory 702, and the random access memory 703 are interconnected via a bus 704. An input / output interface 705 is also connected to the bus 704.

[0141] Typically, the following devices can be connected to the input / output interface 705: input devices 706 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 708 including, for example, magnetic tape, hard disk, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 7 Each box shown can represent a device or multiple devices as needed.

[0142] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 709, or installed from a storage device 708, or installed from a read-only memory 702. When the computer program is executed by the processing device 701, it performs the functions defined in the methods of some embodiments of this disclosure.

[0143] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0144] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0145] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: identify pipelines in pipeline engineering drawings to obtain a pipeline structure diagram, wherein the aforementioned pipeline engineering drawings are electronic drawings for generating corresponding pipeline models, and the aforementioned pipeline structure diagram represents the topology of the pipeline; based on the aforementioned pipeline structure diagram, identify directional equipment components in the aforementioned pipeline engineering drawings at a fixed resolution to obtain a first set of equipment component information, wherein the first set of equipment component information includes: component type, component specification parameters, component connection parameters, component location parameters, and component confidence level; and based on... Based on the aforementioned first set of equipment component information, the aforementioned pipeline structure diagram is region-masked to obtain a masked pipeline structure diagram; based on the masked pipeline structure diagram, the aforementioned pipeline engineering drawing is dynamically resolution-based directional equipment component identification to obtain a second set of equipment component information; based on the aforementioned pipeline structure diagram, the aforementioned first set of equipment component information, and the aforementioned second set of equipment component information, a global topology verification is performed; in response to passing the global topology verification, a three-dimensional pipeline model is rendered based on the aforementioned pipeline structure diagram, the aforementioned first set of equipment component information, and the aforementioned second set of equipment component information to obtain the pipeline model corresponding to the aforementioned pipeline engineering drawing.

[0146] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0148] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0149] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A pipeline modeling method based on engineering drawing recognition, characterized in that, include: Pipeline engineering drawings are used to identify pipelines and obtain pipeline structure diagrams. The pipeline engineering drawings are electronic drawings for generating corresponding pipeline models, and the pipeline structure diagrams represent the topological structure of the pipelines. Based on the pipeline structure diagram, the pipeline engineering drawings are subjected to fixed-resolution directional equipment component identification to obtain a first set of equipment component information, wherein the first set of equipment component information includes: component type, component specification parameters, component connection parameters, component location parameters, and component confidence level; Based on the first set of equipment component information, the pipeline structure diagram is masked to obtain a masked pipeline structure diagram. Based on the pipeline structure diagram after masking, the pipeline engineering drawings are dynamically and resolvingly identified to identify directional equipment components, thereby obtaining a second set of equipment component information. Global topology verification is performed based on the pipeline structure diagram, the first equipment component information set, and the second equipment component information set. In response to passing a global topology verification, a 3D pipeline model is rendered based on the pipeline structure diagram, the first equipment component information set, and the second equipment component information set to obtain the pipeline model corresponding to the pipeline engineering drawing. The step involves dynamically resolving the directional equipment component identification on the pipeline engineering drawings based on the masked pipeline structure diagram to obtain a second set of equipment component information, including: The second device search domain is determined based on the pipeline structure diagram after the shielding is applied. Within the second device search domain, the pipeline engineering drawings are subjected to multi-resolution scale global drawing feature extraction to obtain multi-scale global drawing features; Based on the multi-scale global drawing features, the region of interest is located to obtain a second set of regions of interest; For each second region of interest in the second region of interest set, perform the following second device component identification step: Extract the multi-scale local drawing features corresponding to the second region of interest from the multi-scale global drawing features; Based on the pipeline structure diagram after masking, a local topological feature corresponding to the second region of interest is generated as the second local topological feature. Dynamic feature fusion is performed on the multi-scale local drawing features and the second local topological features to obtain the second fused feature; Based on the second fusion feature, component type classifier, component specification parameter classifier, and component connection parameter classifier, second device component information is generated.

2. The pipeline modeling method based on engineering drawing recognition according to claim 1, characterized in that, The global topology verification based on the pipeline structure diagram, the first equipment component information set, and the second equipment component information set includes: Position conflict verification is performed based on the first device component information set and the second device component information set; In response to passing the location conflict check, a topology node set is generated based on the first device component information set and the second device component information set; Based on the set of topological nodes, the pipeline structure diagram is updated to obtain the updated pipeline structure diagram; The updated pipeline structure diagram is then subjected to connectivity verification. In response to passing the connectivity check, generate a check result representing that the global topology check has passed; In response to failing the location conflict check or the connectivity check, a check result indicating that the global topology check failed is generated.

3. The pipeline modeling method based on engineering drawing recognition according to claim 2, characterized in that, The step of responding to global topology verification by rendering a 3D pipeline model based on the pipeline structure diagram, the first equipment component information set, and the second equipment component information set to obtain the pipeline model corresponding to the pipeline engineering drawing includes: Based on the pipeline structure diagram, the pipeline is rendered to obtain the initial pipeline model; Extract the device component model corresponding to the first device component information in the first device component information set and the device component model corresponding to the second device component information in the second device component information set from the device component model library respectively to obtain the device component model set; Based on the set of equipment component models, the initial pipeline model is updated to obtain the pipeline model corresponding to the pipeline engineering drawings.

4. The pipeline modeling method based on engineering drawing recognition according to claim 3, characterized in that, The process of identifying pipelines in pipeline engineering drawings to obtain pipeline structure diagrams includes: Rotate the pipeline engineering drawing by a preset angle in a clockwise direction to obtain the rotated pipeline engineering drawing; The rotated pipeline engineering drawing is divided into drawing blocks to obtain a set of drawing image blocks, wherein there is an overlapping area between every two adjacent drawing image blocks in the set of drawing image blocks; For each drawing image block in the set of drawing image blocks, perform the following pipeline identification steps: Pipeline features are extracted from the drawing image blocks using a pipeline feature extraction network to obtain a pipeline feature map. The pipeline feature extraction network consists of a multi-scale feature extraction module and a feature fusion module. Candidate pipeline description information is generated using the pipeline localization network and the pipeline feature map. The candidate pipeline description information includes: pipeline location vector and pipeline confidence. The pipeline feature extraction network and the pipeline localization network are included in the pipeline identification network. In response to the candidate pipeline description information having a pipeline confidence level greater than or equal to a preset pipeline confidence level, the candidate pipeline description information is determined as the target pipeline description information; In response to the fact that the pipeline confidence score included in the candidate pipeline description information is less than the preset pipeline confidence score, the candidate pipeline description information is updated by voting according to the target drawing image block group to obtain the target pipeline description information, wherein the target drawing image block in the target drawing image block group is a drawing image block that has an overlapping area with the drawing image block; Based on the obtained target pipeline description information set, the topology is restored to obtain the candidate pipeline structure diagram; Rotate the candidate pipeline structure diagram by the preset angle in a counterclockwise direction to obtain the pipeline structure diagram.

5. The pipeline modeling method based on engineering drawing recognition according to claim 4, characterized in that, The step of identifying directional equipment components on the pipeline engineering drawings at a fixed resolution based on the pipeline structure diagram to obtain a first set of equipment component information includes: Based on the pipeline structure diagram, the first equipment search domain is determined; Within the first device search domain, global drawing features are extracted from the pipeline engineering drawings to generate global drawing features; Based on the global drawing features, the region of interest is located to obtain a first set of regions of interest; For each region of interest in the first set of regions of interest, the following first device component identification steps are performed: Extract the local drawing features corresponding to the first region of interest from the global drawing features; Based on the pipeline structure diagram, local topological features corresponding to the first region of interest are generated as the first local topological features. Dynamic feature fusion is performed on the local drawing features and the first local topological features to obtain the first fused feature; Based on the first fusion feature and component type classifier, generate the first confidence level and the component types included in the first device component information corresponding to the first region of interest; Based on the first fusion feature and the component specification parameter classifier, a second confidence level and the component specification parameters included in the first device component information are generated; Based on the first fusion feature and the component connection parameter classifier, a third confidence level and the component connection parameters included in the first device component information are generated; Based on the location of the region corresponding to the first region of interest, a fourth confidence level and component location parameters included in the first device component information are generated; The confidence levels of the first, second, third, and fourth confidence levels are weighted and fused to obtain the component confidence levels included in the first device component information.

6. The pipeline modeling method based on engineering drawing recognition according to claim 5, characterized in that, The step of performing region masking on the pipeline structure diagram based on the first set of equipment component information to obtain a masked pipeline structure diagram includes: For each piece of first device component information in the first device component information set, the masking area is determined based on the component position parameters included in the first device component information; Based on the obtained set of masking regions, the pipeline structure diagram is topologically masked to obtain the masked pipeline structure diagram.

7. A pipeline modeling device based on engineering drawing recognition, characterized in that, include: The pipeline identification unit is configured to identify pipelines in pipeline engineering drawings to obtain a pipeline structure diagram, wherein the pipeline engineering drawings are electronic drawings for generating a corresponding pipeline model, and the pipeline structure diagram represents the topological structure of the pipeline. The first identification unit is configured to perform fixed-resolution directional equipment component identification on the pipeline engineering drawing based on the pipeline structure diagram to obtain a first equipment component information set, wherein the first equipment component information includes: component type, component specification parameters, component connection parameters, component location parameters, and component confidence level; The region masking unit is configured to perform region masking on the pipeline structure diagram based on the first device component information set to obtain a masked pipeline structure diagram. The second identification unit is configured to perform dynamic resolution orientation device component identification on the pipeline engineering drawing based on the pipeline structure diagram after masking, and obtain a second set of device component information. The global topology verification unit is configured to perform global topology verification based on the pipeline structure diagram, the first device component information set, and the second device component information set. A 3D pipeline model rendering unit is configured to, in response to a global topology check, render a 3D pipeline model based on the pipeline structure diagram, the first set of equipment component information, and the second set of equipment component information to obtain the pipeline model corresponding to the pipeline engineering drawing. The step involves dynamically resolving the directional equipment component identification on the pipeline engineering drawings based on the masked pipeline structure diagram to obtain a second set of equipment component information, including: The second device search domain is determined based on the pipeline structure diagram after the shielding is applied. Within the second device search domain, the pipeline engineering drawings are subjected to multi-resolution scale global drawing feature extraction to obtain multi-scale global drawing features; Based on the multi-scale global drawing features, the region of interest is located to obtain a second set of regions of interest; For each second region of interest in the second region of interest set, perform the following second device component identification step: Extract the multi-scale local drawing features corresponding to the second region of interest from the multi-scale global drawing features; Based on the pipeline structure diagram after masking, a local topological feature corresponding to the second region of interest is generated as the second local topological feature. Dynamic feature fusion is performed on the multi-scale local drawing features and the second local topological features to obtain the second fused feature; Based on the second fusion feature, component type classifier, component specification parameter classifier, and component connection parameter classifier, second device component information is generated.

8. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 6.

9. A computer-readable medium, characterized in that, It stores a computer program thereon, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images