Device identification methods, apparatus, storage media, and electronic equipment based on P&ID drawings

Abstract

This application relates to the field of image processing technology, specifically to a device identification method, apparatus, storage medium, and electronic device based on P&ID drawings. The method includes: cutting a P&ID drawing image using a sliding window to obtain multiple cut images; inputting the multiple cut images into a preset component identification model to extract component information and component regions; removing component regions from the P&ID drawing image to obtain a component-missing image; performing image erosion on the component-missing image to determine pipeline information within the component-missing image; and determining the device identification result based on the component information and the pipeline information. This invention enables component identification while shielding pipeline interference, and pipeline identification while shielding component interference, further improving the automatic identification accuracy of P&ID drawings. It is also applicable to various P&ID drawings and has a wide range of applications.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a device identification method, apparatus, storage medium and electronic device based on P&ID drawings. Background Technology

[0002] P&ID drawings (Process & Instrumentation Drawings) are completed collaboratively by process, piping installation, and automation professionals, based on PFD (Program Flow Diagram). They reflect the comprehensive results of process design, equipment design, equipment and piping layout design, and automation instrumentation design. They are an integral part of the complete technical documentation required for enterprise management, commissioning, operation, maintenance, and start-up / shutdown. During the actual construction of petrochemical equipment and piping, personnel frequently need to consult various P&ID drawings to understand the relationships between equipment and piping. For a petrochemical plant, all equipment and piping information involves a large number of P&ID drawings; therefore, quickly obtaining the information needed by staff has become a common requirement in many petrochemical engineering projects.

[0003] Currently, most P&ID drawing recognition algorithms are limited to the recognition of a small number of components. By recognizing a few fixed key components and combining traditional production experience, the relationships between the components are statistically analyzed to infer the deployment location of the pipeline. This method has a relatively simple function, poor versatility, and poor performance in recognizing complex P&ID drawings. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a device identification method, apparatus, storage medium, and electronic device based on P&ID drawings. The method first identifies the components, then removes the components to identify pipelines, and finally determines the connection relationships between all devices to obtain the device identification result. This method is adaptable to various P&ID drawings and has a higher identification accuracy.

[0005] In a first aspect, embodiments of this application provide a device identification method based on P&ID drawings applied to a server, the method comprising:

[0006] Use a sliding window to cut the P&ID drawing image to obtain multiple cut images;

[0007] The multiple cut images are input into a preset component recognition model to extract component information and component regions;

[0008] Remove the component areas from the P&ID drawing image to obtain a component missing image;

[0009] Image erosion is performed on the missing component image to determine the pipeline information in the missing component image;

[0010] The device identification result is determined based on the component information and the pipeline information.

[0011] In some embodiments, before inputting the plurality of cut images into a preset component recognition model to extract component information and component regions, the method further includes:

[0012] Construct an initial classification model based on a deep residual convolutional neural network;

[0013] The initial classification model is trained using preset component images to obtain a component classification model.

[0014] In some embodiments, after inputting the plurality of cut images into a preset component recognition model to extract component information and component regions, the method further includes:

[0015] A preset OCR recognition model is used to detect the component number and pipeline number in the component area to determine the location of the component corresponding to the component number and the corresponding component type.

[0016] In some embodiments, removing the component area from the P&ID drawing image to obtain a component-missing image includes:

[0017] Based on the component area, the corresponding area on the P&ID drawing image is blackened to obtain a blackened image of the missing component.

[0018] In some embodiments, performing image erosion on the missing component image to determine pipeline information in the missing component image includes:

[0019] Image erosion is performed on the image of the missing component to remove invalid information in the image of the missing component, resulting in a processed image;

[0020] Extract horizontal and vertical lines from the processed image;

[0021] The horizontal line and / or the vertical line corresponding to the pipeline number are determined to obtain pipeline information.

[0022] In some embodiments, performing image erosion on the missing component image to remove invalid information from the missing component image, resulting in a processed image, includes:

[0023] Binarization is performed on the image of the missing component to obtain a binarized image;

[0024] The pixel values ​​of the binarized image are flipped to obtain a flipped image;

[0025] The processed image is obtained by performing image erosion on the flipped image.

[0026] In some embodiments, determining the device identification result based on the component information and the pipeline information includes:

[0027] The component connection relationship is determined based on the pipeline location and the component location, and the equipment identification result is determined based on the component connection relationship, the component number and the pipeline number.

[0028] Secondly, embodiments of this application provide a device identification apparatus based on P&ID drawings, including:

[0029] The image cutting module is used to cut P&ID drawing images using a sliding window to obtain multiple cut images;

[0030] The component recognition module is used to input the multiple cut images into a preset component recognition model to extract component information and component regions;

[0031] The component removal module is used to remove component areas from the P&ID drawing image to obtain a component missing image.

[0032] A pipeline identification module is used to perform image erosion on the missing component image to determine the pipeline information in the missing component image;

[0033] The result determination module is used to determine the device identification result based on the component information and the pipeline information.

[0034] Thirdly, embodiments of this application also provide a storage medium storing a computer program that can be executed by one or more processors and can be used to implement the device identification method based on P&ID drawings as provided in any embodiment of the present invention.

[0035] Fourthly, embodiments of this application also provide an electronic device, including a memory and a processor. The memory stores a computer program, and the memory and the processor are communicatively connected to each other. When the computer program is executed by the processor, it performs a device identification method based on P&ID drawings as provided in any embodiment of the present invention.

[0036] The device identification method based on P&ID drawings provided in this application first uses a sliding window to cut the P&ID drawing image to obtain multiple cut images. The multiple cut images are then input into a preset component identification model to extract component information and component regions. Next, the component regions are removed from the P&ID drawing image to obtain a component missing image. Based on the component missing image, pipeline information in the component missing image is determined through image erosion. Finally, the device identification result is determined based on the component information and pipeline information. This method can perform component identification while shielding pipeline interference, and pipeline identification while shielding component interference, further improving the automatic identification accuracy of P&ID drawings. It is also applicable to a wide range of P&ID drawings. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 A flowchart illustrating a device identification method based on P&ID drawings provided in this application embodiment;

[0039] Figure 2 A flowchart illustrating another device identification method based on P&ID drawings provided in this application embodiment;

[0040] Figure 3 A flowchart illustrating another device identification method based on P&ID drawings provided in this application embodiment;

[0041] Figure 4 A schematic diagram of a sub-process of a device identification method based on P&ID drawings provided in an embodiment of this application;

[0042] Figure 5 A schematic diagram of a device identification device based on P&ID drawings provided in this application embodiment;

[0043] Figure 6 This is a connection block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0044] The following detailed description of the embodiments of this application, in conjunction with the accompanying drawings, will provide a thorough understanding of how this application uses technical means to solve technical problems and achieve corresponding technical effects, enabling its implementation. The embodiments of this application and the various features within them can be combined with each other without conflict, and all resulting technical solutions are within the protection scope of this application.

[0045] As can be seen from the background technology, the existing equipment recognition process based on P&ID drawings has limited applicability and low automatic recognition accuracy. It relies heavily on manual assistance to improve recognition accuracy, resulting in low efficiency.

[0046] In view of this, this application provides a device identification method, apparatus, storage medium and electronic device based on P&ID drawings, which can identify components while shielding pipeline interference and identify pipelines while shielding component interference, further improving the automatic identification accuracy of P&ID drawings, and is applicable to a wide range of P&ID drawing applications.

[0047] Example 1

[0048] Figure 1 This application provides a flowchart illustrating a device identification method based on P&ID drawings. The method can be executed by the pipeline identification device described in this application. This device can be implemented through software and / or hardware. The method specifically includes:

[0049] Step 101: Use a sliding window to cut the P&ID drawing image to obtain multiple cut images.

[0050] P&ID drawing images are images captured based on P&ID drawings. These can be obtained directly from the P&ID drawings or from other devices such as servers. A sliding window is a rectangular window used to segment the P&ID drawing image. The size of the sliding window is set according to specific needs and the size of the P&ID drawing image. For example, if the original P&ID image is stored as 4764×3368 high-pixel data, the sliding window is a rectangular window of 640x640 pixels. A cut image is an image of the same size as the sliding window, cut from the P&ID drawing image. Specifically, during the cutting process, each movement of the sliding window generates a corresponding cut image until the sliding window has traversed every pixel in the P&ID drawing image.

[0051] Step 102: Input the multiple cut images into a preset component recognition model to extract component information and component regions.

[0052] The preset component recognition model is a pre-set neural network model for identifying components. The model's input is an image, and its output is features identified from the image. These features include component type and component location. The component location can be the coordinates of the component's center or boundary in the P&ID drawing image. Specifically, in this embodiment, the component location is determined by the component's bounding box, and the component region refers to the area occupied by the component in the P&ID drawing image, determined based on the component information.

[0053] Step 103: Remove the component areas from the P&ID drawing image to obtain a component missing image.

[0054] The missing component image refers to the image obtained after removing the component area from the P&ID drawing image. Specifically, removal here refers to blackening, that is, blackening the corresponding area on the P&ID drawing image according to the component area to obtain the blackened missing component image.

[0055] Step 104: Perform image erosion on the missing component image to determine the pipeline information in the missing component image.

[0056] The component region has been eliminated from the missing component image, leaving only the pipeline lines and corresponding character information. Further detection is then performed to extract the pipeline information. Specifically, this embodiment uses image erosion to obtain the pipeline information. The image erosion process is similar to image convolution, both requiring a template matrix to control the result. In image erosion and dilation, this template matrix is ​​called the structuring element. Similar to image convolution, the structuring element can arbitrarily specify the center point of the image, and its size and specific content can be defined according to requirements. The purpose is to eliminate the boundary points of objects in the missing component image, causing the boundaries to shrink inwards. Objects smaller than the structuring element can be removed. This process removes invalid information caused by dirt or other interference from the drawings when obtaining pipeline information, ensuring accurate pipeline information identification.

[0057] Step 105: Determine the device identification result based on the component information and the pipeline information.

[0058] After identifying the component information and pipeline information, this paper further determines the connection relationship between pipelines and components based on the component location in the component information and the pipeline location in the relationship information, thereby determining how different components are connected through pipelines, and organizing the connection relationship between pipelines and components to obtain the equipment identification result of the entire P&ID drawing. The equipment identification result records the various devices (including components and pipelines) in the P&ID drawing and the connection relationship between different devices.

[0059] This embodiment provides a device identification method based on P&ID drawings. First, a sliding window is used to cut the P&ID drawing image to obtain multiple cut images. These multiple cut images are then input into a preset component identification model to extract component information and component regions. Next, the component regions are removed from the P&ID drawing image to obtain a component missing image. Based on the component missing image, image erosion is used to determine the pipeline information in the component missing image. Finally, the device identification result is determined based on the component information and pipeline information. This method can perform component identification while shielding pipeline interference, and pipeline identification while shielding component interference, further improving the automatic identification accuracy of P&ID drawings. It is also applicable to various P&ID drawings and has a wide range of applications.

[0060] Example 2

[0061] This application further provides a device identification method based on P&ID drawings in Embodiment 2, which can be implemented based on any of the foregoing embodiments of this application. The main difference between this method and Embodiment 1 is that some steps in Embodiment 1 are further explained or supplemented, such as the specific process of removing invalid information by image erosion. The method specifically includes:

[0062] like Figure 2 As shown, the device identification method based on P&ID drawings provided in this embodiment includes a step for obtaining the component identification model before inputting the multiple cut images into a preset component identification model to extract component information:

[0063] Step 201: Construct an initial classification model based on a deep residual convolutional neural network.

[0064] Step 202: Train the initial classification model using the preset component images to obtain the component classification model.

[0065] Step 203: Use a sliding window to cut the P&ID drawing image to obtain multiple cut images.

[0066] Step 204: Input the multiple cut images into a preset component recognition model to extract component information and component regions.

[0067] Step 205: Remove the component areas from the P&ID drawing image to obtain a component missing image.

[0068] Step 206: Perform image erosion on the missing component image to determine the pipeline information in the missing component image.

[0069] Step 207: Determine the equipment identification result based on the component information and the pipeline information.

[0070] Steps 201-202 are the process of acquiring the preset component recognition model, which is trained based on preset component images. The preset component images include components of different models and sizes, which is used to improve the component recognition model's ability to recognize different components. In this way, even if unfamiliar components appear in the P&ID drawing images, it has a certain recognition ability and will not fail to recognize them.

[0071] Optionally, in some embodiments, such as Figure 3 As shown, after step 204, step S208 is also included:

[0072] Step 208: Use a preset OCR recognition model to detect the component number and pipeline number in the component area, and determine the component location and the corresponding component type corresponding to the component number.

[0073] OCR (Optical Character Recognition) models are used to recognize characters. They can automatically detect areas containing text / numbers in an image and recognize the text / numbers within those areas. In addition to diagrams of components and pipelines, P&ID (Production and Identification) drawings also include process specifications and serial numbers. Simply recognizing components and pipelines is not enough to fully interpret the information in P&ID drawings; it is also necessary to combine this information with the annotations on the drawings.

[0074] Optionally, in some embodiments, such as Figure 4 Specifically, step 206 includes steps 2061-2063:

[0075] Step 2061: Perform image erosion on the missing component image to remove invalid information in the missing component image, and obtain the processed image.

[0076] Step 2062: Extract horizontal and vertical lines from the processed image.

[0077] Step 2063: Determine the horizontal line and / or the vertical line corresponding to the pipeline number to obtain pipeline information.

[0078] More specifically, in some embodiments, step 2061 specifically includes steps 20611-20613 (not shown in the figure):

[0079] Step 20611: Binarize the image of the missing component to obtain a binarized image.

[0080] Step 20612: Flip the pixel values ​​of the binarized image to obtain a flipped image.

[0081] Step 20613: Perform image erosion on the flipped image to obtain the processed image.

[0082] Binarization refers to setting the grayscale value of pixels in an image to 0 or 255, which means presenting the entire image with a clear visual effect of only black and white. The reason for flipping the binarized image in step 20612 is that image erosion actually shrinks the boundary of the bright area. Since a lot of invalid information in the binarized image is black area, in order to remove this invalid information through image erosion, it is necessary to flip the grayscale value of the pixels in the binarized image, that is, 0 becomes 255 and 255 becomes 0.

[0083] More specifically, in some embodiments, step 207, determining the device identification result based on the component information and the pipeline information, specifically includes: determining the component connection relationship based on the pipeline location and the component location, and determining the device identification result based on the component connection relationship, the component number, and the pipeline number.

[0084] Example 3

[0085] like Figure 5 As shown, this embodiment provides a device identification device 300 based on P&ID drawings, which includes:

[0086] Image cutting module 310 is used to cut P&ID drawing images using a sliding window to obtain multiple cut images;

[0087] The component recognition module 320 is used to input the multiple cut images into a preset component recognition model to extract component information and component regions;

[0088] The component removal module 330 is used to remove the component area from the P&ID drawing image to obtain a component missing image.

[0089] Pipeline identification module 340 is used to perform image erosion on the missing component image to determine pipeline information in the missing component image;

[0090] The result determination module 350 is used to determine the device identification result based on the component information and the pipeline information.

[0091] In some embodiments, before inputting the plurality of cut images into a preset component recognition model to extract component information and component regions, the method further includes:

[0092] Construct an initial classification model based on a deep residual convolutional neural network;

[0093] The initial classification model is trained using preset component images to obtain a component classification model.

[0094] In some embodiments, after inputting the plurality of cut images into a preset component recognition model to extract component information and component regions, the method further includes:

[0095] A preset OCR recognition model is used to detect the component number and pipeline number in the component area to determine the location of the component corresponding to the component number and the corresponding component type.

[0096] In some embodiments, removing the component area from the P&ID drawing image to obtain a component-missing image includes:

[0097] Based on the component area, the corresponding area on the P&ID drawing image is blackened to obtain a blackened image of the missing component.

[0098] In some embodiments, performing image erosion on the missing component image to determine pipeline information in the missing component image includes:

[0099] Image erosion is performed on the image of the missing component to remove invalid information in the image of the missing component, resulting in a processed image;

[0100] Extract horizontal and vertical lines from the processed image;

[0101] The horizontal line and / or the vertical line corresponding to the pipeline number are determined to obtain pipeline information.

[0102] In some embodiments, performing image erosion on the missing component image to remove invalid information from the missing component image, resulting in a processed image, includes:

[0103] Binarization is performed on the image of the missing component to obtain a binarized image;

[0104] The pixel values ​​of the binarized image are flipped to obtain a flipped image;

[0105] The processed image is obtained by performing image erosion on the flipped image.

[0106] In some embodiments, determining the device identification result based on the component information and the pipeline information includes:

[0107] The component connection relationship is determined based on the pipeline location and the component location, and the equipment identification result is determined based on the component connection relationship, the component number and the pipeline number.

[0108] This embodiment provides a device identification apparatus based on P&ID drawings. First, a sliding window is used to cut the P&ID drawing image to obtain multiple cut images. The multiple cut images are input into a preset component identification model to extract component information and component regions. Then, the component regions are removed from the P&ID drawing image to obtain a component missing image. Based on the component missing image, the pipeline information in the component missing image is determined by image erosion. Finally, the device identification result is determined based on the component information and pipeline information. This apparatus can perform component identification while shielding pipeline interference, and pipeline identification while shielding component interference, further improving the automatic identification accuracy of P&ID drawings. It is also applicable to a wide range of P&ID drawings.

[0109] Example 4

[0110] This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores a computer program. When the computer program is executed by a processor, it can implement the methods and steps provided in any of the foregoing embodiments of the present invention and has the same technical effects as the corresponding embodiments. This embodiment will not repeat the description here.

[0111] Example 5

[0112] Figure 6 This is a connection block diagram of an electronic device 500 provided in this embodiment, such as... Figure 6 As shown, the electronic device 500 may include: a processor 501, a memory 502, a multimedia component 503, an input / output (I / O) interface 504, and a communication component 505.

[0113] The processor 501 is used to execute all or part of the steps in the device identification method based on P&ID drawings, as described in Embodiment 1. The memory 502 is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0114] The processor 501 may be implemented as an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and is used to execute the device identification method based on P&ID drawings provided in any of the above embodiments.

[0115] The memory 502 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0116] Multimedia component 503 may include a screen, which may be a touchscreen, and an audio component for outputting and / or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory or transmitted via a communication component. The audio component also includes at least one speaker for outputting audio signals.

[0117] I / O interface 504 provides an interface between processor 501 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical buttons.

[0118] Communication component 505 is used for wired or wireless communication between the electronic device 500 and other devices. Wireless communication includes, for example, Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or one or more combinations thereof. Therefore, the corresponding communication component 505 may include: a Wi-Fi module, a Bluetooth module, or an NFC module.

[0119] In summary, this application provides a device identification method, apparatus, storage medium, and electronic device based on P&ID drawings. The method includes: using a sliding window to cut a P&ID drawing image to obtain multiple cut images; inputting the multiple cut images into a preset component identification model to extract component information and extract component regions, wherein the component information includes component type and component location; removing the component regions from the P&ID drawing image to obtain a component missing image; obtaining pipeline information based on the component missing image through image erosion; and determining the device identification result based on the component information and the pipeline information.

[0120] In the several embodiments provided in this application, it should be understood that the disclosed methods can also be implemented in other ways. The method embodiments described above are merely illustrative.

[0121] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0122] Although the embodiments disclosed in this application are as described above, the above content is merely for the purpose of facilitating understanding of this application and is not intended to limit this application. Any person skilled in the art to which this application pertains may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in this application; however, the scope of patent protection of this application shall still be determined by the scope defined in the appended claims.

Claims

1. A P&ID drawing based equipment identification method, characterized by, The method comprises the following steps: cutting the P&ID drawing image using a sliding window to obtain a plurality of cut images; inputting the plurality of cut images into a preset component recognition model to extract component information and component regions; removing the component regions in the P&ID drawing image to obtain a component missing image; performing image erosion on the component missing image to determine pipeline information in the component missing image; determining a device recognition result according to the component information and the pipeline information; The method further comprises the following steps before the step of inputting the plurality of cut images into a preset component recognition model to extract component information and component regions:

2. The method of claim 1, wherein, constructing an initial classification model based on a deep residual convolutional neural network; training the initial classification model using a preset component image to obtain a component classification model. The method further comprises the following steps after the step of inputting the plurality of cut images into a preset component recognition model to extract component information and component regions:

3. The method of claim 1, wherein, detecting component numbers and pipeline numbers using a preset OCR recognition model on the component regions to determine component positions corresponding to the component numbers and component types corresponding to the component numbers. The method further comprises the following steps of removing the component regions in the P&ID drawing image to obtain a component missing image:

4. The method of claim 1, wherein, blackening corresponding regions on the P&ID drawing image according to the component regions to obtain a blackened component missing image. The method further comprises the following steps of determining a device recognition result according to the component information and the pipeline information:

5. The method of claim 1, wherein, determining a component connection relationship according to pipeline positions and component positions, and determining a device recognition result according to the component connection relationship, component numbers and pipeline numbers. The method comprises the following steps:

6. A P&ID drawing based equipment identification apparatus, characterized by, an image cutting module configured to cut a P&ID drawing image using a sliding window to obtain a plurality of cut images; a component recognition module configured to input the plurality of cut images into a preset component recognition model to extract component information and component regions; a component removal module configured to remove component regions in the P&ID drawing image to obtain a component missing image; a pipeline recognition module configured to perform image erosion on the component missing image to determine pipeline information in the component missing image; a result determination module configured to determine a device recognition result according to the component information and the pipeline information. ​ The image erosion on the component missing image is used to determine pipeline information in the component missing image, including: binaryzation of the component missing image to obtain a binaryzation image; pixel value flipping of the binaryzation image to obtain a flipped image; image erosion based on the flipped image to remove invalid information to obtain a processed image; extraction of horizontal lines and vertical lines from the processed image as pipeline information.

7. A storage medium, characterized by The computer program stored in the storage medium can be executed by one or more processors, and can be used to implement the P&ID drawing-based equipment identification method in any one of claims 1-5.

8. An electronic device, comprising: The device comprises a memory and a processor, the memory stores a computer program, and the memory and the processor are in communication connection with each other, and when the computer program is executed by the processor, the P&ID drawing-based equipment identification method in any one of claims 1-5 is executed.

Images