Intelligent drawing auditing method and system for water conservancy project

Abstract

The invention relates to the technical field of intelligent drawing auditing, in particular to a hydraulic engineering intelligent drawing auditing method which comprises the following steps: S1, drawing file conversion: realizing PDF / DXF format generation through CAD secondary development, converting a PDF file into a JPG file by utilizing Python, and preprocessing an image; and S2, target structure detection: performing target detection by taking YOLOv5 as a basic algorithm, identifying information of a gate chamber bottom plate, a blanket, a stilling pool, an apron, an anti-scour groove and a drawing frame of the water gate drawing, accurately identifying each part of the water gate on the drawing and the position thereof, and generating a textbox. Through automatic drawing file conversion and image preprocessing, the steps of manual operation are greatly reduced, the drawing auditing process is accelerated, the dependence of professional drawing auditing personnel on the aspects of drawing information retrieval and preliminary auditing is reduced, and therefore the labor cost is reduced, and through automatic auditing of the drawing, the efficiency of drawing auditing is improved. And the consistency and the accuracy of the drawing information are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent drawing review, and in particular to an intelligent drawing review method and system for a water conservancy project. Background Art

[0002] With the widespread application of mapping software, there are a large number of engineering drawings in the water conservancy industry. How to quickly extract information and search for existing drawings has become an urgent problem to be solved. At present, the drawings in the database are mainly searched by manual comparison, which is cumbersome and inefficient. In recent years, AI intelligent drawing review has developed rapidly in the construction industry. For example, the AI ​​drawing review system jointly developed by the Shenzhen Housing and Construction Bureau and Wanyi Technology. When processing the same batch of drawings, the number and speed of problems found by the AI ​​drawing review system are several times that of manual work, and the cost is only 1 / 10 of that of manpower. More importantly, the AI ​​drawing review system has the algorithmic ability of continuous learning. As more and more data is accumulated, the quality and efficiency of its review will continue to improve.

[0003] Water conservancy projects are important infrastructure for national economic and social development, and their status and importance are self-evident. Water conservancy projects play a vital role in flood control, drought resistance, water supply, power generation, shipping, irrigation, etc. As an important part of water conservancy projects, the design quality of sluices is directly related to the safe operation and benefits of the projects. The review of sluice design drawings is an important link in the field of water conservancy projects. The traditional manual review method has problems such as low efficiency and easy errors. At the same time, the AI ​​review system in the water conservancy industry with strong specificity has not yet been applied. With the rapid development of artificial intelligence technology, it has become possible to apply artificial intelligence to the review of sluice design drawings.

[0004] Therefore, the present invention proposes an intelligent drawing review method and system for water conservancy projects to improve the construction efficiency of water gate projects in my country, reduce construction costs, standardize construction drawings, reduce errors, and ensure the safety of rivers, lakes and reservoirs. Summary of the invention

[0005] In view of the above-mentioned defects and problems, the present invention provides an intelligent drawing review method and system for water conservancy projects, which adopts computer vision, deep learning, large language model and other technologies to realize the automated review of sluice design drawings, improve the review efficiency and accuracy of sluice design drawings, improve the design quality, and ensure the safe operation of water conservancy projects.

[0006] The solution adopted by the present invention to solve the technical problem is: a method for intelligent drawing review of a water conservancy project, comprising the following steps:

[0007] S1. Drawing file conversion: generate PDF / DXF format through CAD secondary development, convert PDF file into JPG file using Python, and pre-process the image;

[0008] S2. Target structure detection: Through target detection based on YOLOv5 algorithm, the lock chamber bottom plate, blanket, energy dissipation pool, sea cover, anti-scouring groove and frame information of the sluice drawings are identified, and the various parts of the sluice on the drawings and their positions are accurately identified and a text box is generated, and the confidence of the text box and its four corner coordinates and center coordinates are output;

[0009] S3. Drawing OCR recognition: Preprocess the JPG image, then slice the saved JPG file, and use the PaddleOCR tool to perform OCR recognition on the segmented sub-areas. During the recognition process, PaddleOCR will analyze the text content in the image and return the location and corresponding content of each recognizable text. This step converts the text information in the image into editable and searchable text data.

[0010] S4, data matching and verification: Matching and verification is performed based on the center x-coordinate of the text box recognized by OCR and the center x-coordinate of the target detection. The error value between the two is calculated and compared with the preset error tolerance range. The judgment principle is whether the position annotation of the text box meets the centering specification. If the error is within the tolerance range, it is considered that the match is successful and the data annotation is accurate. If the error exceeds the tolerance range, an error is reported;

[0011] S5. Parse the CAD file, first detect the frame coordinate mapping, find the mapping relationship between the JPG format image and the DXF file coordinate system, then analyze the CAD data file format and extract the DXF file data.

[0012] S6. Construction of related libraries: Build an expert experience library and knowledge base to store drawing information, and use AI technology to interpret design specifications to achieve automatic matching of specifications;

[0013] S7. Intelligent review: The drawings are reviewed through an intelligent review system with a natural language model as its core.

[0014] Furthermore, the drawing file conversion in S1 is completed by the following steps:

[0015] (1) CAD secondary development and automatic drawing

[0016] 1) Development environment preparation

[0017] Select a suitable integrated development environment (IDE) and install the .NET development kit for AutoCAD;

[0018] 2) Set up the project

[0019] Create a new class library project or Windows Forms application project in Visual Studio and reference the AutoCAD .NET library to access the AutoCAD API in your code.

[0020] 3) Select the cropping range

[0021] Provide a user interface for users to select the cropping area with the mouse, and achieve this by rewriting AutoCAD commands or creating a new command to capture the boundaries of the cropping area selected by the user;

[0022] 4) Extract entities within the scope

[0023] Traverse all entities in the drawing and check whether they intersect with the clipping area or are completely within the clipping area. Use AutoCAD's DBDictionary, BlockTable, and LayerTable to access different elements in the drawing. For each entity, use the BoundingBox or GeometricExtents property to determine its position and compare it with the clipping area. Collect the entities that meet the conditions and prepare them for subsequent operations.

[0024] 5) Generate PDF and DXF files

[0025] Export to a PDF file by calling the PlotToFile method, and do this by calling AutoCAD's SaveAs method and specifying DXF as the file type;

[0026] (2) The PDF to JPG conversion process is implemented through Python programming. The necessary Python libraries are installed, including pdf2image, Pillow, and numpy. pdf2image is used for PDF to image conversion, and Pillow is used for image processing. The walk function of the os library is used to deeply traverse each directory in the specified folder and all its subfolders. The file path can be operated and processed through the os.path sublibrary. The Image class of the Pillow library is used to adjust the image size and resolution to ensure the pixel quality of the output image.

[0027] Furthermore, the target detection model based on the YOLOv5 algorithm is adopted in S2. At the beginning, the YOLO algorithm divides the input image into SxS grid cells, predicts B boundaries (bounding boxes) and the confidence of each bounding box for each cell, and uses a convolutional neural network (CNN) as a feature extraction network to extract features from the input image for subsequent target detection. For each grid cell, each bounding box predicts the category probability of the contained target and the coordinate information of the bounding box.

[0028] Further, in S5, the mapping relationship between the JPG format image and the DXF file coordinate system is found by the following method. The origin of the JPG format image coordinate system is the upper left corner of the image, with the X axis pointing to the right and the Y axis pointing downward. The origin of the CAD coordinate system (DWG / DXF) is at the edge of the drawing, with the X axis pointing to the right and the Y axis pointing upward. For any point (ximage, yimage) in the image coordinate system, the conversion to the point (xcad, ycad) in the CAD coordinate system can be performed by the following formula:

[0029] x cad =x image ×scale+x offset (1)

[0030] y cad =(height image -y image )×scale+y offset (2)

[0031] Among them: scale—scale, x offset and offset —The offset of CAD coordinates relative to image coordinates, height image —Image height.

[0032] Furthermore, the data matching method in S4 is as follows: first, the coordinates of the text boxes of target detection and OCR recognition are extracted respectively, the x-coordinate of the center point is calculated, and the upper and lower horizontal midpoints of the two coordinate boxes are found. If the horizontal coordinates of the two midpoints are equal under a certain error, that is, the x-axes where the center points of the two coordinate boxes coincide, then the text box is determined to match the target detection box, and at the same time, the vertical coordinate of the midpoint is limited, and the text box is only searched a certain distance below the target detection box to prevent matching errors; if the error is within the tolerance range, the match is displayed as successful, and the component name and its corresponding annotation data are output; if the error exceeds the tolerance range, an error is reported, and the possible cause of the violation may be pointed out.

[0033] Furthermore, the method of preprocessing the image in S1 is: first converting the converted JPG file into a grayscale (black and white) image, then using ImageEnhance.Contrast to enhance the contrast of the image, then applying ImageFilter.SHARPEN to sharpen the image, and then applying ImageFilter.SMOOTH_MORE to try to reduce the graininess of the image, then using spatial differentiation to process, using the Sobel operator to perform edge detection on the JPG drawing, highlighting the text box boundary in the image, and finally saving the processed image in black and white JPG format and entering the next step of processing.

[0034] Furthermore, in S6, the construction of the expert experience base includes Word Embedding, experience base extraction and expansion of the experience base; the knowledge base is constructed by the following two schemes: Scheme 1: vector database, converting the specifications into word vectors and storing them in the vector database; Scheme 2: graph database, expressing the specifications in the form of knowledge graph triples, so that strong rules are converted into a form that can be understood by the machine.

[0035] Furthermore, in S7, the intelligent review framework includes a question-answer-based review method, setting a prompt word, and setting an AI agent.

[0036] A system based on an intelligent drawing review method for water conservancy projects includes a drawing data extraction and processing module, a drawing data and specification parsing and storage module, and an intelligent review framework based on NLP, wherein:

[0037] The drawing data extraction and processing module further includes: a drawing file conversion module, which converts CAD into DXF and PDF files, and then converts PDF into jpg format files; an image preprocessing and segmentation module, which preprocesses the converted JPG drawing image to improve image quality, and uses image processing technology to segment the preprocessed image into multiple sub-areas to prepare for subsequent target detection and OCR recognition; a target structure detection model, which uses a target detection model based on the YOLOv5 algorithm, mainly identifies the lock chamber bottom plate, blanket, energy dissipation pool, sea cover, anti-scouring groove and drawing frame information of the sluice drawing, determines its position in the drawing and determines its length; a CAD file parsing module, which uses Python's ezdxf library and uses this as the core strategy for in-depth analysis; a drawing OCR recognition module, which applies OCR technology, including YOLOv5 algorithm and PaddleOCR analysis, to scan and identify the image and extract the relative position of the annotation information; a position matching and verification module, which matches the OCR recognition result with the target detection result to verify whether the position annotation of the text box is accurate.

[0038] The specification parsing and storage module includes data dictionary and specification interpretation. It builds a data dictionary to store and classify information in drawings to facilitate information identification, management and use. It uses AI technology to parse the current design specifications and convert the specification clauses into a format that can be understood and executed by computers.

[0039] The NLP-based intelligent review framework builds an intelligent review system based on natural language processing technology to realize automated review of drawings, including an expert experience library: establishing a database containing various content indicators and correlation relationships, simulating the expert's review experience to make rationality judgments; knowledge base: converting specifications into word vectors or knowledge graph triples, and storing them in the database to support intelligent review.

[0040] Furthermore, the CAD file parsing module includes: detection frame coordinate mapping, finding the mapping relationship between the JPG format image and the DXF file coordinate system; CAD data file format analysis, selecting AutoCAD result data file - DXF file as the identification data source; DXF file data extraction, selecting Python's ezdxf open source tool to implement DXF file extraction.

[0041] The beneficial effects of the present invention are as follows: by converting CAD drawings into DXF format and using Python's ezdxf library for deep parsing, the system can finely extract the geometric information and dimension annotations of the drawings, and utilize the target detection based on the YOLOv5 algorithm and the OCR recognition technology of PaddleOCR to improve the accuracy of structural component recognition and text information extraction in the drawings. In addition, through the data matching and verification steps, the system can intelligently verify whether the position annotations of the text boxes in the drawings are accurate, further improving the reliability of the drawing review system;

[0042] Through data dictionary and specification interpretation, the system optimizes the storage and management of drawing information, making information retrieval and use more efficient. By building a knowledge base and expert experience base, the system realizes automatic matching and rationality judgment of design specifications, and improves the intelligent level of drawing review. The intelligent drawing review system has the ability to continuously learn. With the accumulation of data and the increase of experience, the review quality and efficiency of the system will continue to improve;

[0043] Through automated drawing file conversion and image preprocessing, the number of manual operation steps has been greatly reduced, the review process has been accelerated, and the reliance on professional reviewers for drawing information retrieval and preliminary review has been reduced, thereby reducing labor costs. In addition, automated review of drawings has ensured the consistency and accuracy of drawing information. Accurate drawing review has ensured that the sluice design drawings meet safety standards and specifications, reducing engineering risks and improving design quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a graphic detection block diagram of the present invention;

[0045] Figure 2 It is a technical flow chart of the present invention;

[0046] Figure 3 It is a technical flow chart of the present invention;

[0047] Figure 4 It is a dictionary relationship diagram of each parameter in the data dictionary of the present invention;

[0048] Figure 5A block diagram of building a spatial database based on a data dictionary of the present invention;

[0049] Figure 6 It is a data extraction and retrieval block diagram of the present invention;

[0050] Figure 7 It is a block diagram of the machine review of the present invention;

[0051] Figure 8 It is the target structure detection diagram of the present invention;

[0052] Fig. 9 It is a flow chart of DXF graphic element information extraction of the present invention;

[0053] Fig.10 It is a sluice data relationship diagram of the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0055] See also Figure 1-10 In the field of water conservancy projects, for the sake of project confidentiality and technical protection, the application documents are mainly in PDF format. How to realize intelligent recognition and review of PDF drawings is the key to the technical solution. In order to improve the review efficiency and accuracy of sluice design drawings, the present invention develops a sluice standard review method and system based on artificial intelligence image recognition. Computer vision, deep learning, large language model and other technologies are used to realize the automated review of sluice design drawings, improve the design quality, and ensure the safe operation of water conservancy projects:

[0056] Embodiment 1, a method for intelligent review of water conservancy projects, comprising the following steps:

[0057] S1. Drawing file conversion and preprocessing: PDF / DXF format generation is achieved through CAD secondary development, and then the PDF file is converted into a JPG file using Python. Drawing file conversion is completed by the following steps:

[0058] (1) CAD secondary development and automatic drawing:

[0059] CAD secondary development automatic drawing refers to the process of automatically generating drawings by writing scripts or programs using the secondary development function of CAD software. This method can greatly improve design efficiency and reduce manual operations, especially in large-scale, repetitive or precise layout control situations. CAD secondary development can significantly improve design efficiency and flexibility, especially when it is necessary to crop drawings, extract entities in specific areas and generate files in different formats. CAD secondary development can achieve drawing cropping and entity extraction to generate PDF / DXF formats. This is done by the following steps:

[0060] 1) Development environment preparation

[0061] Choose a suitable integrated development environment (IDE), such as Visual Studio. Install the .NET development kit for AutoCAD in Visual Studio, which includes library files such as AcMgd.dll and AcDbMgd.dll;

[0062] 2) Set up the project

[0063] Create a new class library project or Windows Forms application project in Visual Studio and reference the AutoCAD .NET library to access the AutoCAD API in your code.

[0064] 3) Select the cropping range

[0065] Provide a user interface to let the user select the cropping area with the mouse, by rewriting the AutoCAD command or creating a new command to capture the boundaries of the cropping area selected by the user, usually a rectangular or polygonal area;

[0066] 4) Extract entities within the scope

[0067] Traverse all entities in the drawing and check whether they intersect with the clipping area or are completely within the clipping area. You can use AutoCAD's DBDictionary, BlockTable, LayerTable and other classes to access different elements in the drawing. For each entity, use the BoundingBox or GeometricExtents property to determine its position and compare it with the clipping area. Collect the entities that meet the conditions and prepare them for subsequent operations;

[0068] 5) Generate PDF and DXF files

[0069] Export to a PDF file by calling the PlotToFile method, and by calling AutoCAD's SaveAs method and specifying DXF as the file type.

[0070] (2) PDF to JPG conversion process, using Python, by introducing the PDF2image library to achieve the conversion of PDF files to JPG files. First, read the first page of the PDF to determine its original width (in pixels), then calculate the scaling factor, which is the ratio of the target width to the original width, and then use the calculated scaling factor to estimate the required DPI. Using this DPI value, the entire PDF file is converted again through the program at a higher DPI to obtain a higher resolution image. Through Python programming, the necessary Python libraries are installed, pdf2image, Pillow, and numpy. pdf2image is used for PDF to image conversion, and Pillow is used for image processing. The walk function of the os library is used to deeply traverse each directory in the specified folder and all its subfolders. Through the os.path sub-library, file paths can be manipulated and processed. The Image class of the Pillow library is used to adjust the size and resolution of the image to ensure that the length of the output image is 1280 pixels.

[0071] Then the image is preprocessed: first, the converted JPG file is converted into a grayscale (black and white) image, then, the contrast of the image is enhanced using ImageEnhance.Contrast, then ImageFilter.SHARPEN is applied to sharpen the image, and then ImageFilter.SMOOTH_MORE is applied to try to reduce the graininess of the image, and then the spatial differential method is used for processing. The spatial differential method is a method for enhancing edge information by calculating the gradient between image pixels. In the present invention, the Sobel operator is used to perform edge detection on the JPG drawing to highlight the text box boundary in the image. Finally, the processed image is saved in black and white JPG format and enters the next step of processing.

[0072] Then slice the image: Slice the saved JPG file. This work mainly uses the slice library in the PaddlePaddle architecture. The slice operation is similar to the slice operation in Python, but it is performed on multidimensional tensors. The user needs to specify the axis to be sliced, the starting index and the ending index of the slice on each axis. Slicing the image and then performing OCR recognition can effectively improve its success rate in recognizing small targets, which is of great help in recognizing sluice drawings.

[0073] Next, the processed images are operated on two paths at the same time:

[0074] S2. Target structure detection: Perform target detection on the complete JPG file. Through target detection based on YOLOv5 algorithm, the lock chamber floor, blanket, energy dissipation pool, sea cover, anti-scouring groove and frame information of the sluice drawing are identified. The various parts of the sluice on the drawing and their positions are accurately identified and a text box is generated. The text box confidence and its four corner coordinates and center coordinates are output. This experimental data set is extracted from the longitudinal section of the sluice structure diagram. The process of target structure detection is as follows: Figure 8 As shown in the figure, the image data includes 88 drawings as the data set, 80% of which are training sets. The lock chamber bottom plate, blanket, energy dissipation basin, sea cover, anti-scouring groove and frame information are manually annotated first. After 300 rounds of training, the remaining 20% ​​of the data set is used as the test set for detection.

[0075] For the construction of the sluice structure detection model, the environment configurations of the workbench of the present invention are as follows:

[0076] name Version name Version numpy 1.19.5 six 1.15.0 opencv-python 4.1.2.30 split 3.45.3 openssl 3.0.14 sympy 1.13.0 opt-einsum 3.3.0 tbb 2021.13.0 pandas 1.1.4 tensorboard 2.4.1 pillow 7.1.2 tensorboard-plugin-wit 1.8.1 pip 24.0 tensorflow 2.4.1 protobuf 3.19.0 tensorflow-estimator 2.4.0 pyasn1 0.6.0 tensorflow-gpu 2.4.1 pyasn1-modules 0.4.0 termcolor 1.1.0 pyparsing Python thop 0.1.1-2209072238 Python 3.8.19 torch 2.0.0+cu118 python-dateutil 2.9.0.post0 torchaudio 2.0.1 pytz 2024.1 torchvision 0.1.1+cu118 pyyaml 5.3.1 tq 4.41.0 requests 2.23.0 typing-extensions 3.7.4.3 requests-oauthlib 2.0.0 urllib3 1.25.11 rsa 4.9 vs2015_runtine 14.29.30133 sclpy 1.4.1 wheel 0.43.0 seaborn 0.11.0 wrapt 1.12.1 setuptools 69.5.1 zip 3.19.2

[0077] The present invention uses a target detection model based on the YOLOv5 algorithm. At the beginning, the YOLO algorithm divides the input image into SxS grid cells, each cell predicts B boundaries (bounding box) and the confidence of each bounding box, and uses a convolutional neural network (CNN) as a feature extraction network to extract features from the input image for subsequent target detection. For each grid cell, each bounding box predicts the category probability of the target contained and the coordinate information of the bounding box. Specifically, each bounding box predicts four coordinates (x, y, w, h), where (x, y) represents the center coordinate of the bounding box, and (w, h) represents the width and height of the bounding box. In addition, each bounding box also predicts a confidence score to indicate whether the bounding box contains an object and the accuracy of the prediction. In order to reduce the number of overlapping bounding boxes, YOLO uses non-maximum suppression to filter out the final detection results. This process determines which bounding boxes to keep based on the confidence score and predicted category probability, and eliminates overlapping bounding boxes. Finally, the YOLO algorithm outputs each detected object category, the location of the bounding box, and the confidence. This information can be used to identify the detected target in the image, and because of the overall end-to-end design, YOLO can achieve efficient target detection and real-time processing capabilities. The YOLO algorithm achieves efficient target detection by treating the target detection task as a regression problem, directly mapping the input image to the prediction of the bounding box and category probability, avoiding the complex multi-stage process in traditional methods.

[0078] S3. OCR recognition: Use the PaddleOCR tool to perform OCR recognition on the segmented sub-areas. PaddleOCR is an OCR tool developed based on the PaddlePaddle framework, which has efficient and accurate recognition capabilities. During the recognition process, PaddleOCR analyzes the text content in the image and returns the location of each recognizable text (including text box coordinates) and the corresponding content. This step converts the text information in the image into editable and searchable text data.

[0079] S4, position matching and verification: Match and verify the center x-coordinate of the text box recognized by OCR with the center x-coordinate of the target detection. By calculating the error value between the two and comparing it with the preset error tolerance range, the judgment principle is whether the position annotation of the text box meets the centering specification. If the error is within the tolerance range, it is considered that the match is successful and the data annotation is accurate. If the error exceeds the tolerance range, an error is reported;

[0080] The matching method is as follows: first, extract the coordinates of the text boxes of target detection and OCR recognition respectively, calculate the x-coordinate of their center points, and through understanding the relevant specifications, according to the drawing specifications, the length of the component needs to be marked below the center of the component, find the upper and lower horizontal midpoints of the two coordinate boxes, and if the horizontal coordinates of the two midpoints are equal under a certain error, that is, when the x-axes of the center points of the two coordinate boxes coincide, then the text box is determined to match the target detection box, and at the same time, the vertical coordinate of the midpoint is limited, and the text box is only searched a certain distance below the target detection box to prevent matching errors; if the error is within the tolerance range, it will be displayed as a successful match, and the component name and its corresponding annotation data will be output to facilitate subsequent application in large models; if the error exceeds the tolerance range, an error will be reported, and possible violations may be pointed out, such as OCR recognition errors, inaccurate YOLOv5 detection, or original drawing annotation problems.

[0081] At the same time, based on OCR technology, some aesthetics detection links can be realized. On the one hand, since the size of the text can be judged by the height of the recognized text box, the variance of the height of all text boxes can be calculated to determine whether the size of the text is consistent. If the variance exceeds a certain range, it will remind that there are certain problems with the text aesthetics setting. On the other hand, there will be repeated annotations and adhesion problems of text information in the drawings. To deal with this problem, first extract all digital text box information, and then find the text box with the same y coordinate value of the center point of the text box by calculation under a certain error, that is, find the text box of each row, and then traverse each text box and compare it with the area of ​​other text boxes. If the text box is more than twice the area of ​​other text boxes, then there may be adhesion problems.

[0082] S5, parsing the CAD file, first detecting the frame coordinate mapping, finding the mapping relationship between the JPG format image and the DXF file coordinate system, and then analyzing the CAD data file format and extracting the DXF file data;

[0083] (1) To realize the association between target detection and CAD analysis, we must first find the mapping relationship between the JPG format image and the DXF file coordinate system. The origin of the JPG format image coordinate system is the upper left corner of the image, with the X axis pointing to the right and the Y axis pointing downward; the origin of the CAD coordinate system (DWG / DXF) can be set arbitrarily, usually at the edge of the drawing, with the X axis pointing to the right and the Y axis pointing upward. For any point in the image coordinate system (ximage, yimage), the conversion to the point in the CAD coordinate system (xcad, ycad) can be performed using the following formula:

[0084] x cad =x image ×scale+x offset (1)

[0085] y cad =(height image -y image )×scale+y offset (2)

[0086] Among them: scale—scale, x offset and offset —The offset of CAD coordinates relative to image coordinates, height image —Image height.

[0087] (2) CAD data file format analysis

[0088] DWG (DraWinG) and DXF are both commonly used file formats for CAD models, but the reading and writing operations of DWG data structure are not easy to implement without the AutoCAD software environment, while DXF contains all the same graphic elements of DWG, and is a universal open source graphic data interactive vector file, which is easy to implement secondary application development based on CAD models. Therefore, the present invention selects AutoCAD result data file-DXF file as the identification data source.

[0089] (3) DXF file data extraction

[0090] Choose to use Python's ezdxf open source tool to extract DXF files. Using Python's ezdxf library to extract data from CAD drawings is to combine the target detection part to perform target detection on the JPG image converted from the PDF drawing to identify the structure in the sluice drawing. It will generate a detection frame and give the coordinates of the detection frame. According to the coordinate correspondence between the DXF drawing and the JPG image, the detection frame in the JPG is moved to the DXF to achieve structural correspondence, and then the detection frame is moved up, down, left, and right to traverse its annotation information and extract the size.

[0091] S6. Construction of related libraries: Build an expert experience library and knowledge base to store drawing information, and use AI technology to interpret design specifications to achieve automatic matching of specifications, including:

[0092] (1) Expert experience database: As the scale and complexity of projects increase, a single review method is difficult to meet the requirements of efficient and accurate review. Through the expert experience database, a database containing various content indicators and correlation relationships can be established to improve the efficiency and accuracy of review. The core of the expert experience database is to establish a privacy-protected database, equipped with a MySQL database, and protect data through encryption algorithms such as RSA. Using the cvp architecture, the vector database Milvus is introduced to provide long-term memory for LLM, making the experience database scalable, real-time and cost-effective. Data can be added to the vector database in real time to complete data updates;

[0093] 1) Word Embedding converts text into mathematical representation and assigns a fixed-length vector representation to each word. Through a simple cosine function, the correlation between two data can be calculated. It is simple and efficient:

[0094]

[0095] The vector data converted in the early stage is stored in Milvus. In the later large model answer, several vectors that are most similar to the target vector can be quickly retrieved, and the search of billions of vectors can be achieved in milliseconds. In the scenario of the present invention, Milvus and MySQL can be used together. Milvus can store sluice data, and MySQL stores the text corresponding to the data. Milvus can use MySQL as metadata storage to save description information about the vector;

[0096] 2) Experience database extraction: The data obtained from the early target detection is used to ask questions to the large model through the data corresponding to the keywords (design flow and upstream design water depth), so that it can find similar data of sluices in the experience database, find out the remaining sluice data under this sluice state, use Python code to get the maximum and minimum values, and then reasonably expand the data through y=kx+b, and finally get a certain data range, which is compared with the data in the report to draw a conclusion on whether it is reasonable;

[0097] 3) Expansion of the experience database. After judging the rationality, the qualified drawings are transformed into a vector database through Embedding, so as to achieve the purpose of expanding the experience database. On this basis, the experience database is continuously filled, and the judgment of rationality is more accurate.

[0098] (2) Knowledge base: Solution 1: vector database, convert the specification into word vectors and store them in the vector database. Process: read the document load (currently supported types: pdf, txt, csv, excel); split the document transform; convert into vector embedding; vector library match vectorstores;

[0099] Solution 2: Graph database, which expresses specifications in the form of knowledge graph triples, converts strong rules into a form that machines can understand, simplifying the specification expression.

[0100] S7. Intelligent review: The drawings are reviewed through an intelligent review system with natural language models as the core. Natural language models (NLP), as an advanced natural language processing technology, have made significant progress in the field of artificial intelligence in recent years, including question-answering review methods, setting prompts, and setting AI agents:

[0101] (1) Based on the question-and-answer review method, in the consistency review of drawings and reports, it is necessary to extract the design indicators of the sluice gate from the report according to certain rules and convert them into structured data and store them in the database for comparison with drawing data. In order to achieve the above work requirements, the following solutions are proposed:

[0102] Solution 1: Use the big model to intelligently extract the required content from the document. Process: Segment long text → Convert word vectors into the database → Set prompt words for question sentences → Retrieve relevant information from the vector database → Set the AI ​​agent to ask questions automatically;

[0103] Solution 2: Integrate large models with knowledge graphs.

[0104] (2) Set the prompt word

[0105] 1) AI understanding ability. Different design reports have different descriptions of size. For the concept of "size", the large model understands it as "length, width, height" and cannot understand deeper and more professional expressions. In order to achieve the work requirements, the following solutions are proposed:

[0106] Solution 1: It is necessary to collect relevant design reports, summarize relevant expression rules and design prompt word templates;

[0107] Solution 2: Professional interpretation of “size” as context to fine-tune the big model;

[0108] 2) Review prompt words

[0109] Solution 1: Directly set the prompt word

[0110] prompt=f"Based on the following known content, answer the question very concisely to see if it meets the requirements of the known content, "\

[0111] f"If you can't get the answer from it, please say 'This question cannot be answered based on the known information'. It is not allowed to add fabricated elements to the answer, "\

[0112] f"Known content: {known_info},"\

[0113] f"Question: {question}."

[0114] Solution 2: Set a prompt word template

[0115] prompt_template="""Based on the following known content, concisely and professionally judge whether the question meets the requirements of the known content. If you cannot get an answer from it, please say "I cannot answer this question based on the known information". Fabricated elements are not allowed in the answer. Please use Chinese for the answer.

[0116] Known Content:

[0117] {context}

[0118] question:

[0119] {question}"""

[0120] Solution 3: Example of setting prompt words

[0121] (3) Setting up an AI agent, the agent involves a large prediction model to make an action decision, execute the action, view an observation result, and repeat the process until completion. LangChain provides a standard agent interface, a series of agents to choose from, and an end-to-end agent example.

[0122] In S7, the key points of drawing review include: 1) Based on the consistency review of the report description, the following key points are determined: whether the structural attributes in the drawing are consistent with those described in the design report; whether the design description in the drawing conflicts with the content of the design report; whether the elevation, water level and other annotation information in the drawing are the same as those described in the design report;

[0123] Key points for table consistency review

[0124] Subject of review Structural dimensions Structure Elevation Structural Materials Sluice gate size drawing Report

[0125] 2) Based on the technical review of standards and regulations, standard drawing reading mainly includes reading drawings using drawing standards and reading drawings using design standards;

[0126] 3) Rationality review based on expert experience. For sluice gate review, experienced experts can make a rough judgment on the remaining data based on the design flow and upstream design water depth. Data beyond the judgment range can quickly determine the design irrationality. For machine review, the relationship between the design flow and upstream design water depth and other data can be established through the database, and the data range can be defined using the formula y=kx+b, which can be compared with the data on the drawing. Fig.10 .

[0127] Embodiment 2: This embodiment provides a system based on the intelligent drawing review method for water conservancy projects, including: 1. Drawing data extraction and processing module; 2. Drawing data and specification parsing storage module; 3. NLP-based intelligent review framework.

[0128] 1. Drawing data extraction and processing module also includes:

[0129] Drawing file conversion module, convert CAD to dxf and PDF files, and then convert PDF to jpg format files;

[0130] Image preprocessing and segmentation module preprocesses the converted JPG drawing image to improve image quality, and uses image processing technology to segment the preprocessed image into multiple sub-areas to prepare for subsequent target detection and OCR recognition

[0131] The target structure detection model uses a target detection model based on the YOLOv5 algorithm. It mainly identifies the lock chamber bottom plate, blanket, energy dissipation pool, sea cover, anti-scouring groove and frame information of the sluice drawings, determines their positions in the drawings and determines their lengths;

[0132] The CAD file parsing module uses Python's ezdxf library as the core strategy for in-depth analysis. The CAD file parsing module includes: detection frame coordinate mapping, finding the mapping relationship between the JPG format image and the DXF file coordinate system; CAD data file format analysis, selecting AutoCAD result data file - DXF file as the identification data source; DXF file data extraction, selecting Python's ezdxf open source tool to realize DXF file extraction;

[0133] Drawing OCR recognition module: Apply OCR technology, including YOLOv5 algorithm and PaddleOCR analysis. YOLOv5: As an advanced real-time target detection algorithm, YOLOv5 performs well in detection speed and accuracy, and is suitable for processing text box detection tasks in sluice drawings; PaddleOCR: An OCR tool library developed based on the PaddlePaddle deep learning framework, which supports text recognition in multiple languages ​​and has strong adaptability to complex backgrounds. It is suitable for extracting text information and annotation data from sluice drawings. The YOLOv5 algorithm is used to identify component names, generate text boxes, output text box confidence, and its four corner coordinates and center coordinates; and the sliced ​​JPG file is subjected to OCR recognition. During the recognition process, PaddleOCR will analyze the text content in the image, return the location and corresponding content of each recognizable text, convert the text information in the image into editable and searchable text data, scan and recognize the image, extract the relative position of the annotation information, and match it with the coordinates detected by the graphic;

[0134] The position matching and verification module matches the OCR recognition results with the target detection results to verify whether the position marking of the text box is accurate.

[0135] 2. The drawing data and specification interpretation storage module includes data dictionary and specification interpretation. A data dictionary is constructed to store and classify information in drawings. The data dictionary is used to classify and store drawing information to facilitate information identification, management and use. At the same time, AI technology is used to parse the current design specifications and convert the specification clauses into a format that can be understood and executed by computers.

[0136] The data dictionary (DD) is a collection of descriptions of various types of data in the database system. Most database management systems today use the data dictionary as a criterion for defining various types of data objects in the database and their relationships. The data dictionary in the spatial database is essentially a data table in the relational database, which records the various parameter information of the spatial data model structure. When describing a data model, which sub-patterns should be included, the structure of the sub-patterns, the relationship between the sub-patterns, and the existing constraint rules, etc. The description of the sub-pattern generally includes the pattern name, attribute name, type and length of the attribute domain, keyword information, etc. For spatial database structure models in different fields, although the data stored in each parameter dictionary table is different, since the structure of each dictionary itself is the same, the application developed based on the data dictionary is universal and can be applied to different spatial database model management;

[0137] For standard interpretation, AI can identify the meaning of words, label the text in natural language, and then translate the text content into the corresponding syntax tree structure, which can be converted into code in any computer language. In the process of organizing the specifications into strong clauses, they must be organized in a certain order, and then organized in points, and then organized according to the content that can be achieved in the drawing review.

[0138] 3. NLP-based intelligent review framework Based on natural language processing technology, an intelligent review system is constructed, and natural language processing technology is used to conduct consistency review of drawings and design reports to ensure that the drawings are consistent with the design intent, including expert experience library: establish a database containing various content indicators and correlation relationships, simulate the expert's review experience to make rationality judgments; knowledge base: convert the specifications into word vectors or knowledge graph triples, and store them in the database to support intelligent review.

[0139] The above description is only a preferred embodiment of the present invention and does not limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for intelligent drawing review of water conservancy projects, characterized in that: The following steps are involved: S1. Drawing file conversion: generate PDF / DXF format through CAD secondary development, convert PDF file into JPG file using Python, and pre-process the image; S2. Target structure detection: Through target detection based on YOLOv5 algorithm, the lock chamber bottom plate, blanket, energy dissipation pool, sea cover, anti-scouring groove and frame information of the sluice drawings are identified, and the various parts of the sluice on the drawings and their positions are accurately identified and a text box is generated, and the confidence of the text box and its four corner coordinates and center coordinates are output; S3. Drawing OCR recognition: Preprocess the JPG image, then slice the saved JPG file, and use the PaddleOCR tool to perform OCR recognition on the segmented sub-areas. During the recognition process, PaddleOCR will analyze the text content in the image and return the location and corresponding content of each recognizable text. This step converts the text information in the image into editable and searchable text data. S4, data matching and verification: Matching and verification is performed based on the center x-coordinate of the text box recognized by OCR and the center x-coordinate of the target detection. The error value between the two is calculated and compared with the preset error tolerance range. The judgment principle is whether the position annotation of the text box meets the centering specification. If the error is within the tolerance range, it is considered that the match is successful and the data annotation is accurate. If the error exceeds the tolerance range, an error is reported; S5. Parse the CAD file, first detect the frame coordinate mapping, find the mapping relationship between the JPG format image and the DXF file coordinate system, then analyze the CAD data file format and extract the DXF file data. S6. Construction of related libraries: Build an expert experience library and knowledge base to store drawing information, and use AI technology to interpret design specifications to achieve automatic matching of specifications; S7. Intelligent review: The drawings are reviewed through an intelligent review system with a natural language model as its core.

2. According to claim 1, the intelligent drawing review method for water conservancy projects is characterized by: The drawing file conversion in S1 is completed by the following steps: (1) CAD secondary development and automatic drawing 1) Development environment preparation Select a suitable integrated development environment (IDE) and install the .NET development kit for AutoCAD; 2) Set up the project Create a new class library project or Windows Forms application project in Visual Studio and reference the AutoCAD .NET library to access the AutoCAD API in your code. 3) Select the cropping range Provide a user interface for users to select the cropping area with the mouse, and achieve this by rewriting AutoCAD commands or creating a new command to capture the boundaries of the cropping area selected by the user; 4) Extract entities within the scope Traverse all entities in the drawing and check whether they intersect with the clipping area or are completely within the clipping area. Use AutoCAD's DBDictionary, BlockTable, and LayerTable to access different elements in the drawing. For each entity, use the BoundingBox or GeometricExtents property to determine its position and compare it with the clipping area. Collect the entities that meet the conditions and prepare them for subsequent operations. 5) Generate PDF and DXF files Export to a PDF file by calling the PlotToFile method, and do this by calling AutoCAD's SaveAs method and specifying DXF as the file type; (2) The PDF to JPG conversion process is implemented through Python programming. The necessary Python libraries are installed, including pdf2image, Pillow, and numpy. pdf2image is used for PDF to image conversion, and Pillow is used for image processing. The walk function of the os library is used to deeply traverse each directory in the specified folder and all its subfolders. The file path can be operated and processed through the os.path sublibrary. The Image class of the Pillow library is used to adjust the size and resolution of the image to ensure the pixel quality of the output image.

3. The intelligent drawing review method for water conservancy projects according to claim 1 is characterized in that: The S2 adopts a target detection model based on the YOLOv5 algorithm. At the beginning, the YOLO algorithm divides the input image into SxS grid cells, predicts B boundaries (bounding boxes) and the confidence of each bounding box for each cell, and uses a convolutional neural network (CNN) as a feature extraction network to extract features from the input image for subsequent target detection. For each grid cell, each bounding box predicts the category probability of the contained target and the coordinate information of the bounding box.

4. The intelligent drawing review method for water conservancy projects according to claim 1 is characterized in that: In S5, the mapping relationship between the JPG format image and the DXF file coordinate system is found by the following method. The origin of the JPG format image coordinate system is the upper left corner of the image, with the X axis pointing right and the Y axis pointing downward. The origin of the CAD coordinate system (DWG / DXF) is at the edge of the drawing, with the X axis pointing right and the Y axis pointing upward. For any point (ximage, yimage) in the image coordinate system, the conversion to the point (xcad, ycad) in the CAD coordinate system can be performed by the following formula: x cad =x image ×scale+x offset (1) and cad (height) image -and image )×scale+y offset (2) Among them: scale—scale, x o ff se t and o ff se t —The offset of CAD coordinates relative to image coordinates, height image —Image height.

5. The intelligent drawing review method for water conservancy projects according to claim 1 is characterized in that: The data matching method in S4 is as follows: first, the coordinates of the text boxes of target detection and OCR recognition are extracted respectively, the x-coordinate of the center point is calculated, and the upper and lower horizontal midpoints of the two coordinate boxes are found. If the horizontal coordinates of the two midpoints are equal under a certain error, that is, the x-axes where the center points of the two coordinate boxes coincide, then it is determined that the text box matches the target detection box, and at the same time, the vertical coordinate of the midpoint is limited, and the text box is only searched for a certain distance below the target detection box to prevent matching errors; if the error is within the tolerance range, it is displayed that the match is successful, and the component name and its corresponding annotation data are output; If the error is outside the tolerance range, an error is reported and the possible cause of the violation may be indicated.

6. The intelligent drawing review method for water conservancy projects according to claim 1 is characterized in that: The method of preprocessing the image in S1 is: first convert the converted JPG file into a grayscale (black and white) image, then use ImageEnhance.Contrast to enhance the contrast of the image, then apply ImageFilter.SHARPEN to sharpen the image, and then apply ImageFilter.SMOOTH_MORE to try to reduce the graininess of the image, then use spatial differentiation to process, use the Sobel operator to perform edge detection on the JPG drawing, highlight the text box boundary in the image, and finally save the processed image in black and white JPG format and enter the next step of processing.

7. The intelligent drawing review method for water conservancy projects according to claim 1 is characterized in that: In S6, the construction of the expert experience base includes Word Embedding, experience base extraction and experience base expansion; the knowledge base is constructed by the following two schemes: Scheme 1: vector database, converting the specifications into word vectors and storing them in the vector database; Scheme 2: graph database, expressing the specifications in the form of knowledge graph triples, so that strong rules are converted into a form that can be understood by the machine.

8. The intelligent drawing review method for water conservancy projects according to claim 1 is characterized in that: In S7, the intelligent review framework includes a question-answer-based review method, setting a prompt word, and setting an AI agent.

9. A system based on the intelligent drawing review method for water conservancy projects according to claim 1, characterized in that: It includes drawing data extraction and processing module, drawing data and specification parsing storage module and NLP-based intelligent review framework, among which: The drawing data extraction and processing module further includes: a drawing file conversion module, which converts CAD into DXF and PDF files, and then converts PDF into jpg format files; an image preprocessing and segmentation module, which preprocesses the converted JPG drawing image to improve image quality, and uses image processing technology to segment the preprocessed image into multiple sub-areas to prepare for subsequent target detection and OCR recognition; a target structure detection model, which uses a target detection model based on the YOLOv5 algorithm, mainly identifies the lock chamber bottom plate, blanket, energy dissipation pool, sea cover, anti-scouring groove and drawing frame information of the sluice drawing, determines its position in the drawing and determines its length; a CAD file parsing module, which uses Python's ezdxf library and uses this as the core strategy for in-depth analysis; a drawing OCR recognition module, which applies OCR technology, including YOLOv5 algorithm and PaddleOCR analysis, to scan and identify the image and extract the relative position of the annotation information; a position matching and verification module, which matches the OCR recognition result with the target detection result to verify whether the position annotation of the text box is accurate. The specification parsing and storage module includes data dictionary and specification interpretation. It builds a data dictionary to store and classify information in drawings to facilitate information identification, management and use. It uses AI technology to parse the current design specifications and convert the specification clauses into a format that can be understood and executed by computers. The NLP-based intelligent review framework builds an intelligent review system based on natural language processing technology to realize automated review of drawings, including an expert experience library: establishing a database containing various content indicators and correlation relationships, simulating the expert's review experience to make rationality judgments; knowledge base: converting specifications into word vectors or knowledge graph triples, and storing them in the database to support intelligent review.

10. The intelligent drawing review system for water conservancy projects according to claim 9 is characterized in that: The CAD file parsing module includes: detection frame coordinate mapping, finding the mapping relationship between the JPG format image and the DXF file coordinate system; CAD data file format analysis, selecting AutoCAD result data file - DXF file as the identification data source; DXF file data extraction, selecting Python's ezdxf open source tool to implement DXF file extraction.

Images