Cast-in-place box girder image cross-key parameter identification method based on target detection and key information extraction

Abstract

The invention relates to a cast-in-place box girder image cross-key parameter identification method based on target detection and key information extraction. The method comprises the steps that a bridge design drawing is acquired and preprocessed; a bridge structure detection model is obtained and trained; according to the image after bridge multi-view structure target detection, bridge structure images under different views are intercepted; selecting different key parameters needing to be identified; an editable Excel document is obtained; and summarizing into a new Excel document of the key parameter information of the bridge structure in the same span. According to the method, semantic information is given to key parameters in the drawing, parameter names correspond to values, modularization of plane graph parameter identification is achieved, and the key parameters of the PDF drawing can be efficiently and automatically extracted; and finally, a cross-recognition and integration method is adopted to successfully solve the multi-view parameter recognition problem, and beneficial exploration is carried out for an artificial intelligence algorithm to understand engineering design drawings. And key parameters of multiple views in the cast-in-place box girder PDF drawing can be rapidly and accurately extracted.

Description

Technical Field

[0001] The invention relates to the technical field of PDF image recognition, and in particular to a method for cross-key parameter recognition of a cast-in-place box girder image based on target detection and key information extraction. Background Art

[0002] Bridge design drawings carry the core information of bridge engineering, including size, location, material specifications and key construction requirements. This information is crucial to ensure the structural safety, construction feasibility and overall quality of the bridge. Effective identification and archiving of design drawings can not only provide important data for 3D reconstruction, but also help construction personnel better understand the bridge structure, improve installation and assembly efficiency, and reduce errors and delays in the construction process. In practical applications, there are many difficulties in parameter recognition of drawings in the bridge field. First, the design habits of different designers in bridge design are different, which leads to differences in annotations and symbols on the drawings, increasing the complexity of recognition; secondly, the parameters on the PDF drawings of bridge structures are usually just numbers and lack semantic information, which makes it difficult to understand the specific meaning of the parameters; in addition, the design of PDF design drawings of cast-in-place box girders is more complex, with multiple information levels, symbol occlusion and line segment overlap, which is not conducive to parameter recognition; compared with drawing recognition in the construction field, there is no public data set available for bridge drawing recognition, more parameters need to be identified, and the recognition accuracy requirements are also higher; finally, the parameters that need to be identified in cast-in-place box girders are distributed in different planes, involving comprehensive recognition and parameter integration of multiple views.

[0003] With the rapid development of computer science, more and more researchers have devoted themselves to the research of drawing recognition technology, and related processing technologies have gradually matured. Although mainstream CAD software now has professional drawing information management functions, it is still difficult to effectively extract key information from non-vectorized drawings, such as scanned documents. Current methods are only suitable for information extraction from vectorized drawings, or the image feature matching method faces the problem of poor robustness.

[0004] How to design a structural detection and key parameter identification method for the general structural drawing PDF design drawings of cast-in-place box girders has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to solve the problem of difficulty in multi-view parameter recognition in general structural drawings of cast-in-place box girders in PDF design drawings, the object of the present invention is to provide a cast-in-place box girder image cross-key parameter recognition method based on target detection and key information extraction, which can quickly and accurately extract key parameters of multiple views in PDF drawings of cast-in-place box girders.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method for identifying cross-key parameters of cast-in-place box girder images based on target detection and key information extraction, the method comprising the following steps in order:

[0007] (1) Obtaining bridge design drawings and preprocessing them to obtain preprocessed bridge design drawings;

[0008] (2) Improve the YOLOv10 model to obtain a bridge structure detection model;

[0009] (3) Inputting the preprocessed bridge design drawings into the bridge structure detection model for training to obtain a trained bridge structure detection model;

[0010] (4) Obtaining the design drawings of the bridge to be inspected and preprocessing them, inputting the preprocessed design drawings of the bridge to be inspected into the trained bridge structure detection model, and the trained bridge structure detection model outputs an image of the bridge multi-view structure target after detection, and marking the relative position of the bridge structure target on the image of the bridge multi-view structure target after detection;

[0011] (5) According to the bridge multi-view structure target detection image, intercept the bridge structure image under different views, wherein the different views include a cross-sectional view, an elevation view, a top plan view, and a bottom plan view;

[0012] (6) Select different key parameters that need to be identified based on the bridge structure images under different views;

[0013] (7) Input the bridge structure images under different views and the different key parameters to be identified into the paddleocr model for feature extraction and feature fusion to obtain multimodal features. Input the multimodal features into the vi_layoutxlm_ser model for prediction to obtain an editable Excel document.

[0014] (8) Read the editable Excel document and, based on the logical relationship between the parameters of multiple views of the bridge structure within the same span, merge the editable Excel documents of the key parameters of each view into a new Excel document of the key parameter information of the bridge structure within the same span.

[0015] The step (1) specifically includes the following steps in order:

[0016] (1a) Use the pdf2image open source library to convert the archived PDF bridge design drawings into JPG images. The resolution of each JPG image is 3573×2526.

[0017] (1b) Perform data enhancement processing, rotate all JPG format images and add Gaussian noise;

[0018] (1c) Labeling: Use the Labelimg tool to label the bridge structure in multiple views of the image after data enhancement, which are divided into cross-section, elevation, top plan and bottom plan. The labeling format is YOLO format.

[0019] The step (2) specifically refers to: improving the YOLOv10 model, replacing all C2f modules in the YOLOv10 model with DualConv modules; and replacing the detection head in the YOLOv10 model with the detection head of the RT-DETR model.

[0020] The step (5) specifically includes the following steps in order:

[0021] (5a) Obtain the target frame information in the image after the bridge multi-view structure target detection, that is, the relative position of the bridge structure target in the image. The target frame information includes:<class_label><x_center><y_center> <width>and <height>;<class_label> Refers to the target category label, which uses an integer to represent the category number to which the detected target belongs;<x_center> and<y_center> Respectively refer to the x-coordinate and y-coordinate of the center point of the target frame; <width>and <height>Refers to the width and height of the target box respectively;

[0022] (5b) Convert the target frame information into coordinate information according to the following conversion relationship:

[0023] left = int((x_center - width / 2) * image.shape[1])

[0024] top = int((y_center - height / 2) * image.shape[0])

[0025] right = int((x_center + width / 2) * image.shape[1])

[0026] bottom = int((y_center + height / 2) * image.shape[0])

[0027] In the formula, image.shape[1] represents the width of the image, image.shape[0] represents the height of the image, left represents the horizontal coordinate position of the left boundary of the target box in the image; right represents the horizontal coordinate position of the right boundary of the target box in the image; top represents the vertical coordinate position of the upper boundary of the target box in the image; bottom represents the vertical coordinate position of the lower boundary of the target box in the image;

[0028] For each target, according to the target's coordinate information, a python script is used to capture the image of the bridge multi-view structure target after detection, and the bridge structure images under different views are captured, namely, cross-section, elevation, top plan and bottom plan. The captured bridge structure images under different views are named "original image name + view name" and saved in the folders of each view accordingly.

[0029] The step (6) specifically refers to:

[0030] The key parameters that need to be identified in determining the cross-section diagram include: serial number, box beam upper flange width, box beam lower flange width, flange left cantilever width, flange right cantilever width, beam height, cantilever end thickness and cantilever root thickness, a total of 8 parameters;

[0031] The key parameter names that need to be identified in the elevation drawing include: section symbol, pier number, span and beam height, a total of 4 parameters;

[0032] The key parameters that need to be identified in determining the top plan include: pier number, span and beam height, a total of 3 parameters;

[0033] The key parameter names that need to be identified to determine the bottom plan include: pier number, span and beam height, a total of 3 parameters.

[0034] The step (7) specifically includes the following steps in order:

[0035] (7a) annotating the bridge structure images under different views, marking the key parameters on the bridge structure images under different views, and obtaining the annotated bridge structure images under different views;

[0036] (7b) Input the annotated bridge structure images under different views into the paddleocr model for feature extraction, and extract the text information, text coordinate information and layout feature information; then, encode and convert the extracted text information, text coordinate information and layout feature information, and fuse them into multimodal features; then, input these multimodal features into the vi_layoutxlm_ser model for forward reasoning calculation, predict the category, position and specific text content of each semantic entity in the image, and output the recognition result image and text file;

[0037] (7c) Processing the text file: The text file includes the image path and each text recognition result entry, which includes the transcription, bbox, points, pred_id and pred fields and their corresponding values; transcription represents the text content recognized in the image; bbox represents the bounding box coordinate information of the area where the recognized text is located, which is given in the form of a list, and represents the upper left corner horizontal coordinate, upper left corner vertical coordinate, lower right corner horizontal coordinate, and lower right corner vertical coordinate in order; points is used to describe the contour coordinate information of the area where the text is located, which is presented in the form of polygon vertex coordinates, and each element is a list containing horizontal and vertical coordinates; pred_id represents the number of the prediction result, and different numbers correspond to different recognition categories; pred represents the key parameter name corresponding to the recognized text, and the unlabeled text category is "NONE";

[0038] Use the python script to filter and retain all entries whose pred value is not "NONE" and the "transcription" entry in the dictionary where the entry is located, and generate an editable Excel document.

[0039] The step (8) specifically includes the following steps in order:

[0040] (8a) The logical relationship between the parameters of multiple views of the bridge structure within the same span includes:

[0041] The elevation, top plan and bottom plan are identified by the pier number to which they belong. Piers with the same pier number belong to the same span.

[0042] The relationship between the serial number of the cross-section drawing and the cut symbol number in the elevation drawing needs to be analyzed; within the same span, the serial number of the cross-section drawing must fall within the cut symbol number range between the two pier numbers in the elevation drawing;

[0043] (8b) Based on the logical relationship between the parameters of multiple views of the bridge structure within the same span, the editable Excel documents are merged and summarized into a new Excel document of key parameter information of the bridge structure within the same span to realize parameter extraction.

[0044] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, compared with the prior art, the present invention uses the improved YOLOv10 model to locate and intercept the image after the multi-view structure target detection of the bridge, and intercepts the bridge structure images under different views, namely, the cross-section view, the elevation view, the top plan view and the bottom plan view; uses the paddleocr model and the vi_layoutxlm_ser model to extract the position and value of the key parameters that need to be identified in each view; filters the content in the text file of the recognition result according to specific conditions, and finally integrates the Excel document of the same span key parameters according to the logical relationship of the parameters between multiple views; second, the present invention replaces the original structure corresponding to the YOLOv10 model with the DualConv module and the RT-DETR detection head, which can not only reduce the parameters, but also improve the model's recognition of the characteristics of components of different scales in the bridge design drawing. The extraction accuracy can also be combined with the real-time detection capability of the YOLOv10 model and the deep feature understanding capability of the RT-DETR detection head; thirdly, the present invention also assigns semantic information to the key parameters in the drawings, so that the parameter names and values ​​correspond to each other, realizes the modularization of the parameter recognition of each plane view, and can efficiently and automatically extract the key parameters of the PDF drawings; finally, the method of recognition and integration according to the span is adopted to successfully solve the parameter recognition problem of multiple views, and to conduct useful exploration for the artificial intelligence algorithm to understand engineering design drawings; fourth, the experimental results show that the accuracy and recall rate of the improved YOLOv10 model on the multi-view test set are 94.2% and 95.6% respectively; the key parameter results identified by the key information extraction method are basically consistent with the manual recognition results; the present invention can quickly and accurately extract the key parameters of multiple views in the PDF drawings of cast-in-place box girders. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of the method of the present invention;

[0046] Figure 2 It is a schematic diagram of the key parameters to be identified in the cross-section of the cast-in-place box girder;

[0047] Figure 3 This is a schematic diagram of key parameters to be identified for the cast-in-place box girder elevation;

[0048] Figure 4 It is a schematic diagram of key parameters to be identified in the top plan of the cast-in-place box girder;

[0049] Figure 5 It is a schematic diagram of the key parameters to be identified in the bottom plane of the cast-in-place box girder;

[0050] Figure 6 This is the result diagram of cross-section identification of cast-in-place box girder;

[0051] Figure 7 This is the result diagram of cast-in-place box girder elevation identification;

[0052] Figure 8 This is the result diagram of the top plane identification of the cast-in-place box girder;

[0053] Fig. 9 This is the result diagram of the bottom plane identification of the cast-in-place box girder;

[0054] Fig.10 It is a schematic diagram showing the overall results of extracting key parameters of the same span of the cast-in-situ box girder involved in the present invention. DETAILED DESCRIPTION

[0055] like Figure 1 As shown, a method for identifying cross-key parameters of cast-in-place box girder images based on target detection and key information extraction includes the following steps in sequence:

[0056] (1) Obtaining bridge design drawings and preprocessing them to obtain preprocessed bridge design drawings;

[0057] (2) Improve the YOLOv10 model to obtain a bridge structure detection model;

[0058] (3) Inputting the preprocessed bridge design drawings into the bridge structure detection model for training to obtain a trained bridge structure detection model;

[0059] (4) Obtain the design drawings of the bridge to be inspected and perform preprocessing on them. Input the preprocessed design drawings of the bridge to be inspected into the trained bridge structure detection model. The trained bridge structure detection model outputs an image of the bridge multi-view structure target after detection. The relative position of the bridge structure target is marked on the image of the bridge multi-view structure target after detection. The relative position information of the identified target is given in the image, so that the target area can be accurately captured, avoiding the complexity of manual cropping and realizing an automated process.

[0060] If there is a bridge structure target that needs to be identified in the input image, the model will output a recognition result image with a target detection box and the relative position information of the detection box in the image; if it is determined that there is no bridge structure target that meets its recognition conditions in the input image, the model will output the original image without any annotations and output the prompt sentence "no detections".

[0061] (5) According to the bridge multi-view structure target detection image, intercept the bridge structure image under different views, wherein the different views include a cross-sectional view, an elevation view, a top plan view, and a bottom plan view;

[0062] (6) Select different key parameters that need to be identified based on the bridge structure images under different views;

[0063] (7) Input the bridge structure images under different views and the different key parameters to be identified into the paddleocr model for feature extraction and feature fusion to obtain multimodal features. Input the multimodal features into the vi_layoutxlm_ser model for prediction to obtain an editable Excel document.

[0064] (8) Read the editable Excel document and, based on the logical relationship between the parameters of multiple views of the bridge structure within the same span, merge the editable Excel documents of the key parameters of each view into a new Excel document of the key parameter information of the bridge structure within the same span.

[0065] In order to identify different bridge structures from bridge design drawings, the YOLOv10 model is used to identify multiple views. YOLOv10 has fast detection speed and high efficiency in real time. Its advanced architecture design includes a backbone network with strong feature extraction capabilities, a lightweight classification head, and spatial channel decoupling downsampling, which improves operating efficiency and generalization capabilities. The advantage of no NMS training can cope with target detection problems in complex scenarios, improve recognition accuracy and reliability, and has high-precision recognition capabilities, a variety of flexible model sizes that can adapt to different resources and accuracy requirements, and good scalability for fine-tuning for specific tasks, thereby facilitating accurate bridge structure recognition.

[0066] The step (1) specifically includes the following steps in order:

[0067] (1a) Use the pdf2image open source library to convert the archived PDF bridge design drawings into JPG images. The resolution of each JPG image is 3573×2526.

[0068] (1b) Perform data enhancement processing, rotate all JPG format images and add Gaussian noise;

[0069] (1c) Labeling: Use the Labelimg tool to label the bridge structure in multiple views of the image after data enhancement, which are divided into cross-section, elevation, top plan and bottom plan. The labeling format is YOLO format.

[0070] The step (2) specifically refers to: improving the YOLOv10 model, replacing all C2f modules in the YOLOv10 model with DualConv modules; and replacing the detection head in the YOLOv10 model with the detection head of the RT-DETR model.

[0071] The DualConv module is an innovative convolutional network structure designed to build a lightweight deep neural network. It combines the advantages of 3*3 and 1*1 convolution kernels to process the same input feature map channels. The 3*3 convolution kernel can capture more spatial information when performing feature extraction, while the 1*1 convolution kernel can interact and integrate information between feature channels without increasing too many parameters and computational complexity. The DualConv module also uses group convolution technology to efficiently arrange convolution filters. In group convolution, the input and output feature maps are divided into multiple groups, and the convolution filters of each group only process part of the corresponding input feature map, which reduces the complexity of the model. Since different convolution kernels in the group (such as 3*3 and 1*1) can process the same group of input channels in parallel, the information flow and feature extraction efficiency are optimized, while maintaining the representation ability of the network and further reducing the computational cost.

[0072] The detection head of the RT-DETR model uses a more advanced and efficient feature fusion method, which can carefully integrate features of different scales and levels, making the model's feature extraction of the target more comprehensive and accurate. At the same time, compared with the detection head of YOLOv10, the detection head of the RT-DETR model is more lightweight in design, which means that while maintaining good detection performance, it greatly reduces the number of model parameters and computational complexity, significantly improving detection efficiency. Not only that, the detection head of RT-DETR also has excellent multimodal information fusion capabilities. In addition to conventional visual features, it can also integrate information from other modalities such as depth and semantics, which helps to further clarify the category of the target, thereby greatly enhancing the model's ability to detect and identify targets in complex scenarios.

[0073] The step (5) specifically includes the following steps in order:

[0074] (5a) Obtain the target frame information in the image after the bridge multi-view structure target detection, that is, the relative position of the bridge structure target in the image. The target frame information includes:<class_label><x_center><y_center> <width>and <height>;<class_label> Refers to the target category label, which uses an integer to represent the category number to which the detected target belongs;<x_center> and<y_center> Respectively refer to the x-coordinate and y-coordinate of the center point of the target frame; <width>and <height>Refers to the width and height of the target box respectively;

[0075] (5b) Convert the target frame information into coordinate information according to the following conversion relationship:

[0076] left = int((x_center - width / 2) * image.shape[1])

[0077] top = int((y_center - height / 2) * image.shape[0])

[0078] right = int((x_center + width / 2) * image.shape[1])

[0079] bottom = int((y_center + height / 2) * image.shape[0])

[0080] In the formula, image.shape[1] represents the width of the image, image.shape[0] represents the height of the image, left represents the horizontal coordinate position of the left boundary of the target box in the image; right represents the horizontal coordinate position of the right boundary of the target box in the image; top represents the vertical coordinate position of the upper boundary of the target box in the image; bottom represents the vertical coordinate position of the lower boundary of the target box in the image;

[0081] For each target, according to the target's coordinate information, a python script is used to capture the image of the bridge multi-view structure target after detection, and the bridge structure images under different views are captured, namely, cross-section, elevation, top plan and bottom plan. The captured bridge structure images under different views are named "original image name + view name" and saved in the folders of each view accordingly.

[0082] The step (6) specifically refers to:

[0083] like Figure 2 As shown in the figure, the key parameters that need to be identified in determining the cross-section diagram include: serial number, box beam upper flange width, box beam lower flange width, flange left cantilever width, flange right cantilever width, beam height, cantilever end thickness and cantilever root thickness, a total of 8 parameters;

[0084] like Figure 3 As shown, the key parameter names that need to be identified in determining the elevation drawing include: section symbol, pier number, span and beam height, a total of 4 parameters;

[0085] like Figure 4 As shown, the key parameters that need to be identified to determine the top plan include: pier number, span and beam height, a total of 3 parameters;

[0086] like Figure 5 As shown, the key parameters that need to be identified to determine the bottom plan include: pier number, span and beam height, a total of 3 parameters.

[0087] As a blueprint for guiding bridge construction, bridge design drawings cover a wide variety of parameters and a large number of them. These parameters involve all aspects of the bridge, including but not limited to the size of structural components, material characteristics, mechanical performance indicators, and various structural details. However, in the actual workflow, for a specific task, it is often necessary to focus on only a part of the key parameters. If all parameters are identified and extracted indiscriminately, the workload will undoubtedly increase exponentially, which will not only consume a lot of manpower, material resources and time resources, but also may lead to low work efficiency and unnecessary waste of resources. Therefore, it is undoubtedly a more wise and appropriate strategy to select those parameters that are closely related to the current task in a targeted manner according to specific needs. This approach can not only accurately meet the requirements of actual work, but also optimize resource allocation to the greatest extent, improve work efficiency, and ensure the efficiency and accuracy of bridge design drawing parameter identification and extraction.

[0088] The step (7) specifically includes the following steps in order:

[0089] (7a) annotating the bridge structure images under different views, marking the key parameters on the bridge structure images under different views, and obtaining the annotated bridge structure images under different views;

[0090] (7b) The annotated bridge structure images under different views are input into the paddleocr model for feature extraction, and the text information, text coordinate information and layout feature information are extracted; then, the extracted text information, text coordinate information and layout feature information are encoded and converted, and fused into multimodal features; subsequently, these multimodal features are input into the vi_layoutxlm_ser model for forward reasoning calculation, and the category, position and specific text content of each semantic entity in the image are predicted, and the recognition result image and text file are output. The recognition result image is shown in the figure. Figure 6 , Figure 7 , Figure 8 , Fig. 9 shown.

[0091] Considering the particularity and complexity of bridge design drawings, which contain rich text information, including parameter information, names, etc., which are extremely meaningful for parameter reference, it is necessary to explore effective methods of character recognition. Paddleocr is an open source ultra-lightweight OCR system model of Baidu PaddlePaddle, which can be widely used in many fields such as document recognition, bill recognition, digital reading, etc. It has detection + classification + recognition functions and supports multiple language recognition, such as mixed recognition of multiple character types such as Chinese, English, and numbers.

[0092] (7c) Processing the text file: The text file includes the image path and each text recognition result entry, which includes the transcription, bbox, points, pred_id and pred fields and their corresponding values; transcription represents the text content recognized in the image; bbox represents the bounding box coordinate information of the area where the recognized text is located, which is given in the form of a list, and represents the upper left corner horizontal coordinate, upper left corner vertical coordinate, lower right corner horizontal coordinate, and lower right corner vertical coordinate in order; points is used to describe the contour coordinate information of the area where the text is located, which is presented in the form of polygon vertex coordinates, and each element is a list containing horizontal and vertical coordinates; pred_id represents the number of the prediction result, and different numbers correspond to different recognition categories; pred represents the key parameter name corresponding to the recognized text, and the unlabeled text category is "NONE";

[0093] Use the python script to filter and retain all entries whose pred value is not "NONE" and the "transcription" entry in the dictionary where the entry is located, and generate an editable Excel document. The summary document only requires the key parameter name category pred and the specific content transcription. Simply summarizing all output results will result in repeated key parameters of each subview, causing data redundancy and increasing the workload of subsequent information processing and maintenance. Therefore, it is necessary to filter out irrelevant information.

[0094] The step (8) specifically includes the following steps in order:

[0095] (8a) The logical relationship between the parameters of multiple views of the bridge structure within the same span includes:

[0096] The elevation, top plan and bottom plan are identified by the pier number to which they belong. Piers with the same pier number belong to the same span.

[0097] The relationship between the serial number of the cross-section drawing and the cutting symbol number in the elevation drawing needs to be analyzed; within the same span, the serial number of the cross-section drawing must fall within the range of the cutting symbol numbers between the two pier numbers in the elevation drawing; for example, if the cutting symbol number range of the elevation drawing is 1 to 4, then the serial number of the cross-section drawing in the same span should be 1-1 to 4-4.

[0098] (8b) Based on the logical relationship between the parameters of multiple views of the bridge structure within the same span, the editable Excel documents are merged and summarized into a new Excel document of key parameter information of the bridge structure within the same span to realize parameter extraction.

[0099] Cast-in-place box girder design drawings are usually complex, and identifying the design parameters of the entire bridge can be very difficult, especially when a lot of details and complex structures are involved. In bridge design and engineering terminology, the entire bridge is usually divided into several spans, each span representing the distance between two piers (or supports). Therefore, during the parameter identification process, the entire bridge can be divided into each span for identification, breaking down the large task into small tasks, and the identification of each span is relatively independent, which reduces the complexity and possibility of confusion and ensures the accuracy and integrity of the data. Span-by-span identification can also improve work efficiency. The identification of each span can be carried out in parallel without waiting for all the data of the entire bridge to be identified. Data management after span-by-span identification is simpler and clearer. The data of each span can be archived and managed independently, and can be processed separately when updated, without affecting the management and maintenance of the entire data set.

[0100] like Fig.10 As shown, Fig.10 All key parameters included in the first span of the general structure diagram of the cast-in-place box girder are shown. In the first span, there are section symbols 1, 2, 3 and 4 and pier numbers 0 and 1. The span of this span is 2495 cm and the beam height is 160 cm. Each section symbol position corresponds to a section, and there are 4 sections in total in this span. The width of the upper flange and the lower flange of the box girder of section 1-1 are 850.1 cm and 450.1 cm respectively, the width of the upper flange and the lower flange of the box girder of section 2-2 are 853.2 cm and 453.2 cm respectively, the width of the upper flange and the lower flange of the box girder of section 3-3 are 874.9 cm and 474.8 cm respectively, the width of the upper flange and the lower flange of the box girder of section 4-4 are 903.1 cm and 503.1 cm respectively; the thickness of the cantilever end of section 1-1 is 50 cm, and the thickness of the cantilever ends of the other three sections is 20 cm; the thickness of the cantilever root, the cantilever width on the left side of the flange and the cantilever width on the right side of the flange of each section are the same, which are 50 cm, 200.1 cm and 200 cm respectively.

[0101] In summary, the present invention uses the improved YOLOv10 model to locate and intercept the image after the multi-view structure target detection of the bridge, and intercepts the bridge structure images under different views, namely the cross-section view, elevation view, top plan view and bottom plan view; uses the paddleocr model and the vi_layoutxlm_ser model to extract the position and value of the key parameters that need to be identified in each view; filters the content in the text file of the recognition result according to specific conditions, and finally integrates the Excel document of the same span key parameters according to the logical relationship of the parameters between multiple views; the present invention replaces the original structure corresponding to the YOLOv10 model with the DualConv module and the RT-DETR detection head, which can not only reduce parameters and improve the model's extraction accuracy of the features of components of different scales in the bridge design drawing, but also can combine Y The real-time detection capability of the OLOv10 model and the deep feature understanding capability of the RT-DETR detection head; the present invention also assigns semantic information to the key parameters in the drawings, makes the parameter names and values ​​correspond to each other, realizes the modularization of the parameter recognition of each plane drawing, and can efficiently and automatically extract the key parameters of the PDF drawings; finally, the method of recognition and integration according to the span is adopted to successfully solve the parameter recognition problem of multiple views, and a useful exploration is made for the artificial intelligence algorithm to understand the engineering design drawings; experimental results show that the accuracy and recall rate of the improved YOLOv10 model on the multi-view test set are 94.2% and 95.6% respectively; the results of the key parameters identified by the key information extraction method are basically consistent with the manual recognition results; the present invention can quickly and accurately extract the key parameters of multiple views in the PDF drawings of cast-in-place box girders.< / height> < / width> < / height> < / width> < / height> < / width> < / height> < / width>

Claims

1. A method for identifying cross-key parameters of cast-in-place box girder images based on target detection and key information extraction, characterized in that: The method comprises the following steps in order: (1) Obtaining bridge design drawings and preprocessing them to obtain preprocessed bridge design drawings; (2) Improve the YOLOv10 model to obtain a bridge structure detection model; (3) Inputting the preprocessed bridge design drawings into the bridge structure detection model for training to obtain a trained bridge structure detection model; (4) Obtaining the design drawings of the bridge to be inspected and preprocessing them, inputting the preprocessed design drawings of the bridge to be inspected into the trained bridge structure detection model, and the trained bridge structure detection model outputs an image of the bridge multi-view structure target after detection, and marking the relative position of the bridge structure target on the image of the bridge multi-view structure target after detection; (5) According to the bridge multi-view structure target detection image, intercept the bridge structure image under different views, wherein the different views include a cross-sectional view, an elevation view, a top plan view, and a bottom plan view; (6) Select different key parameters that need to be identified based on the bridge structure images under different views; (7) Input the bridge structure images under different views and the different key parameters to be identified into the paddleocr model for feature extraction and feature fusion to obtain multimodal features. Input the multimodal features into the vi_layoutxlm_ser model for prediction to obtain an editable Excel document. (8) Read the editable Excel document and, based on the logical relationship between the parameters of multiple views of the bridge structure within the same span, merge the editable Excel documents of the key parameters of each view into a new Excel document of the key parameter information of the bridge structure within the same span.

2. The method for identifying cross-key parameters of cast-in-place box girder images based on target detection and key information extraction according to claim 1 is characterized in that: The step (1) specifically includes the following steps in order: (1a) Use the pdf2image open source library to convert the archived PDF bridge design drawings into JPG images. The resolution of each JPG image is 3573×2526. (1b) Perform data enhancement processing, rotate all JPG format images and add Gaussian noise; (1c) Labeling: Use the Labelimg tool to label the bridge structure in multiple views of the image after data enhancement, which are divided into cross-section, elevation, top plan and bottom plan. The labeling format is YOLO format.

3. The method for identifying cross-key parameters of cast-in-place box girder images based on target detection and key information extraction according to claim 1 is characterized in that: The step (2) specifically refers to: improving the YOLOv10 model, replacing all C2f modules in the YOLOv10 model with DualConv modules; and replacing the detection head in the YOLOv10 model with the detection head of the RT-DETR model.

4. The method for identifying cross-key parameters of cast-in-place box girder images based on target detection and key information extraction according to claim 1 is characterized in that: The step (5) specifically includes the following steps in order: (5a) Obtain the target frame information in the image after the bridge multi-view structure target detection, that is, the relative position of the bridge structure target in the image. The target frame information includes:<class_label><x_center><y_center> <width>and <height>;<class_label> Refers to the target category label, which uses an integer to represent the category number to which the detected target belongs;<x_center> and<y_center> Respectively refer to the x-coordinate and y-coordinate of the center point of the target frame; <width>and <height> Refers to the width and height of the target box respectively;< / height> < / width> < / height> < / width> (5b) Convert the target frame information into coordinate information according to the following conversion relationship: left = int((x_center - width / 2) * image.shape[1]) top = int((y_center - height / 2) * image.shape[0]) right = int((x_center + width / 2) * image.shape[1]) bottom = int((y_center + height / 2) * image.shape[0]) In the formula, image.shape[1] represents the width of the image, image.shape[0] represents the height of the image, left represents the horizontal coordinate position of the left boundary of the target box in the image; right represents the horizontal coordinate position of the right boundary of the target box in the image; top represents the vertical coordinate position of the upper boundary of the target box in the image; bottom represents the vertical coordinate position of the lower boundary of the target box in the image; For each target, according to the target's coordinate information, a python script is used to capture the image of the bridge multi-view structure target after detection, and the bridge structure images under different views are captured, namely, cross-section, elevation, top plan and bottom plan. The captured bridge structure images under different views are named "original image name + view name" and saved in the folders of each view accordingly.

5. The method for identifying cross-key parameters of cast-in-place box girder images based on target detection and key information extraction according to claim 1 is characterized in that: The step (6) specifically refers to: The key parameters that need to be identified in determining the cross-section diagram include: serial number, box beam upper flange width, box beam lower flange width, flange left cantilever width, flange right cantilever width, beam height, cantilever end thickness and cantilever root thickness, a total of 8 parameters; The key parameter names that need to be identified in the elevation drawing include: section symbol, pier number, span and beam height, a total of 4 parameters; The key parameters that need to be identified in determining the top plan include: pier number, span and beam height, a total of 3 parameters; The key parameter names that need to be identified to determine the bottom plan include: pier number, span and beam height, a total of 3 parameters.

6. The method for identifying cross-key parameters of cast-in-place box girder images based on target detection and key information extraction according to claim 1 is characterized in that: The step (7) specifically includes the following steps in order: (7a) annotating the bridge structure images under different views, marking the key parameters on the bridge structure images under different views, and obtaining the annotated bridge structure images under different views; (7b) Inputting the annotated bridge structure images under different views into the PaddleOCR model for feature extraction, extracting text information, text coordinate information and layout feature information; Then, the extracted text information, text coordinate information and layout feature information are encoded and converted, and then fused into multimodal features. Subsequently, these multimodal features are input into the vi_layoutxlm_ser model for forward reasoning calculation, and the category, position and specific text content of each semantic entity in the image are predicted, and the recognition result image and text file are output. (7c) Processing the text file: The text file includes the image path and each text recognition result entry. The entry includes the transcription, bbox, points, pred_id and pred fields and their corresponding values. Transcription represents the text content recognized in the image. Bbox represents the bounding box coordinate information of the recognized text area, which is given in the form of a list, indicating the upper left corner horizontal coordinate, upper left corner vertical coordinate, lower right corner horizontal coordinate, and lower right corner vertical coordinate in order. Points is used to describe the contour coordinate information of the text area, which is presented in the form of polygon vertex coordinates, and each element is a list containing horizontal and vertical coordinates. pred_id represents the number of the prediction result. Different numbers correspond to different recognition categories. pred indicates the key parameter name of the corresponding recognized text. The unlabeled text category is "NONE"; Use the python script to filter and retain all entries whose pred value is not "NONE" and the "transcription" entry in the dictionary where the entry is located, and generate an editable Excel document.

7. The method for identifying cross-key parameters of cast-in-place box girder images based on target detection and key information extraction according to claim 1 is characterized in that: The step (8) specifically includes the following steps in order: (8a) The logical relationship between the parameters of multiple views of the bridge structure within the same span includes: The elevation, top plan and bottom plan are identified by the pier number to which they belong. Piers with the same pier number belong to the same span. The relationship between the serial number of the cross-section drawing and the cut symbol number in the elevation drawing needs to be analyzed; within the same span, the serial number of the cross-section drawing must fall within the cut symbol number range between the two pier numbers in the elevation drawing; (8b) Based on the logical relationship between the parameters of multiple views of the bridge structure within the same span, the editable Excel documents are merged and summarized into a new Excel document of key parameter information of the bridge structure within the same span to realize parameter extraction.

Images