An identification method and device for engineering drawings and electronic equipment

Abstract

The application relates to the technical field of image detection, and discloses a method and device for identifying indexes of engineering drawings and electronic equipment, the method comprising the following steps: acquiring a to-be-detected image, the to-be-detected image comprising an engineering drawing; detecting indexes of the to-be-detected image by using a preset index detection model to obtain index positions; detecting all indexes in the to-be-detected image to obtain first index positions corresponding to to-be-identified indexes; detecting secondary indexes in the to-be-detected image to obtain second index positions corresponding to the secondary indexes; extracting the first index positions from the second index positions by using the index positions and the second index positions to obtain size index positions; the to-be-identified indexes comprising size indexes, index indexes and secondary indexes; and identifying the index positions and / or the size index positions to obtain an identification result corresponding to the engineering drawing.

Description

Technical Field

[0001] This application relates to the field of image detection technology, specifically to a method, apparatus, and electronic device for identifying markings on engineering drawings. Background Technology

[0002] Engineering drawings, as technical documents describing the design, dimensions, materials, and construction requirements of architectural, structural, mechanical, and electrical engineering projects, are indispensable reference documents in the construction process. They are commonly used to guide construction, review designs, procure materials, estimate costs, control quality, and maintain the project. The digital storage of engineering drawings and the extraction of information from them have significant practical value in fields such as mechanical and electronic systems, and have become a hot research topic in academia and engineering.

[0003] With the rapid development of drafting technology, the number of engineering drawings is increasing daily, making the retrieval and querying of these large quantities of drawings a pressing problem. Manual comparison for searching is cumbersome and inefficient, and repetitive work over extended periods can lead to high error rates. Traditional feature-based detection methods are inefficient; however, deep learning, with its powerful learning and feature representation capabilities, can achieve better results. Text detection technologies based on OCR and deep learning are gradually becoming the mainstream methods for text detection due to their higher efficiency and accuracy.

[0004] While general OCR models have achieved encouraging results in various fields such as digital tube recognition, general form recognition, and license plate recognition, these models cannot be directly used for text recognition on engineering drawings. Due to the varying sizes of engineering drawings, potential text rotation within the drawings, and the easy misidentification of symbols as text, related text recognition methods for engineering drawings are prone to omissions and misidentifications. Therefore, a new method for identifying markings on engineering drawings is urgently needed to more accurately and comprehensively locate and identify index and dimension markings within them.

[0005] It should be noted that the above content only provides background information related to this application and does not necessarily constitute prior art. Summary of the Invention

[0006] In view of the shortcomings of the prior art described above, this application provides a method, apparatus and electronic device for identifying markings in engineering drawings, so as to more accurately and comprehensively locate and identify index markings and dimension markings in engineering drawings.

[0007] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0008] According to one aspect of the embodiments of this application, a method for identifying identifiers in engineering drawings is provided, comprising: acquiring an image to be detected, wherein the image to be detected includes an engineering drawing; performing index identifier detection on the image to be detected using a preset index identifier detection model to obtain index identifier positions, wherein the index identifier detection model is trained based on a base model, the base model being a YOLO model in which the C2F structure in the backbone is replaced with a Swing Transformer structure; detecting all identifiers in the image to be detected to obtain a first identifier position corresponding to an identifier to be identified, and detecting secondary identifiers in the image to be detected to obtain a second identifier position corresponding to a secondary identifier; extracting a size identifier position from the first identifier position using the index identifier position and the second identifier position, wherein the identifier to be identified includes a size identifier, an index identifier, and a secondary identifier; and performing identifier identification on the index identifier position and / or the size identifier position to obtain an identification result corresponding to the engineering drawing.

[0009] In one embodiment of this application, based on the aforementioned scheme, index identification detection is performed on the image to be detected using a preset index identification detection model, including: cutting the image to be detected according to a first preset size to obtain one or more image segments, and performing index identification detection on the image segments using the index identification detection model.

[0010] In one embodiment of this application, based on the foregoing scheme, the method further includes: determining a sliding window from the image segment; controlling the sliding window to slide within the image segment, and performing index identifier detection on the sliding window to obtain at least one original detection box's sub-image position information corresponding to the image segment and the index identifier confidence level corresponding to the original detection box, wherein the original detection box contains an index identifier; performing non-maximum suppression when integrating the image segment based on the offset position of the image to be detected; determining the original image position information of the original detection box in the image to be detected based on the sub-image position information corresponding to the original detection box in the image segment; and determining any one of the original detection boxes as the first detection box. The original detection boxes other than the first detection box are respectively identified as second detection boxes; the first area intersection-union ratio is calculated based on the intersection of the first area and the second area, and the union of the first area and the second area. The first area is obtained based on the original image position information of the first detection box, and the second area is obtained based on the original image position information of the second detection box; if the first area intersection-union ratio is greater than a preset first area threshold, the index identifier confidence of the first detection box and the index identifier confidence of the second detection box are compared to determine the target detection box with the highest index identifier confidence from the first detection box and the second detection box, and the original image position information corresponding to the target detection box is used as the index identifier position.

[0011] In one embodiment of this application, based on the aforementioned scheme, detecting secondary identifiers in the image to be detected and obtaining the second identifier position corresponding to the secondary identifier includes: scaling down the image to be detected proportionally to a second preset size with its long side as the size, and recording the scaling down ratio; inputting the adjusted image to be detected into a preset secondary identifier detection model to obtain the secondary identifier position, wherein the secondary identifier detection model is trained based on identifier samples with secondary identifier labels, and the secondary identifier position is the location of the secondary identifier in the adjusted image to be detected; and calculating the second identifier position based on the secondary identifier position and the scaling down ratio.

[0012] In one embodiment of this application, based on the aforementioned scheme, the size identifier position is obtained by extracting it from the first identifier position using the index identifier position and the second identifier position. This includes: determining the area corresponding to the first identifier position as the first area, determining the area corresponding to the index identifier position as the index area, and determining the area corresponding to the second identifier position as the second area; calculating a second area intersection-union ratio based on the intersection of the first area area and the index area, and the union of the first area area and the index area; if the second area intersection-union ratio is greater than a first preset threshold, then determining that the first identifier position and the index identifier position are the same position; calculating the intersection area of ​​the first area area and the second area area, and calculating the area ratio based on the intersection area and the second area area; if the area ratio is greater than a second preset threshold, then determining that the first identifier position is located inside the second identifier position; and filtering the index identifier position and the second identifier position from the first identifier position to obtain the size identifier position.

[0013] In one embodiment of this application, based on the aforementioned scheme, identification is performed on the index identifier position and / or the dimension identifier position to obtain the identification result corresponding to the engineering drawing, including: integrating the index identifier position and the dimension identifier position to obtain an integrated result, wherein the integrated result includes identifier position and identifier category; cropping from the image to be detected according to the identifier position to obtain a cropped image that only includes identifier content; and performing identifier identification on the cropped image to obtain the identification result corresponding to the engineering drawing.

[0014] In one embodiment of this application, based on the aforementioned scheme, the Swing Transformer structure sequentially includes: a first stage, which includes a Patch Embedding unit and multiple Block units, wherein the Patch Embedding unit is used to segment the input first image and embed it into a vector to obtain a feature map; the Block unit is used to perform layer normalization on the feature map, segment the feature map into multiple windows, calculate Attention and determine whether it is Window Attention or Shift Window Attention, merge the windows, if a Shift operation has been performed, perform ReverseShift, perform Dropout and residual connections, and then perform layer normalization, fully connected layers, Dropout and residual connections; and multiple second stages connected sequentially, wherein the second stage includes a Patch Merging unit and multiple Block units, wherein the Patch Merging unit is used to perform downsampling to obtain a downsampling result, and concatenate the downsampling result to obtain a concatenated result, and adjust the number of channels of the concatenated result through a fully connected layer.

[0015] In one embodiment of this application, based on the aforementioned scheme, the base model is a YOLO model in which the CBS structure in the neck is replaced with the GSConv structure; wherein, the GSConv structure is used to perform ordinary convolution downsampling on the input second image to obtain the downsampling result, and to perform depth convolution on the input second image to obtain the depth convolution result; the downsampling result and the depth convolution result are concatenated and a shuffle operation is performed.

[0016] According to one aspect of the embodiments of this application, a device for identifying markings on engineering drawings is provided, comprising: an acquisition module for acquiring an image to be detected, wherein the image to be detected includes an engineering drawing; an index marking module for detecting index markings on the image to be detected using a preset index marking detection model to obtain index marking positions, wherein the index marking detection model is trained based on a base model, the base model being a YOLO model with the C2F structure in the backbone replaced by a Swin Transformer structure; a size marking module for detecting all markings in the image to be detected to obtain a first marking position corresponding to a marking to be identified, and detecting secondary markings in the image to be detected to obtain a second marking position corresponding to a secondary marking, and extracting a size marking position from the first marking position using the index marking position and the second marking position, wherein the markings to be identified include size markings, index markings, and secondary markings; and an identification module for identifying the markings at the index marking positions and / or the size marking positions to obtain an identification result corresponding to the engineering drawing.

[0017] This application also provides an electronic device, the electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device enables the identification method for engineering drawings as described in any of the above embodiments.

[0018] The beneficial effects of this application are as follows: This application uses a pre-defined index identifier detection model to detect index identifiers in the image to be detected, thereby obtaining the index identifier positions. The index identifier detection model is trained based on a base model, which is a YOLO model with the C2F structure in the backbone replaced by a Swin Transformer structure. The YOLOv8 network is improved by using the Swin Transformer module, which allows the model to learn cross-window information without reducing the computation speed. All identifiers in the image to be detected are detected to obtain the first identifier position corresponding to the identifier to be identified. Secondary identifiers in the image to be detected are also detected to obtain the second identifier position corresponding to the secondary identifier. The size identifier position is extracted from the first identifier position using the index identifier position and the second identifier position, removing obvious erroneous identification results and thus reducing misidentification. Identification is performed on the index identifier position and / or the size identifier position to obtain the identification result corresponding to the engineering drawing, so as to more accurately and comprehensively locate and identify index identifiers and size identifiers in the engineering drawing.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0021] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application;

[0022] Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of the present application of a method for identifying markings on engineering drawings;

[0023] Figure 3 This is an exemplary embodiment of the present application illustrating a method for identifying markings on engineering drawings;

[0024] Figure 4a This is a schematic diagram of the network structure of a SwinTransformer for an identification method for engineering drawings, as illustrated in an exemplary embodiment of this application.

[0025] Figure 4b This is a schematic diagram of a two-stage Swing Transformer Block structure for an identification method for engineering drawings, illustrating an exemplary embodiment of this application.

[0026] Figure 5 This is a schematic diagram of the structure of GSConv for an identification method for engineering drawings, as illustrated in an exemplary embodiment of this application;

[0027] Figure 6 This is a PR curve showing the test results of a method for identifying markings on engineering drawings, as illustrated in an exemplary embodiment of this application.

[0028] Figure 7 This is a schematic diagram of a non-dimensional marking area illustrating an exemplary embodiment of the marking and identification method for engineering drawings.

[0029] Figure 8 This is an exemplary embodiment of the present application illustrating a non-dimensional marking example of a marking and identification method for engineering drawings;

[0030] Figure 9 This is a flowchart illustrating a method for identifying markings on engineering drawings, as shown in another exemplary embodiment of this application.

[0031] Figure 10 This is a block diagram illustrating an identification device for engineering drawings, as shown in an exemplary embodiment of this application;

[0032] Figure 11 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0033] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0034] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the illustrations only show the components related to this application and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0035] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0036] First, it's important to understand that YOLOv8 is an object detection algorithm. Building upon the strengths of its predecessors, it incorporates numerous improvements and optimizations, significantly enhancing the accuracy and speed of object detection. YOLO (You Only Look Once) is a one-stage object detection algorithm, meaning it can identify the category and bounding box of objects in an image with just one "look." YOLOv8 builds upon previous versions of the YOLO series, introducing new features and improvements, such as a new backbone network, an anchorless detection head, and a new loss function, to enhance the model's performance and flexibility. The YOLOv8 detection model is the latest object detection model based on the YOLO (You Only Look Once) series. While maintaining the real-time detection capabilities of the YOLO series, it further improves performance and flexibility by introducing new features and improvements.

[0037] Optical Character Recognition (OCR) is a technology that converts text in scanned images into editable and searchable digital text. OCR technology is widely used in document management, automated processing, information extraction, and accessible reading.

[0038] Text detection based on OCR (Optical Character Recognition) and deep learning is an important research area, with wide applications in document analysis, autonomous driving, image search, and robot vision. OCR technology is primarily used to identify and extract text from images, while deep learning provides powerful tools for OCR, making text detection more accurate and efficient.

[0039] Intersection over Union (IoU) is a commonly used evaluation metric in tasks such as object detection, semantic segmentation, and tracking. It measures the degree of overlap between the predicted bounding box and the actual target bounding box, and can also be used to measure the degree of overlap between two bounding boxes.

[0040] Depthwise separable convolution is an efficient convolution operation that breaks down the standard convolution operation into two steps: depthwise convolution and pointwise convolution, also known as 1x1 convolution. This decomposition significantly reduces the computational cost and number of parameters in the model while maintaining its performance. In feature fusion, depthwise separable convolution can be used to effectively combine features from different levels or different sources.

[0041] Non-dimensional marking inspection, also known as non-standard inspection, is a quality inspection model that does not follow uniformly promulgated industry standards, but rather uses specialized testing instruments designed according to actual testing needs for precision testing. This inspection method mainly focuses on markings and characteristics other than dimensions, such as surface defects, material properties, and shape.

[0042] Layer normalization is a commonly used technique in deep neural networks, primarily used to accelerate model training and improve model performance. Layer normalization normalizes the neurons in a specific layer of the neural network, rather than normalizing the entire batch of data. Specifically, layer normalization normalizes the activation values ​​of each sample independently, unlike batch normalization which normalizes the entire batch. Layer normalization calculates the mean and variance of the neurons in a specific layer, and then uses these mean and variance to normalize the activation values ​​of that layer.

[0043] Non-maximum suppression (NMS) is a commonly used technique in image processing, computer vision, and other fields. It is primarily used to suppress non-maximum elements and can be understood as a form of local maximum search. By suppressing non-maximum elements, key information in an image can be effectively extracted, facilitating subsequent processing and analysis.

[0044] Confidence level is a key concept in statistics, used to describe the degree of certainty or confidence in an estimated interval of a population parameter constructed from a sample statistic. It is a probability value representing the probability that the true value of a population parameter falls within a certain interval constructed from the sample statistic.

[0045] In machine learning and data science, negative examples generally refer to samples that are labeled as not belonging to the target class in a classification task. In other words, they are instances that do not belong to the class we are trying to predict or classify. In contrast, positive examples are samples that are labeled as belonging to the target class.

[0046] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0047] Reference Figure 1As shown, the system architecture may include an image acquisition device 101 and a computer device 102. The computer device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, or a neural network computer. The image acquisition device 101 is used to acquire images to be detected, wherein the images to be detected include engineering drawings. In this embodiment, after acquiring the images, the image acquisition device 101 provides them to the computer device 102 for processing. Those skilled in the art can use the computer device 102 to perform index mark detection on the image to be detected using a preset index mark detection model to obtain the index mark positions. The index mark detection model is trained based on a base model, which is a YOLO model where the C2F structure in the backbone is replaced with a Swin Transformer structure. All marks in the image to be detected are detected to obtain the first mark position corresponding to the mark to be identified. Secondary marks in the image are also detected to obtain the second mark position corresponding to the secondary mark. The size mark position is then extracted from the first mark position using the index mark position and the second mark position. The mark to be identified includes size marks, index marks, and secondary marks. Marker recognition is performed on the index mark position and / or the size mark position to obtain the recognition result corresponding to the engineering drawing. It should be noted that the image acquisition device 101 and computer device 102 provided in this embodiment are merely examples and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0048] It should be noted that the identification method for engineering drawings provided in this application embodiment is generally executed by computer device 102, and correspondingly, the identification device for engineering drawings is generally installed in computer device 102.

[0049] Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of the method for identifying markings on engineering drawings. This method can be executed by a computing processing device, which may be... Figure 1 The computer device 102 shown is illustrated. (Refer to...) Figure 2 As shown, the identification method for engineering drawings includes at least steps S210 to S240, which are described in detail below:

[0050] In step S210, the image to be detected is acquired.

[0051] In one embodiment of this application, the image to be detected includes, but is not limited to, engineering drawings, which include, but are not limited to, architectural drawings and mechanical drawings. Methods for acquiring the image to be detected include, but are not limited to, one or more of the following: scanning the engineering drawings, taking screenshots of the engineering drawings, and storing the engineering drawings.

[0052] In step S220, the index identifiers of the image to be detected are detected using a preset index identifier detection model to obtain the index identifier positions.

[0053] The index identifier detection model is trained based on a base model, which is a YOLO model where the C2F structure in the backbone is replaced with a Swin Transformer structure. It's worth noting that the C2F structure in the YOLOv8 backbone is a new network structure introduced in YOLOv8. The C2F module borrows the ELAN idea from YOLOv7, using more parallel gradient flow branches to obtain richer gradient flow information while maintaining lightweight architecture. In C2F, the number of channels in the input tensor of each BottleNeck is only 0.5 times that of the previous stage, thus significantly reducing computational cost. Furthermore, the increased gradient flow significantly improves convergence speed and performance. The Swin Transformer is a transformer-based image classification network structure characterized by: image patch partitioning (dividing the input image into multiple non-overlapping patches); window partitioning (partitioning the patches according to a certain window size); and deep network (processing the features of each partition through a deep network to obtain a higher-level feature representation). Swin Transformer's self-attention computation is limited to a local window, making the computational complexity linear rather than quadratic with image size. This design not only improves the model's efficiency but also maintains strong feature extraction capabilities. Swin Transformer can capture image details and global information at different levels, making it a powerful and general-purpose backbone network for various vision tasks.

[0054] In this embodiment, a pre-trained improved YOLOv8 detection model, namely the index marker detection model, is used to detect the index markers in the engineering drawing to be detected, i.e., the image to be detected, to obtain the index marker positions. Specifically, the YOLOv8 network, i.e., the base model, is improved based on the features of the index markers to obtain the improved YOLOv8 detection model, i.e., the index marker detection model. This model adopts a hierarchical design to extract features in the backbone network, which includes a total of four stages. Each stage reduces the resolution of the input feature map and expands the receptive field layer by layer, and uses depthwise separable convolution to fuse features. The input drawing needs to be cut according to a sliding window of a fixed size (first preset size) to ensure that the input drawing size is the same.

[0055] In this embodiment, the index identifiers include three categories: leader line identifiers, section plane identifiers, and sectional view identifiers. The index identifier information in the drawings is pre-annotated, categorizing the index identifiers in the engineering drawings into three types: leader line identifiers, section plane identifiers, and cross-sectional identifiers (i.e., sectional view identifiers). For example... Figure 3 As shown, Figure 3 This is an exemplary embodiment of the present application illustrating a method for identifying markings on engineering drawings, wherein the serial number “(26)” indicates a leader line marking, the letter “L” next to the broken line arrow indicates a cross-section marking, and “AA” indicates a cross-section marking. Labelme is used to annotate the three types of marking information in the drawings, and the resulting annotation data is converted into a dataset in Ultralytics YOLO format. In engineering drawings, there is a type of data with a leader line below it (the leader line is drawn with a thin solid line, preferably a straight line at 30°, 45°, 60°, or 90° to the horizontal, or a broken line that bends to the horizontal after passing through the aforementioned angles). This type of data is called leader line annotation data, i.e., leader line markings. Leader line annotation data consists of numbers and numbers in parentheses, as well as a small number of Roman numerals. Not every engineering drawing contains leader line annotation information. In engineering drawings, there's a type of data marked with an arrow (which may be followed by a straight line or a curve, indicating the direction from which to view the drawing). This data is called section data, or section identifier. Section data is usually composed of letters, and not every engineering drawing contains section data. Each section data point in an engineering drawing will have a corresponding sectional view data point on that drawing or a sub-drawing with the same drawing number. Sectional view data generally falls into six categories: A-direction, AA, View A, Part 1, Part Number 1, and III (Roman numerals). The sectional view data corresponds to the section data.

[0056] In one embodiment of this application, the Swin Transformer structure sequentially includes: a first stage, comprising a Patch Embedding unit and multiple Block units, wherein the Patch Embedding unit is used to segment the input first image and embed it into a vector to obtain a first feature map; the Block units are used to perform layer normalization on the first feature map, segment the first feature map into multiple windows, calculate attention and determine whether it is Window Attention or ShiftWindow Attention, merge the windows, if a Shift operation has been performed, perform ReverseShift, perform Dropout and residual connections, and then perform layer normalization, a fully connected layer, Dropout and residual connections; and multiple second stages connected sequentially, each comprising a Patch Merging unit and multiple Block units, wherein the Patch Merging unit is used to perform downsampling, obtain downsampling results, and concatenate the downsampling results to obtain a concatenated result, and adjust the number of channels of the concatenated result through a fully connected layer to obtain a second feature map; the Block units are used to perform layer normalization on the second feature map, segment the second feature map into multiple windows, calculate attention and determine whether it is Window Attention or ShiftWindow Attention. Attention, window merging, if a Shift operation has been performed, ReverseShift, Dropout and residual connections, then layer normalization, fully connected layer, Dropout and residual connections.

[0057] In this embodiment, reference Figure 4a As shown, Figure 4a This is a schematic diagram of the network structure of a Swin Transformer for an identification method in engineering drawings, illustrating an exemplary embodiment of this application. The Swin Transformer structure includes four stages. In the first stage, a PatchEmbedding is performed followed by multiple Blocks. In subsequent stages, each stage consists of a PatchMerging and multiple Blocks. PatchEmbedding cuts the image into patches and embeds them into vectors; preferably, patch_size = 4. PatchMerging performs downsampling to reduce the resolution and adjust the number of channels; preferably, each downsampling is a factor of 2. Therefore, elements are selected at intervals of 2 in both the row and column directions, and then concatenated as a single tensor, which is then expanded. At this point, the channel dimension becomes 4 times the original. A fully connected layer is then used to adjust the channel dimension back to twice its original value. (See reference...) Figure 4b As shown, Figure 4b This is a schematic diagram of a two-stage Swin Transformer Block structure for an identification method for engineering drawings, illustrating an exemplary embodiment of this application. The number of blocks in each stage must be even. First, the feature map is normalized layer by layer. Then, it is determined whether a shift operation is needed, and the feature map is sliced ​​into windows. Attention is calculated, and it is determined whether it is Window Attention or Shift Window Attention. The windows are then merged. If a shift operation was performed previously, ReverseShift is performed. Dropout and residual connections are then implemented, followed by layer normalization, fully connected layers, Dropout, and residual connections. In this process, ... Figure 4b In this context, LN refers to layer normalization, MLP refers to fully connected layers, W-MSA is Window multi-head self-attention, and Dropout and residual connections are implemented at each node. Figure 4b It was done at the point where there is a cross symbol inside a circle.

[0058] In this embodiment, the Swin Transformer structure can also be described as follows. The basic process of the Swin Transformer is as follows: In the first stage, an image [H*W*3][H*W*3][H*W*3] is input. The input image to be detected is segmented through a Patch Partition layer. The segmented data is then processed through a Linear Embedding layer for feature mapping. The feature-mapped data is then input into a Transformer block (SwinTransformer) with improved self-focus computation. In stages 2 to 4, Patch Merging is performed before inputting the data into the model to downsample and generate hierarchical representations. Finally, the data from stage 4 is classified by the output module.

[0059] In this embodiment, the Patch Partition structure divides the image data into non-overlapping M*M patches. Each patch is treated as a "tag," and its features are set as the concatenation of the original pixel RGB values. In this application, a 4x4 patch size is used, so the feature dimension of each patch is 4x4x3 = 48. A linear embedding layer is applied to these original value features, projecting them onto an arbitrary dimension (denoted as C). In practice, Patch Partition and Linear Embedding are implemented through a two-dimensional convolutional layer (the output channel is the Embedding dimension, the convolution kernel size is patch_size, and the stride size is patch_size).

[0060] In this embodiment, the Patch Merging layer primarily performs downsampling to generate a hierarchical representation. As the network deepens, the Patch Merging layer reduces the number of tokens. The first patch merging layer concatenates the features of each 2x2 adjacent patch and applies a linear layer to the concatenated 4C-dimensional features. This reduces the number of tokens by a factor of 2x2 = 4 (a 2x downsampling of resolution, with the length and width each becoming half of their original values), and sets the output dimension to 2C. A SwinTransformer block is then used for feature transformation, maintaining the resolution at h8 x w8. This first block of patch merging and feature transformation is referred to as the "second stage." This process is repeated twice, as "stage 3" and "stage 4," with output resolutions of h16 x w16 and h32 x w32, respectively. As explained above, after passing through the Patch Merging layer, the data's length and width become half of their original values, and the depth becomes twice the original value. Tokens are used to represent the features of the input data. Specifically, when processing visual data such as images, the input image is first divided into several patches. Each patch is then transformed linearly (such as through convolution) and represented as a token. These tokens are then fed into different layers of the Transformer model for processing. In this embodiment, the original feature dimension of each patch is C (i.e., the dimension of each token).

[0061] In this embodiment, the Swin Transformer Block, i.e. the Block unit mentioned above, generally appears in a two-stage serial structure. In the first stage, Window-based Multi-headed Self-Attention (W-MSA) is used, and in the second stage, Shifted Window-based Multi-headed Self-Attention (SW-MSA) is used. Different self-attention calculation methods are selected according to whether the current Swin Transformer Block is odd or even.

[0062] In this embodiment, the Swin Transformer structure can also be described as follows: Patch Partition: The input image is divided into several small patches, each treated as an independent unit for processing, to transform image information into processable features. Linear Embedding: Linear embedding is performed on the patches to transform them into high-dimensional feature vectors. Transformer Encoder: In the Swin Transformer, the Transformer structure is used as the core computational unit. Complex feature interactions and information fusion are achieved through the stacking of multiple Transformer encoders. In each encoder layer, self-attention and positional attention mechanisms are used to capture long-distance dependencies in the image. Shift Window: By introducing the Shift Window mechanism, interaction occurs between adjacent windows, thereby enhancing the model's receptive field. The Shift Window operation shifts the window to the right or down by a certain number of positions, allowing features within different windows to interact. Patch Merging: To increase the model's receptive field and improve feature extraction capabilities, adjacent small patches are merged into larger patches. This reduces the dimensionality of the feature map while preserving important feature information. The purpose of Patch Merging is similar to downsampling in CNN networks, reducing resolution and adjusting the number of channels. Each downsampling process reduces the feature map resolution by a factor of two; therefore, adjacent patches of size 2×2 with a spacing of 2 are selected for merging in both row and column directions. Stage-wise Feature Extraction: The Swin Transformer employs a multi-stage design, with the entire model containing four stages. Except for the first stage, each stage first uses a Patch Merging layer to reduce the resolution of the input feature map, performing downsampling operations and progressively expanding the receptive field to capture global information. Multi-head Self-Attention (MSA): In the Swin Transformer's blocks, there are two types of MSA: Window-based MSA (W-MSA) and Shifted Window-based MSA (SW-MSA).W-MSA focuses on each window, while SW-MSA addresses the issue of W-MSA's inability to capture global information by using a sliding window approach, allowing the model to learn information across windows. The Swin Transformer, as a general-purpose backbone network in computer vision, has achieved excellent performance in tasks such as object classification, object detection, semantic processing, and instance segmentation. Through this structure, the Swin Transformer captures both local and global information in images while maintaining high computational efficiency, providing powerful feature representation capabilities.

[0063] In this embodiment, the improved YOLOv8 detection model uses a Swing Transformer to replace the C2F module in layer 5 of the backbone network, and its network structure is as follows: Figure 4a As shown, the Swin Transformer constructs hierarchical feature maps through a sliding window mechanism, allowing the model to learn information across windows. Simultaneously, through downsampling layers, it enables the model to handle images of various resolutions, reducing computational requirements and allowing the model to focus on local details and overall structural information. The Swin Transformer retains the Transformer's self-attention mechanism, enabling the model to capture long-range dependencies in images. Self-attention is the core of the Transformer, allowing the model to establish weight relationships between different positions in a sequence. The attention mechanism allows the model to focus on specific parts of a sequence when processing it, improving its sensitivity to key information, thus helping the model better understand the text layout and structure in images during text recognition. Given its powerful feature extraction capabilities, the Swin Transformer can effectively capture long-range dependencies in input data and extract rich features, thus handling inputs of any size without complex downsampling operations. Furthermore, through hierarchical design and sliding window operations, the Swin Transformer reduces computational cost and improves computational efficiency, solving the problem of excessive computation for global attention-based Transformers.

[0064] In one embodiment of this application, the base model is a YOLO model in which the CBS structure in the neck is replaced with the GSConv structure; wherein, the GSConv structure is used to perform ordinary convolution downsampling on the input second image to obtain the downsampling result, and to perform depth convolution on the input second image to obtain the depth convolution result; the downsampling result and the depth convolution result are concatenated and a shuffle operation is performed to concatenate the corresponding number of channels of the ordinary convolution and the corresponding number of channels of the depth convolution.

[0065] In this embodiment, the GSConv structure first performs a downsampling operation on the input second image using a regular convolution, then uses a depthwise convolution, and concatenates the results of the two convolutions; finally, a shuffle operation is performed to concatenate the corresponding channel numbers of the two previous convolutions. It should be noted that the neck part of the YOLO series models is responsible for connecting the backbone and head, performing feature fusion and processing. YOLOv8's neck retains the PAN concept and may use other types of structures (such as C2f or Slim-Neck) for feature fusion. The GSConv structure is a design introduced in the neck part of the object detector, designed to optimize feature fusion and processing in convolutional neural networks (CNNs). GSConv is introduced to preserve as many hidden connections between channels as possible while maintaining low time complexity. In the Slim-Neck structure, GSConv may be followed by GS bottleneck and cross-stage partial network (GSCSP) modules to further improve performance.

[0066] In this embodiment, the improved YOLOv8 identifier detection network structure replaces the CBS module in layer 6 of the neck network with a GSConv structure, which improves detection speed while maintaining accuracy. Its network structure is as follows: Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the structure of GSConv, an exemplary embodiment of this application, for a method of identifying markings on engineering drawings. To reduce the number of parameters and computational complexity of the model, making it more lightweight, GSConv incorporates a Slim-Neck design. This is a novel detector architecture design paradigm that maintains the structure of the model's head and backbone, using a channel shuffling operation in the neck to mix information from different channels. This helps to preserve the information interaction between feature maps while reducing computational load, thus enabling better processing of feature maps obtained from the Swin Transformer module.

[0067] This embodiment illustrates the training process of a YOLOv8-based identifier detection model. The dataset is divided into training, validation, and test sets in a 0.7 / 0.15 / 0.15 ratio, as shown in Table 1. Both training and testing of the network model are performed on a server with the following configuration: CPU: Intel(R) Core(TM) i7-9700K / GPU: RTX 2080Ti; operating system: Ubuntu 20.04.2LTS; PyTorch version: 1.12. It should be noted that this embodiment is for illustrative purposes only and does not impose limitations on the dataset division, number of datasets, server configuration, operating system version, etc. These can be modified according to actual needs in practical applications.

[0068] Table 1

[0069] type training set Validation set test set Contains index identifier 2461 525 527 No index identifier 3791 810 810

[0070] As shown in Table 1, the dataset is divided into training, validation, and test sets with indexes, and training, validation, and test sets without indexes. The corresponding data quantities for these sets are referenced in Table 1 above and will not be repeated here. It should be noted that this embodiment is only for illustrative purposes. The classification of the dataset can be increased, decreased, or changed according to actual needs, and its quantity can also be modified according to actual needs. This application does not impose any limitations on this.

[0071] Precision, recall, and the F1 score are used as evaluation criteria for the engineering drawing identification detection model. The Precision-Recall Curve is used to describe the relationship between precision P and recall R of the classifier at different thresholds.

[0072] The formula for calculating accuracy is:

[0073]

[0074] The formula for calculating recall rate is:

[0075]

[0076] The formula for calculating the F1 value is:

[0077]

[0078] In equations (1) to (3), P represents precision; R represents recall; F1 represents the composite index; TP (TruePositive) represents a true positive, that is, the model correctly classifies the sample as a positive example; FP (False Positive) represents a false positive, that is, the model incorrectly classifies the sample as a positive example; FN (False Negative) represents a false negative, that is, the model incorrectly classifies the sample as a negative example.

[0079] The test results are shown in Table 2. Figure 6 The corresponding PR curve is provided. Figure 6 This is a PR curve showing the test results of a method for identifying markings on engineering drawings, as illustrated in an exemplary embodiment of this application.

[0080] Table 2

[0081] Index Identifier Category Accuracy Recall rate F1 value Lead wire marking 97.74％ 99.20％ 0.9846 Section marking 97.06％ 97.06％ 0.9706 Cross-section marking 96.09％ 96.85％ 0.9647

[0082] As shown in Table 2, the test results include precision, recall, and F1 score for lead wire marking, cross-sectional marking, and cross-sectional marking. The corresponding values ​​are referenced in Table 2 above and will not be repeated here. It should be noted that this embodiment is merely illustrative; the test items in the test results can be added, reduced, or changed according to actual needs, and their values ​​should be filled in based on the actual results. This application does not impose any limitations in this regard.

[0083] In one embodiment of this application, the image to be detected is cut according to a first preset size to obtain one or more image segments, and the image segments are indexed and identified using an index identification detection model.

[0084] In this embodiment, a sub-image segmentation method is used to cut sub-images of a first preset size (e.g., a fixed resolution of 1280*720) from the original drawing to serve as the dataset for detection index identification. The dataset in this embodiment originates from dedicated engineering drawings provided by enterprises, and these drawings undergo format conversion, such as from PDF to PNG. Since the number of engineering drawings is small and their sizes vary, they cannot be directly used as the training set; preprocessing is required. Preprocessing includes, but is not limited to, at least one of the following: format conversion, drawing segmentation, geometric transformation, edge detection, and global threshold binarization. Because engineering drawings have high resolutions, some reaching tens of thousands of pixels, drawing segmentation is necessary to use the dataset for training, for example, segmenting them into 1280*720 resolution images, and using these images as the basis for the dataset. Redundancy is left between each image segmentation to prevent damage to text information at the segmentation lines, thereby increasing the number of labeled samples. To enhance the recognition ability for rotated text, geometric transformations, such as rotation, flipping, scaling, and translation, are performed on the drawings in the training set. These operations change the spatial structure of the image but do not alter its content. Simultaneously, edge information is extracted through edge detection methods such as gradient calculation or Laplacian operation, which helps preserve important structural information of the image while denoising. Since engineering drawings only contain black and white, the resulting dataset after cutting in high-resolution drawings has low grayscale values, leading to poor training results. To make the outlines and shapes of target objects in the drawings more distinct, global threshold binarization is required. A single threshold is selected to process the entire image; all pixels above this threshold are set to 1, and all pixels below the threshold are set to 0. This operation separates text from the background for subsequent text recognition. The main purposes of preprocessing include: 1. Increasing the size of the training set; 2. Standardizing the size of the training set; 3. Improving the quality of the training set.

[0085] In one embodiment of this application, a sliding window is determined from an image segment; the sliding window is controlled to slide within the image segment, and index identifier detection is performed on the sliding window to obtain at least one original detection box's position information in the sub-image corresponding to the image segment and the confidence level of the index identifier corresponding to the original detection box, wherein the original detection box contains an index identifier; non-maximum suppression is performed when integrating the image segment based on the offset position of the image to be detected; the original image position information of the original detection box in the image to be detected is determined according to the sub-image position information of the original detection box in the sub-image corresponding to the image segment; any original detection box is determined as the first detection box, and the remaining detection boxes are... The original detection boxes outside the first detection box are respectively identified as the second detection boxes. The first area intersection-union ratio is calculated based on the intersection of the first area and the second area, and the union of the first area and the second area. The first area is obtained based on the original image position information of the first detection box, and the second area is obtained based on the original image position information of the second detection box. If the first area intersection-union ratio is greater than the preset first area threshold, the index identifier confidence of the first detection box and the index identifier confidence of the second detection box are compared to determine the target detection box with the highest index identifier confidence from the first detection box and the second detection box, and the original image position information corresponding to the target detection box is used as the index identifier position.

[0086] In this embodiment, each time the control window slides horizontally or vertically, the horizontal and vertical slides represent half the width (640) and half the height (360) of the sub-image, respectively. In the detection results, the same identifier on a single drawing may be detected multiple times. Finally, when integrating the sub-images based on the offset position of the original image, non-maximum suppression is applied, and the area intersection-union ratio (IoU) between the detection boxes is calculated. If it exceeds a preset first area threshold (e.g., 0.6), it is determined to be the same identifier, and the one with higher confidence is retained; that is, the detection box with higher confidence is determined as the index identifier. It should be noted that the sliding size of the sliding window is a preset size. The half-width and half-height of the sliding sub-image in this embodiment are only illustrative and can be changed according to actual needs.

[0087] In step S230, all identifiers in the image to be detected are detected to obtain the first identifier position corresponding to the identifier to be identified, and secondary identifiers in the image to be detected are detected to obtain the second identifier position corresponding to the secondary identifier. The size identifier position is obtained by extracting from the first identifier position through the index identifier position and the second identifier position.

[0088] The identifiers to be identified include size identifiers, index identifiers, and secondary identifiers.

[0089] In one embodiment of this application, the image to be detected is scaled down proportionally to a second preset size with its long side as the size, and the scaling ratio is recorded. The adjusted image to be detected is input into a preset secondary identifier detection model to obtain the secondary identifier position. The secondary identifier detection model is trained based on identifier samples with secondary identifier labels, and the secondary identifier position is the position of the secondary identifier in the adjusted image to be detected. The second identifier position is calculated based on the secondary identifier position and the scaling ratio.

[0090] In this embodiment, the drawing image to be inspected is first scaled down proportionally, shortening the longer side to a second preset size, for example, a resolution of 640. The scaling ratio is recorded and then input into the model to obtain the location of non-size text information in the scaled-down image, i.e., the secondary identifier position. Finally, the image is restored to its original size position according to the scaling ratio. Areas in the drawing containing text that are not size identifiers (excluding the aforementioned index identifiers, such as…) Figure 7 As shown, Figure 7 This is a schematic diagram of a non-dimensional identifier area for an engineering drawing identifier recognition method, as illustrated in an exemplary embodiment of this application. The non-dimensional identifier areas mainly include various tables in the drawing (including parts lists, drawing information tables, responsible personnel lists, etc.) and textual descriptions such as technical specifications of the drawing content. YOLOv8 is used to classify and detect these non-dimensional targets. Unlike the index identifier detection training described above, where index identifiers are typically small and can be fully contained within a certain size window, the bounding boxes of tables and textual descriptions in the drawing are larger and difficult to contain with a fixed-size window. Therefore, the drawing is first scaled down to a preset size (e.g., 640*640 resolution) before being used as the dataset. During inference, the scaling ratio is recorded first, and then the non-dimensional target position information output by the model is restored to its original drawing size based on this ratio. Furthermore, because the non-dimensional targets are sufficiently large, it has been verified that even after the drawing is scaled down, it still retains enough key features for the model to learn and recognize.

[0091] In this embodiment, the general text detection function in PaddleOCR is used to detect and locate all text information in the drawing image to obtain the first identifier position corresponding to the identifier to be identified. A pre-trained YOLOv8 detection model, i.e., the secondary identifier detection model, is used to perform non-dimensional identifier detection on the secondary identifiers in the engineering drawing to obtain the second identifier position corresponding to the secondary identifier. The pre-trained YOLOv8 detection model is called the non-dimensional text information detection model. Secondary identifiers include non-dimensional identifiers, which are identifiers in the drawing that contain text but do not belong to the dimension identifier area. Engineering drawings contain a large amount of part dimension data, which is numerous, randomly located, and in various formats. Manually marking the dimension data would consume a lot of time and manpower. Using the general text detection model provided by PaddleOCR to detect and recognize all text in the engineering drawing and exclude identifier data and other irrelevant text, thereby filtering out the dimension data, not only saves time and manpower costs but also improves the accuracy of detection and recognition.

[0092] In one embodiment of this application, the area corresponding to the first identifier position is determined as the first area, the area corresponding to the index identifier position is determined as the index area, and the area corresponding to the second identifier position is determined as the second area. A second area intersection-union ratio is calculated based on the intersection of the first area and the index area, and the union of the first area and the index area. If the second area intersection-union ratio is greater than a first preset threshold, the first identifier position and the index identifier position are determined to be the same position. The intersection area of ​​the first area and the second area is calculated, and the area ratio is calculated based on the intersection area and the second area. If the area ratio is greater than a second preset threshold, the first identifier position is determined to be located inside the second identifier position. The index identifier position and the second identifier position are filtered from the first identifier position to obtain the size identifier position.

[0093] In this embodiment, the detected text information location (i.e., the first identifier location) is compared with the detected index identifier location and the non-size text location (i.e., the second identifier location). The relevant area corresponding to each text information location with the index identifier location and the non-size text location is calculated and filtered to obtain the size identifier location. When comparing with the index identifier, the intersection-union ratio (IoU) of the candidate box areas of the two is calculated to determine whether the text candidate box represents the same identifier location as the index candidate box. If it is greater than a first preset threshold (e.g., 0.6), it is determined to be the same identifier, and the candidate box is discarded. When comparing with non-size text information, the ratio of the intersection area of ​​the two to the area of ​​the non-size text information is calculated. If it is greater than a second preset threshold (e.g., 0.9), it is determined that the text candidate box belongs to the non-size region, and the candidate box is discarded. The remaining candidate boxes are identified as size identifiers.

[0094] In this embodiment, the filtering strategy uses the candidate size sample set as all text boxes detected by PaddleOCR, and the negative samples include index boxes, table boxes, and text description boxes. For each candidate box, a second area intersection-union ratio r1 needs to be calculated with the boxes in the negative samples, and the calculation method is referenced in equation (4):

[0095]

[0096] In equation (4), r1 is the second area intersection-union ratio, S intersection is the intersection of the area of ​​the text box and the box in the negative sample, that is, the intersection of the area of ​​the first region and the area of ​​the index region, and S union is the union of the area of ​​the text box and the box in the negative sample, that is, the union of the area of ​​the first region and the area of ​​the index region.

[0097] If r1 exceeds the first preset threshold (e.g., 0.6), the candidate box is determined to be a non-size identifier and should be removed. Furthermore, due to the large size of the table and text description areas, some candidate boxes may be included within them, such as... Figure 8 As shown within the gray box area, Figure 8 This is an exemplary embodiment of the present application illustrating a non-dimensional labeling example of a labeling and identification method for engineering drawings. In this case, it is also necessary to calculate the ratio of the area where the two intersect to the area of ​​the candidate frame, and the calculation method is referenced in formula (5):

[0098]

[0099] In equation (5), r2 is the ratio of the intersection area to the area of ​​the region corresponding to the second identifier position, S intersection is the intersection of the areas of the text box and the candidate box, that is, the intersection area of ​​the area of ​​the first region and the area of ​​the second region, and S candidate box is the area of ​​the candidate box, that is, the area of ​​the second region.

[0100] If r2 exceeds the second preset threshold (e.g., 0.9), the candidate box is determined to be contained within a non-size target and is also considered a non-size identifier, and should be removed. The candidate boxes that are ultimately retained are the size identifiers.

[0101] In step S240, the index identifier position and / or dimension identifier position are identified to obtain the identification result corresponding to the engineering drawing.

[0102] In one embodiment of this application, the index identifier position and the size identifier position are integrated to obtain an integrated result, wherein the integrated result includes the identifier position and the identifier category; the image to be detected is cropped according to the identifier position to obtain a cropped image that only includes the identifier content; the cropped image is subjected to identifier recognition to obtain the recognition result corresponding to the engineering drawing.

[0103] In this embodiment, PaddleOCR is used to perform text recognition on all index and size marker regions. The detected index and size markers are integrated and output as a JSON file, which includes information such as filename, marker location, and marker category. The marker location is represented by four point coordinates, each representing one of the four vertices of the marker detection box. The general text recognition tool provided by PaddleOCR is used to recognize the marker content. Based on the marker locations in the JSON file, an image containing only the marker content is cropped from the drawing. The PP-OCRv4 model provided by PaddleOCR is used to recognize the text content of the cropped image, and the recognition result is written to the JSON file.

[0104] Figure 9 This is a schematic flowchart illustrating a method for identifying markings on engineering drawings, as shown in another exemplary embodiment of this application. (Reference) Figure 9 As shown, in an exemplary embodiment, the original image of the engineering drawing, i.e., the image to be detected, is segmented into sub-images and input into an index labeling model. After position restoration and non-maximum suppression, the index label positions are obtained. The original image of the engineering drawing, i.e., the image to be detected, is input into a general text detection model to obtain the text positions, i.e., the first label positions. The original image of the engineering drawing, i.e., the image to be detected, is input into a non-size detection model, i.e., a secondary label detection model, to obtain the non-size positions, i.e., the second label positions. The index label positions and non-size positions are extracted from the text positions and filtered to obtain the size label positions. The PaddleOCR general text recognition model is used to identify the index label positions and size label positions to obtain the key label content text information. The specific implementation of this embodiment has been described in detail in the foregoing embodiments and will not be repeated here.

[0105] This application first locates and identifies relatively distinctive index markers, including one or more of leader markers, section markers, and cross-sectional markers. The index marker detection model is an improved YOLOv8 model, which employs a hierarchical design to extract features in the backbone network and uses depthwise separable convolutions to fuse features. Next, all text information in the original drawing (i.e., the image to be detected) is located. Then, dimension markers are selected from the text candidate boxes by calculating the area intersection-union ratio of the candidate boxes. Finally, the PaddleOCR general text recognition model is used to perform character recognition on all index marker and dimension marker regions. This provides a high-precision and high-recall method for detecting and recognizing engineering drawing serial numbers.

[0106] This application improves the YOLOv8 network by leveraging the characteristic that index markers in engineering drawings are surrounded by clear markings. By employing a Swing Transformer module in the backbone network, the model learns cross-window information without reducing computational speed. Furthermore, by introducing the depthwise separable convolutional module GSConv, this application ensures that the semantic information of the engineering drawing feature maps is not lost due to spatial compression and channel expansion.

[0107] This application identifies index markers and dimension markers separately. By first identifying the more distinctive index markers, it can accurately identify them among dimension markers with diverse symbols. When calculating dimension markers, after performing detailed text recognition on the entire drawing, the intersection-union ratio of candidate box areas is calculated between the recognition results and non-dimensional markers as well as the index markers, eliminating obviously erroneous recognition results and thus reducing misidentification. In this application, redundancy processing is performed when cropping the original drawing, increasing the sample size and improving detection accuracy when the number of drawings is limited.

[0108] Figure 10 This is a block diagram illustrating an identification device for engineering drawings, as shown in an exemplary embodiment of this application. The device can be applied to... Figure 1 The implementation environment shown is specifically configured in computer device 102. This device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0109] like Figure 10 As shown, the exemplary identification device for engineering drawings includes: an acquisition module 1010, an index identification module 1020, a dimension identification module 1030, and an identification module 1040.

[0110] The system includes: an acquisition module 1010 for acquiring an image to be detected, which includes an engineering drawing; an index identifier module 1020 for detecting index identifiers in the image to be detected using a preset index identifier detection model to obtain the index identifier positions, wherein the index identifier detection model is trained based on a base model, which is a YOLO model with the C2F structure in the backbone replaced by a Swin Transformer structure; a size identifier module 1030 for detecting all identifiers in the image to be detected to obtain the first identifier position corresponding to the identifier to be identified, and detecting secondary identifiers in the image to be detected to obtain the second identifier position corresponding to the secondary identifier, and extracting the size identifier position from the first identifier position using the index identifier position and the second identifier position, wherein the identifiers to be identified include size identifiers, index identifiers, and secondary identifiers; and a recognition module 1040 for identifying the index identifier position and / or the size identifier position to obtain the recognition result corresponding to the engineering drawing.

[0111] It should be noted that the identification and recognition device for engineering drawings provided in the above embodiments and the identification and recognition method for engineering drawings provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the identification and recognition device for engineering drawings provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0112] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the identification method for engineering drawings provided in the above embodiments.

[0113] Figure 11 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 11 The computer system 1100 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0114] like Figure 11As shown, the computer system 1100 includes a Central Processing Unit (CPU) 1101, which can perform various appropriate actions and processes, such as executing the methods provided in the various embodiments described above, based on a program stored in Read-Only Memory (ROM) 1102 or a program loaded from Storage Unit 1108 into Random Access Memory (RAM) 1103. The RAM 1103 also stores various programs and data required for system operation. The CPU 1101, ROM 1102, and RAM 1103 are interconnected via a bus 1104. An Input / Output (I / O) interface 1105 is also connected to the bus 1104.

[0115] The following components are connected to I / O interface 1105: an input section 1106 including a keyboard, mouse, etc.; an output section 1107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to I / O interface 1105 as needed. Removable media 1111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1110 as needed so that computer programs read from them can be installed into storage section 1108 as needed.

[0116] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1109, and / or installed from removable medium 1111. When the computer program is executed by central processing unit (CPU) 1101, it performs various functions defined in the system of this application.

[0117] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0119] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0120] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the identification method for engineering drawings as provided in the various embodiments above. This computer-readable storage medium may be included in the electronic devices described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0121] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0122] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the identification method for engineering drawings provided in the various embodiments described above.

[0123] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0124] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0125] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for identifying markings on engineering drawings, characterized in that, include: Acquire an image to be detected, wherein the image to be detected includes engineering drawings; The image to be detected is subjected to index identification detection by a preset index identification detection model to obtain the index identification position. The index identification detection model is trained based on an improved YOLOv8 model. The improved YOLOv8 model is a YOLO model that replaces the C2F structure in the backbone with a Swin Transformer structure and replaces the CBS structure in the neck with a GSConv structure. All identifiers in the image to be detected are detected to obtain the first identifier position corresponding to the identifier to be identified. Secondary identifiers in the image to be detected are detected to obtain the second identifier position corresponding to the secondary identifier. The size identifier position is extracted from the first identifier position by the index identifier position and the second identifier position. The identifier to be identified includes size identifiers, index identifiers and secondary identifiers. The index identifiers include one or more of the following: leader line identifiers, section surface identifiers and sectional view identifiers. The secondary identifiers include non-size identifiers, which are identifiers in the drawing that contain text but do not belong to the size identifier area. The index identifier position and the dimension identifier position are identified to obtain the identification result corresponding to the engineering drawing; Detecting secondary identifiers in the image to be detected to obtain the second identifier position corresponding to the secondary identifier includes: The image to be detected is scaled down proportionally to a length side that is a second preset size, and the scaling down ratio is recorded. The adjusted image to be detected is input into a preset secondary identifier detection model to obtain the secondary identifier position. The secondary identifier detection model is trained based on identifier samples with secondary identifier labels. The secondary identifier position is the location of the secondary identifier in the adjusted image to be detected. The second identifier position is calculated based on the secondary identifier position and the reduction ratio.

2. The identification method for engineering drawings according to claim 1, characterized in that, The image to be detected is subjected to index identifier detection using a preset index identifier detection model, including: The image to be detected is segmented to obtain one or more image segments, and the index identifier detection model is used to detect the index identifiers of the image segments.

3. The identification method for engineering drawings according to claim 2, characterized in that, The method further includes: Determine a sliding window from the image segment; The sliding window is controlled to slide in the image segment, and index identifier detection is performed on the sliding window to obtain at least one original detection box position information in the sub-image corresponding to the image segment and the index identifier confidence score of the original detection box, wherein the original detection box contains an index identifier; Non-maximum suppression is performed when integrating the image segments based on the offset position of the image to be detected; The original detection box position information in the original image corresponding to the image to be detected is determined based on the sub-image position information of the original detection box in the image segment; Any original detection box is designated as the first detection box, and all original detection boxes other than the first detection box are designated as the second detection boxes. The intersection-union ratio of the first area is calculated based on the intersection of the first area and the second area, and the union of the first area and the second area. The first area is obtained based on the original image position information of the first detection box, and the second area is obtained based on the original image position information of the second detection box. If the first area intersection-union ratio is greater than the preset first area threshold, the index identifier confidence of the first detection box and the index identifier confidence of the second detection box are compared to determine the target detection box with the highest index identifier confidence from the first detection box and the second detection box, and the original image position information corresponding to the target detection box is used as the index identifier position.

4. The identification method for engineering drawings according to any one of claims 1 to 3, characterized in that, The size identifier position is obtained by extracting it from the first identifier position using the index identifier position and the second identifier position, including: The area corresponding to the first identifier position is determined as the first area, the area corresponding to the index identifier position is determined as the index area, and the area corresponding to the second identifier position is determined as the second area. The second area intersection-union ratio is calculated based on the intersection of the area of ​​the first region and the area of ​​the index region, and the union of the area of ​​the first region and the area of ​​the index region. If the second area intersection-union ratio is greater than the first preset threshold, the first identifier position and the index identifier position are determined to be the same position. Calculate the intersection area of ​​the first region area and the second region area, and calculate the area ratio based on the intersection area and the second region area. If the area ratio is greater than a second preset threshold, it is determined that the first marker position is located inside the second marker position. The index identifier position and the second identifier position are filtered from the first identifier position to obtain the size identifier position.

5. The identification method for engineering drawings according to any one of claims 1 to 3, characterized in that, The index identifier position and the dimension identifier position are identified to obtain the identification result corresponding to the engineering drawing, including: The index identifier position and the size identifier position are integrated to obtain an integration result, wherein the integration result includes identifier position and identifier category; Cropping is performed from the image to be detected based on the location of the marker to obtain a cropped image that includes only the marker content; The cropped image is identified to obtain the recognition result corresponding to the engineering drawing.

6. The identification method for engineering drawings according to any one of claims 1 to 3, characterized in that, The Swin Transformer structure includes, in sequence: The first stage includes a Patch Embedding unit and multiple Block units. The Patch Embedding unit is used to segment the input first image and embed it into a vector to obtain a feature map. The Block unit is used to perform layer normalization on the feature map, segment the feature map into multiple windows, calculate attention and determine whether it is Window Attention or Shift Window Attention, merge the windows, if a Shift operation has been performed, perform ReverseShift, perform Dropout and residual connections, and then perform layer normalization, fully connected layers, Dropout and residual connections. Multiple second stages are connected in sequence. Each second stage includes a Patch Merging unit and multiple Block units. The Patch Merging unit is used to perform downsampling to obtain downsampling results, and then splices the downsampling results to obtain spliced ​​results. The number of channels in the spliced ​​results is adjusted through a fully connected layer.

7. The identification method for engineering drawings according to any one of claims 1 to 3, characterized in that, The GSConv structure is used to perform ordinary convolution downsampling on the input second image to obtain the downsampling result, and to perform depth convolution on the input second image to obtain the depth convolution result; The downsampling result and the depthwise convolution result are concatenated and then shuffled.

8. A marking and identification device for engineering drawings, characterized in that, include: An acquisition module is used to acquire an image to be detected, wherein the image to be detected includes engineering drawings; The index identification module is used to detect the index identification in the image to be detected by a preset index identification detection model to obtain the index identification position. The index identification detection model is trained based on an improved YOLOv8 model. The improved YOLOv8 model is a YOLO model that replaces the C2F structure in the backbone with a Swin Transformer structure and replaces the CBS structure in the neck with a GSConv structure. The size identification module is used to detect all identifications in the image to be detected, obtain the first identification position corresponding to the identification to be identified, and detect secondary identifications in the image to be detected, obtain the second identification position corresponding to the secondary identification. The size identification position is extracted from the first identification position by the index identification position and the second identification position. The identification to be identified includes size identifications, index identifications, and secondary identifications. The index identifications include one or more of the following: leader line identifications, section identifications, and sectional view identifications. The secondary identifications include non-size identifications, which are identifications in the drawing that contain text but do not belong to the size identification area. Specifically, the module is used to proportionally reduce the image to be detected to a second preset size with its long side as the size, record the reduction ratio, and input the adjusted image to be detected into a preset secondary identification detection model to obtain the secondary identification position. The secondary identification detection model is trained based on identification samples with secondary identification labels. The secondary identification position is the position of the secondary identification in the adjusted image to be detected. The second identification position is calculated based on the secondary identification position and the reduction ratio. The identification module is used to identify the index mark position and the dimension mark position to obtain the identification result corresponding to the engineering drawing.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the identification method for engineering drawings as described in any one of claims 1 to 7.

Images