An engineering drawing serial number detection method based on an inhibiting function

Abstract

The present application relates to a kind of engineering drawing serial number detection method based on inhibition function, belong to text detection field.The method specifically includes the following steps: S1: self-made engineering drawing data set, network model training;S2: the pre-processing of input engineering drawing;S3: using convolutional neural network to the feature extraction of input picture, different scale features are fused using adaptive scale fusion module;S4: the probability graph and inhibition graph obtained by prediction;S5: obtain the final prediction graph by inhibition function;S6: post-processing obtains text box coordinate, collates result.The problem that engineering drawing is too large to directly predict is solved, and the coordinates of serial number text in the whole engineering drawing are extracted by post-processing process.The present application can effectively help staff to quickly detect serial number in engineering drawing and complete the collation and display of result.

Description

Technical Field

[0001] This invention belongs to the field of text detection and relates to a method for detecting engineering drawing serial numbers based on a suppression function. Background Technology

[0002] Using computers to quickly and accurately identify numbers on drawings can not only play an important role in the digitization of old drawings, but also help employees quickly perform information statistics and verification during the production process, thereby improving work efficiency and reducing labor costs.

[0003] Optical Character Recognition (OCR) is an indispensable part of automated business systems. It involves analyzing and processing scanned document images to detect and recognize text information within them. While maturely applied in various service scenarios to improve productivity, such as bank drafts, invoices, and document analysis, its application in engineering drawings is relatively limited. Conventional detection and recognition algorithms are not suitable for characters on drawings, and open-source OCR software can only perform line-by-line text recognition for simple text content like articles, exhibiting poor robustness and significant room for improvement in text recognition efficiency. With the development of deep learning, more and more deep learning-based methods are being applied to various sub-tasks of OCR (text detection, text recognition, table detection, formula detection, etc.) and have achieved better results than traditional methods. However, in the structured recognition of engineering drawings, deep learning-based methods cannot effectively detect specified text content. Furthermore, the lack of public datasets for engineering drawing digit recognition tasks makes data acquisition and annotation time-consuming and labor-intensive for small and medium-sized enterprises, resulting in high training and deployment costs for recognition systems. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method for detecting engineering drawing serial numbers based on a suppression function. The limitations of this method include: 1) a lack of engineering drawing text-related datasets; 2) inability to effectively detect specified text content; and 3) inability to automatically detect and integrate results.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for detecting engineering drawing serial numbers based on a suppression function, the method specifically includes the following steps:

[0007] S1: Self-made engineering drawing dataset for network model training;

[0008] S2: Preprocess the input engineering drawings;

[0009] S3: Use a convolutional neural network to extract features from the input image, and use an adaptive scale fusion module to fuse features at different scales;

[0010] S4: Predict the probability map and suppression map;

[0011] S5: Obtain the final prediction map through the suppression function;

[0012] S6: Post-processing to obtain the coordinates of the text boxes and organizing the results.

[0013] Optionally, S1 specifically involves: organizing the training images of engineering drawings and uniformly labeling the serial number text to obtain a self-made engineering drawing dataset; during training, first using a general text detection dataset for training, and then using the self-made dataset for adjustment to obtain the network model required for prediction.

[0014] Optionally, S2 specifically involves: performing noise reduction and sharpening preprocessing on the engineering drawing to remove noise from the engineering drawing; segmenting the input engineering drawing, ensuring that the image to be segmented overlaps with the images on the top, bottom, left, and right sides during the segmentation process to ensure that no detection is missed; and then sequentially sending the multiple segmented images into the detection network for detection.

[0015] Optionally, S3 specifically involves: inputting an engineering drawing into a convolutional neural network, and obtaining feature maps of different scales after multiple convolutions and pooling; the feature maps of different scales contain information from different receptive fields of the original image, and using an adaptive scale fusion module to dynamically fuse feature maps of different scales;

[0016] After training, the adaptive scale fusion module predicts the weights of feature maps at each scale in terms of channels and space. It is an attention mechanism that combines channels and space, as shown below:

[0017] F=concat(W0*X0, W1*X1,...,W n-1 *X n-1 )

[0018] Wherein, concat means concatenating the feature maps along the channel dimension; {X 0, X1……, X n-1} represents a feature map of different scales that has been upsampled to a uniform size using different ratios; {W 0, W1……, W n-1} represents a weight map of the same size as the feature map, which is predicted by the network and contains channel and spatial weights; F represents the total feature map obtained by stitching together the individual feature maps after weight adjustment, which is used for subsequent prediction.

[0019] Optionally, S4 specifically involves: designing two prediction branches based on the total feature map, which are used to predict the probability map and the suppression map, respectively;

[0020] The probability map is the same size as the original image, and the value at each position represents the probability that the pixel at that position belongs to the serial number text, ranging from 0 to 1;

[0021] The suppression map is the same size as the original image. Each position in the suppression map represents the intensity of suppression required for the pixel at that position, ranging from 0 to 1. For the serial number text area, the suppression intensity is 0, and for non-serial number text and non-text areas, the suppression intensity is 1. For the area surrounding the serial number text, the suppression intensity is a non-linear transition value of 0 to 1, with the intensity being greater the closer it is to the serial number text area and less the further away it is from the text area.

[0022] Optionally, S5 specifically includes:

[0023] Propose a dynamic suppression function:

[0024] β=k d *(1-A (i，j) );k e ∈(0, ∞)

[0025]

[0026]

[0027] Where k d Indicates the regulatory factor during inhibition; A (i，j) E represents the modulation strength at position (i, j) in the inhibition graph; β represents the final inhibition strength in the inhibition graph after calculation; (i，j) This represents the probability of suppressing the text at position (i, j) in the graph; C is a variable in the calculation process used to ensure that the output range is from 0 to 1; E (i，j) This represents the text probability at position (i, j) in the final prediction graph.

[0028] The final prediction map is obtained by dynamically suppressing the probability map through the adjustment map. The stronger the adjustment, the more severe the suppression, and the closer the probability value is to 0. Regions with an adjustment strength of 0 are not suppressed. After dynamic suppression, the probability values ​​of incorrectly predicted non-sequence number text regions and non-text regions will be suppressed, while the correctly predicted sequence number regions will not be affected. In this way, the final prediction map is closer to the true result.

[0029] The final predicted image is processed using standard binarization to obtain a binary image, as follows:

[0030]

[0031] Among them B (i，j) P represents the value at position (i, j) in the binary graph. (i，j) This represents the value at position (i, j) in the prediction graph; t represents the pre-set threshold, which is set to 0.3.

[0032] Use the findcontours method in OpenCV to obtain the text box coordinates of all serial number texts in the image from the obtained binary image.

[0033] Optionally, S6 specifically includes:

[0034] For the final predicted image, the coordinates of the segmented image are offset by a corresponding distance based on its position in the original image, and then saved uniformly to the final result set. For all coordinates in the final result set, a non-maximum suppression algorithm is used to filter them, removing duplicate and similar text boxes, and obtaining the coordinates of all serial number text boxes in the entire engineering drawing.

[0035] The beneficial effects of this invention are as follows: This invention provides a method for detecting engineering drawing serial numbers based on an inhibition function. The method creates its own engineering drawing serial number dataset and uses it for training a detection network, solving the current problem of insufficient engineering drawings. Leveraging the advantages of deep learning in object detection, instance segmentation, etc., it proposes an adaptive scale fusion module and a dynamic inhibition module, combining them with state-of-the-art methods to improve text detection accuracy and achieve the function of detecting serial numbers in engineering drawings. The proposed process of segmenting the engineering drawing before sequentially feeding it into the detection network for text detection solves the problem of not being able to directly predict large engineering drawings. Furthermore, a post-processing workflow extracts the coordinates of the serial number text within the entire engineering drawing. This invention can effectively help staff quickly detect serial numbers in engineering drawings and complete the organization and display of the results.

[0036] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0037] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0038] Figure 1 This is a flowchart of the present invention;

[0039] Figure 2The diagram below is a schematic diagram of an embodiment of the present invention; (a) is an input image; (b) is a probability graph; (c) is an adjustment graph; and (d) is a final prediction graph. Detailed Implementation

[0040] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0041] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0042] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0043] like Figure 1 As shown, the present invention provides an overall flowchart of an engineering drawing serial number detection method based on a suppression function, which may specifically include the following steps:

[0044] S1: Self-made dataset, network model training:

[0045] A self-made engineering drawing dataset was created. Due to the lack of publicly available engineering drawing datasets, training images of engineering drawings needed to be compiled independently, and the serial number text was uniformly labeled to obtain a self-made engineering drawing dataset. During training, common text detection datasets (such as icdar2015, ctw1500, etc.) were first used, followed by adjustments using the self-made dataset, ultimately resulting in the network model required for prediction.

[0046] S2: Preprocess the input engineering drawings:

[0047] In step S2, the engineering drawings are first preprocessed, including noise reduction and sharpening, to remove noise and facilitate subsequent detection. Secondly, since the input engineering drawings are generally large and cannot be directly detected, they need to be segmented. During segmentation, it is ensured that the image to be segmented overlaps with the images on all four sides to prevent missed detections. The segmented images are then sequentially fed into the detection network for testing.

[0048] S3: Use a convolutional neural network to extract features from the input image, and use an adaptive scale fusion module to fuse features at different scales:

[0049] A convolutional neural network (CNN) is a deep learning neural network with a convolutional structure. Through training, it can extract desired features from images for prediction. The input engineering drawing undergoes multiple convolutions and pooling to obtain feature maps of different scales. These feature maps at different scales contain information from different receptive fields of the original image; therefore, an adaptive scale fusion module is needed to dynamically fuse feature maps of different scales to obtain better detection results.

[0050] After training, the adaptive scale fusion module can predict the weight values ​​of feature maps at various scales in both channels and space. It is an attention mechanism that combines channels and space, and can be represented as follows:

[0051] F=concat(W0*X0, W1*X1,...,W n-1 *X n-1 )

[0052] Wherein, concat means concatenating the feature maps along the channel dimension; {X 0, X1……, X n-1} represents a feature map of different scales that has been upsampled to a uniform size using different ratios; {W 0, W1……, W n-1} represents a weight map of the same size as the feature map, which is predicted by the network and contains channel and spatial weights; F represents the total feature map obtained by stitching together the individual feature maps after weight adjustment, which is used for subsequent prediction.

[0053] S4: Predicting the probability map and suppression map:

[0054] Based on the total feature map, two prediction branches are designed to predict the probability map and the suppression map, respectively.

[0055] Probability graph. The probability graph is the same size as the original image. The value at each position represents the probability that the pixel at that position belongs to the serial number text, ranging from 0 to 1. The probability graph can predict the serial number text region relatively accurately, but it may also incorrectly predict some non-serial number text and non-text regions as serial number text regions.

[0056] Suppression Map. The suppression map is the same size as the original image, but unlike the probability map, each position in the suppression map represents the intensity of suppression required for that pixel, ranging from 0 to 1. For the serial number text region, the suppression intensity is 0; for non-serial number text and non-text regions, the suppression intensity is 1. For the region surrounding the serial number text, the suppression intensity is a transitional value between 0 and 1, with the intensity increasing closer to the serial number text region and decreasing further away.

[0057] S5: Obtain the final prediction map using the suppression function:

[0058] To obtain a clearer prediction map, a dynamic suppression function is proposed:

[0059] β=k d *(1-A (i，j) ): k e ∈(0, ∞)

[0060]

[0061]

[0062] Where k d Indicates the regulatory factor during inhibition; A (i，j) This represents the modulation strength at position (i, j) in the inhibition graph; β represents the final inhibition strength in the inhibition graph after calculation; E (i，j) This represents the probability of suppressing the text at position (i, j) in the graph; C is a variable in the calculation process used to ensure that the output range is 0 to 1; E (i，j) This represents the text probability at position (i, j) in the final prediction graph.

[0063] As the formula shows, the final predicted map is obtained by dynamically suppressing the probability map through the adjustment map. Regions with stronger adjustment are subject to more severe suppression, resulting in probability values ​​closer to 0, while regions with zero adjustment are not suppressed. After dynamic suppression, the probability values ​​of incorrectly predicted non-sequence number text regions and non-text regions are suppressed, while correctly predicted sequence number regions remain unaffected. This results in a final predicted map that is closer to the true result.

[0064] S6: Post-processing to obtain the text box coordinates, and then refining the results:

[0065] The final predicted image is processed using standard binarization to obtain a binary image, as follows:

[0066]

[0067] Among them B (i，j) P represents the value at position (i, j) in the binary graph. (i，j) This represents the value at position (i, j) in the prediction graph; t represents the pre-set threshold, which is set to 0.3.

[0068] Use the findcontours method in OpenCV to obtain the text box coordinates of all serial number texts in the obtained binary image.

[0069] Results Processing. Since S2 segments the engineering drawing into multiple images and then sequentially feeds them into the prediction network, for each image's prediction result, the coordinates of the segmented image are first offset according to its position in the original image, and then saved uniformly into the final result set. For all coordinates in the final result set, a non-maximum suppression algorithm is used to filter them, removing duplicate and similar text boxes, thus obtaining the coordinates of all sequence number text boxes in the entire engineering drawing. Finally, all the obtained text box coordinates are plotted on the original engineering drawing to obtain the final detection result image.

[0070] Based on a self-made engineering drawing sequence number dataset, this method leverages the advantages of deep learning in object detection, instance segmentation, and other areas. It proposes an adaptive scale fusion module and a dynamic suppression module, combining these with state-of-the-art methods to improve the accuracy of text detection and recognition, thus enabling text detection in engineering drawings. This invention effectively helps staff quickly detect sequence numbers in engineering drawings and organize and display the results.

[0071] Figure 2 The diagram below is a schematic diagram of an embodiment of the present invention; (a) is an input image; (b) is a probability graph; (c) is an adjustment graph; and (d) is a final prediction graph.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting engineering drawing serial numbers based on a suppression function, characterized in that: The method specifically includes the following steps: S1: Self-made engineering drawing dataset for network model training; S2: Preprocess the input engineering drawings; S3: Use a convolutional neural network to extract features from the input image, and use an adaptive scale fusion module to fuse features at different scales; S4: Predict the probability map and suppression map; S5: Obtain the final prediction map through the suppression function; specifically: Propose a dynamic suppression function: Where k d Indicates the regulatory factor during inhibition; A （i,j） E represents the modulation strength at position (i, j) in the inhibition plot; β represents the final inhibition strength in the inhibition plot after calculation; （i,j） This represents the probability of suppressing the text at position (i, j) in the graph; C is a variable in the calculation process to ensure that the output range is 0 to 1; FP （i,j） This represents the probability of the text at position (i, j) in the final prediction graph; The final prediction map is obtained by dynamically suppressing the probability map through the adjustment map. The stronger the adjustment, the more severe the suppression, and the closer the probability value is to 0. Regions with an adjustment strength of 0 are not suppressed. After dynamic suppression, the probability values ​​of incorrectly predicted non-sequence number text regions and non-text regions will be suppressed, while the correctly predicted sequence number regions will not be affected. In this way, the final prediction map is closer to the true result. The final predicted image is processed using standard binarization to obtain a binary image, as follows: Among them B （i,j） P represents the value at position (i, j) in the binary graph. （i,j） This represents the value at position (i, j) in the prediction graph; t represents the pre-set threshold, which is set to 0.3; Use the findcontours method in OpenCV to obtain the text box coordinates of all serial number texts in the image; S6: Post-processing to obtain the coordinates of the text boxes and organizing the results.

2. The method for detecting engineering drawing serial numbers based on a suppression function according to claim 1, characterized in that: S1 specifically involves: organizing the training images of engineering drawings and uniformly labeling the serial number text to obtain a self-made engineering drawing dataset; during training, first using a general text detection dataset for training, and then using the self-made dataset for adjustment to obtain the network model required for prediction.

3. The method for detecting engineering drawing serial numbers based on a suppression function according to claim 2, characterized in that: S2 specifically involves: performing noise reduction and sharpening preprocessing on the engineering drawing to remove noise from the engineering drawing; segmenting the input engineering drawing, ensuring that the image to be segmented overlaps with the images on the top, bottom, left, and right sides during the segmentation process to ensure that no detection is missed; and then sequentially sending the multiple segmented images into the detection network for detection.

4. The engineering drawing serial number detection method based on a suppression function according to claim 3, characterized in that: S3 specifically refers to: inputting engineering drawings into a convolutional neural network, and obtaining feature maps of different scales after multiple convolutions and pooling; Feature maps at different scales contain information from different receptive fields of the original image. An adaptive scale fusion module is used to dynamically fuse feature maps at different scales. After training, the adaptive scale fusion module predicts the weights of feature maps at each scale in terms of channels and space. It is an attention mechanism that combines channels and space, as shown below: in, concat This indicates that the feature maps are concatenated along the channel dimension; {X0, X1, ..., X n-1 } represents a feature map of different scales that has been upsampled to a uniform size by different ratios; {W0, W1, ..., W n-1 } represents a weight map of the same size as the feature map, which is predicted by the network and contains channel and spatial weights; F represents the total feature map obtained by stitching together the individual feature maps after weight adjustment, which is used for subsequent prediction.

5. The engineering drawing serial number detection method based on a suppression function according to claim 4, characterized in that: Specifically, S4 involves designing two prediction branches based on the total feature map, which are used to predict the probability map and the suppression map, respectively. The probability map is the same size as the original image, and the value at each position represents the probability that the pixel at that position belongs to the serial number text, ranging from 0 to 1; The suppression map is the same size as the original image. Each position in the suppression map represents the intensity of suppression required for the pixel at that position, ranging from 0 to 1. For the serial number text area, the suppression intensity is 0, and for non-serial number text and non-text areas, the suppression intensity is 1. For the area surrounding the serial number text, the suppression intensity is a non-linear transition value of 0 to 1, with the intensity being greater the closer it is to the serial number text area and less the further away it is from the text area.

6. The method for detecting engineering drawing serial numbers based on a suppression function according to claim 1, characterized in that: Specifically, S6 is: For the final predicted image, the coordinates of the segmented image are offset by a corresponding distance based on its position in the original image, and then saved uniformly to the final result set. For all coordinates in the final result set, a non-maximum suppression algorithm is used to filter them, removing duplicate and similar text boxes, and obtaining the coordinates of all serial number text boxes in the entire engineering drawing.

Images