Robot service-oriented CAD drawing intelligent analysis method and device
By employing a domain-matching-based image-text segmentation and matching method, combined with data augmentation techniques and the YOLO v8 model, the domain gap and complexity issues in CAD drawing recognition are addressed, achieving high-precision primitive recognition suitable for robot services.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2024-11-12
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies struggle to effectively identify and understand complex CAD drawings, especially in robotic services, where domain gaps and the complexity and diversity of drawing content lead to low recognition accuracy and reliance on human experience.
We employ a domain-matching-based image-text segmentation and matching method, combined with data augmentation techniques, and use the YOLO v8 model for primitive recognition. By constructing a primitive dataset through text detection, image preprocessing, and data augmentation, we improve recognition accuracy.
It achieves high-precision recognition of CAD drawings in robot services, adapts to diverse application scenarios, and improves the efficiency and accuracy of graphic element recognition.
Smart Images
Figure CN119763142B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent recognition technology for CAD drawings, and in particular to intelligent analysis methods and devices for CAD drawings for robot services. Background Technology
[0002] Engineering design drawings are crucial for the completion and construction of projects, and drawing analysis has significant research value and engineering application value. For example, based on CAD drawing information, we can analyze the quality of construction and completion, and eliminate potential engineering problems. With the development of robot applications, intelligent drawing analysis has become particularly important, enabling robots to deeply understand their working environment and assist in navigation, inspection, and other tasks.
[0003] CAD element recognition is the foundation of CAD drawing analysis. CAD drawings consist of elements such as component symbols and text. Identifying component symbols in CAD drawings is key to CAD element recognition. Traditional CAD drawing analysis relies on manual review and interpretation, requiring professionals to examine and interpret each symbol and connection. This process is not only time-consuming and labor-intensive but also dependent on human experience, making it difficult to guarantee the comprehensiveness and consistency of the analysis.
[0004] With the development of technologies such as artificial intelligence, intelligent recognition of CAD drawings has gradually become a research hotspot. In the early stages, manual feature extraction methods were mainly used to extract the shape features of graphic elements, and classifiers were designed to achieve element recognition. In recent years, object detection technology has developed rapidly. More and more scholars have begun to study how to use these models to extract depth features and achieve efficient object detection.
[0005] However, intelligent recognition of CAD drawings still faces many difficulties and challenges. First, there is the domain gap. Current artificial intelligence technology is typically data-driven, primarily using common image and video data, which presents a significant domain gap compared to engineering design drawing data. Therefore, existing AI models and data are difficult to directly transfer to the engineering design field. Second, the complexity and diversity of CAD drawings pose a significant challenge to intelligent recognition. For example, CAD drawings typically contain a large number of complex electrical components, connection information (straight lines, dashed lines, line combinations), text, and other graphic elements, and the relationships between different elements are complex, making intelligent understanding of CAD drawings extremely challenging. Summary of the Invention
[0006] This application aims to at least partially address one of the technical problems in the related art.
[0007] Therefore, the first objective of this application is to propose an intelligent analysis method for CAD drawings for robot services, which can adapt to various application scenarios and has high accuracy in element recognition.
[0008] The second objective of this application is to propose an intelligent CAD drawing analysis device for robot services.
[0009] To achieve the above objectives, the first aspect of this application proposes a method for intelligent analysis of CAD drawings for robot services, comprising:
[0010] Obtain the original primitive dataset;
[0011] Text detection is performed on primitive images in the original primitive data set to obtain text detection results, which include text location and text label;
[0012] Based on the text detection results, the primitive images in the original image data set are preprocessed, and primitive detection is performed on the preprocessed images to obtain image detection results, which include image location and primitive images.
[0013] Text labels and primitive images are paired based on text location and image location, and a primitive dataset is constructed based on the paired primitive images and text labels;
[0014] Data augmentation is performed on the primitive images in the primitive data set, and the processed primitive images are used to train the target detection model to obtain the trained target detection model.
[0015] Data augmentation is performed on the primitive images to be detected, and the processed primitive images are input into the trained target detection model to output the primitive detection results.
[0016] This application's intelligent CAD drawing analysis method for robot services acquires diverse and comprehensive original primitive data. Based on this, it proposes an image-text segmentation and matching method based on domain matching to pair primitive images and text, forming a commonly used basic primitive dataset. Secondly, this application proposes various data augmentation methods such as image filling and proportional scaling to enrich and expand the primitive dataset. Finally, considering both efficiency and accuracy, YOLO v8 is selected as the base model to build an intelligent primitive recognition model, which is trained on the augmented primitive data, effectively improving the model's primitive recognition capability.
[0017] Optionally, in one embodiment of this application, text detection is performed on primitive images in the original primitive data set, including:
[0018] Text detection is performed using an OCR detection model to identify and locate the position information and text content of text boxes in an image, and to determine text labels based on the text content.
[0019] Optionally, in one embodiment of this application, preprocessing of the primitive image based on the text detection result includes:
[0020] The text region is determined based on the text location, the text region is covered with the background color, and the straight line part is removed using Hough transform.
[0021] The preprocessed image is subjected to primitive detection to obtain the image detection results, including:
[0022] The image is converted to grayscale, and the edges of primitives are located using the Canny edge detection algorithm. Neighboring primitives are then merged using morphological dilation.
[0023] Contour detection is performed using binarized images, and valid primitive contours are selected based on the hierarchical structure and size of the contours.
[0024] The primitive image is cut based on the primitive contour, and the position information of the primitive image is recorded.
[0025] Optionally, in one embodiment of this application, pairing text labels and primitive images based on text location and image location includes:
[0026] The distance between text and primitive images is determined based on text and image locations, and the text label of the nearest text is paired with the primitive image.
[0027] Optionally, in one embodiment of this application, data augmentation of the primitive image includes:
[0028] The primitive image is scaled proportionally, and a solid color background is used to fill the area around the scaled primitive image to make it fill a fixed image size.
[0029] The target detection model is YOLOv8. The target detection model uses an improved CSPDarknet as the backbone network for feature extraction and performs multi-scale feature fusion through a feature pyramid network and a path aggregation network.
[0030] Training the object detection model includes:
[0031] The target detection model is optimized by using localization loss, classification loss, and confidence loss.
[0032] To achieve the above objectives, a second aspect of the present invention provides a CAD drawing intelligent analysis device for robot services, comprising:
[0033] The data acquisition module is used to acquire the original primitive dataset;
[0034] The text detection module is used to perform text detection on primitive images in the original image data set and obtain text detection results, which include text location and text label;
[0035] The image detection module is used to preprocess the primitive images in the original image data set based on the text detection results, and to perform primitive detection on the preprocessed images to obtain the image detection results, which include image location and primitive images.
[0036] The pairing module is used to pair text labels and primitive images based on text location and image location, and to construct a primitive dataset based on the paired primitive images and text labels;
[0037] The model training module is used to perform data augmentation on primitive images in the primitive data set, and to train the target detection model using the processed primitive images to obtain the trained target detection model.
[0038] The primitive recognition module is used to perform data augmentation on the primitive image to be detected, and input the processed primitive image into the trained target detection model to output the primitive detection result.
[0039] Optionally, in one embodiment of this application, text detection is performed on primitive images in the original primitive data set, including:
[0040] Text detection is performed using an OCR detection model to identify and locate the position information and text content of text boxes in an image, and to determine text labels based on the text content.
[0041] Optionally, in one embodiment of this application, preprocessing of the primitive image based on the text detection result includes:
[0042] The text region is determined based on the text location, the text region is covered with the background color, and the straight line part is removed using Hough transform.
[0043] The preprocessed image is subjected to primitive detection to obtain the image detection results, including:
[0044] The image is converted to grayscale, and the edges of primitives are located using the Canny edge detection algorithm. Neighboring primitives are then merged using morphological dilation.
[0045] Contour detection is performed using binarized images, and valid primitive contours are selected based on the hierarchical structure and size of the contours.
[0046] The primitive image is cut based on the primitive contour, and the position information of the primitive image is recorded.
[0047] Optionally, in one embodiment of this application, pairing text labels and primitive images based on text location and image location includes:
[0048] The distance between text and primitive images is determined based on text and image locations, and the text label of the nearest text is paired with the primitive image.
[0049] Optionally, in one embodiment of this application, data augmentation of the primitive image includes:
[0050] The primitive image is scaled proportionally, and a solid color background is used to fill the area around the scaled primitive image to make it fill a fixed image size.
[0051] The target detection model is YOLOv8. The target detection model uses an improved CSPDarknet as the backbone network for feature extraction and performs multi-scale feature fusion through a feature pyramid network and a path aggregation network.
[0052] Training the object detection model includes:
[0053] The target detection model is optimized by using localization loss, classification loss, and confidence loss.
[0054] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0055] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:
[0056] Figure 1 This is a flowchart illustrating a robot-oriented intelligent analysis method for CAD drawings provided in Embodiment 1 of this application.
[0057] Figure 2 A schematic diagram illustrating the process of constructing the primitive dataset for an embodiment of this application;
[0058] Figure 3 This is an example diagram of the dataset matching results in an embodiment of this application;
[0059] Figure 4 This is a schematic diagram of the element matching process according to an embodiment of this application;
[0060] Figure 5 This is a schematic diagram of the class distribution in the dataset of an embodiment of this application;
[0061] Figure 6 This is a schematic diagram of the intelligent recognition process for CAD elements in an embodiment of this application;
[0062] Figure 7 This is a schematic diagram of the confusion matrix according to an embodiment of this application;
[0063] Figure 8 This is a schematic diagram of the ablation experiment results of an embodiment of this application;
[0064] Figure 9 This is a schematic diagram of a CAD drawing intelligent analysis device for robot services provided in an embodiment of this application. Detailed Implementation
[0065] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0066] The following description, with reference to the accompanying drawings, illustrates an embodiment of the intelligent CAD drawing analysis method and apparatus for robot services.
[0067] Figure 1 This is a flowchart illustrating a robot-oriented intelligent analysis method for CAD drawings provided in Embodiment 1 of this application.
[0068] like Figure 1 As shown, this intelligent CAD drawing analysis method for robot services includes the following steps:
[0069] Step 101: Obtain the original primitive dataset;
[0070] Step 102: Perform text detection on the primitive images in the original primitive data set to obtain text detection results, including text location and text label;
[0071] Step 103: Based on the text detection results, preprocess the primitive images in the original image data set, and perform primitive detection on the preprocessed images to obtain image detection results, wherein the image detection results include image location and primitive images;
[0072] Step 104: Pair text labels and primitive images based on text location and image location, and construct a primitive dataset based on the paired primitive images and text labels;
[0073] Step 105: Perform data augmentation on the primitive images in the primitive data set, and use the processed primitive images to train the target detection model to obtain the trained target detection model.
[0074] Step 106: Perform data augmentation on the primitive image to be detected, and input the processed primitive image into the trained target detection model to output the primitive detection result.
[0075] This application's intelligent CAD drawing analysis method for robot services acquires diverse and comprehensive original primitive data. Based on this, it proposes an image-text segmentation and matching method based on domain matching to pair primitive images and text, forming a commonly used basic primitive dataset. Secondly, this application proposes various data augmentation methods such as image filling and proportional scaling to enrich and expand the primitive dataset. Finally, considering both efficiency and accuracy, YOLO v8 is selected as the base model to build an intelligent primitive recognition model, which is trained on the augmented primitive data, effectively improving the model's primitive recognition capability.
[0076] Embodiment 2 of this application provides another intelligent analysis method for CAD drawings for robot services, which is applied to the power industry.
[0077] This embodiment includes constructing a power CAD primitive dataset and CAD primitive recognition. Specifically,
[0078] Figure 2 The process for constructing the primitive dataset in this paper is as follows: First, basic primitive image data is acquired to construct the original primitive dataset; second, text labels are obtained through text detection, and the text labels are preprocessed to merge labels with the same primitive semantics; then, primitive image detection is performed by combining the original image and the detected text information; finally, image-text matching is performed, and the processed text labels and primitive images are matched one-to-one to obtain primitive images and their text label information.
[0079] The detailed processing is as follows:
[0080] (1) Data Acquisition
[0081] Obtain existing publicly available image source data, including several image data, such as a single image. Figure 3 As shown, Figure 3 The left side shows basic graphic primitive images, mainly containing device symbols and their meanings. Figure 3 The right side shows the image-text matching results, with the same color indicating the matched image-text pairs.
[0082] (2) Text recognition
[0083] An OCR detection model was used for text detection. This model, through the advanced structure of CRNN, integrates the feature extraction capabilities of Convolutional Neural Networks (CNN) and the sequence processing capabilities of Long Short-Term Memory Networks (LSTM), effectively identifying and locating Chinese text in images. This step not only captures the positional information of the text boxes but also obtains the text content, providing important information for subsequent image preprocessing.
[0084] (3) Image segmentation
[0085] Next, the goal of this embodiment is to remove elements that may interfere with primitive detection, such as text and long straight lines in the image. Therefore, based on the text detection results, this embodiment performs some detailed preprocessing on the original image, including covering the text areas with background color and using Hough transform to remove straight lines. This process not only cleans the image but also preserves the integrity of the primitive image, laying a solid foundation for subsequent primitive detection.
[0086] Based on the processed image, this embodiment further implements primitive detection. First, the image is converted to a grayscale image, and the Canny edge detection algorithm is used to locate the edges of primitives. Morphological dilation is used to merge neighboring primitives, enhancing their recognizability. Subsequently, contour detection is performed using the binarized image, and valid primitive contours are selected based on their hierarchical structure and size. Finally, the primitive images are precisely segmented using bounding boxes, and their positional information is recorded, laying the foundation for further analysis and application. The specific process is as follows: Figure 4 As shown.
[0087] (4) Image and text matching
[0088] This embodiment proposes a domain-based image-text matching method, which performs a one-to-one pairing of the image and text information obtained above to acquire primitive images and their text label information. Matching is performed with the horizontal axis of the text detection box as the center, pairing primitives near the horizontal line and close to the text, traversing all text information to achieve primitive image and text label pairing, such as... Figure 4 As shown. The specific process is to first determine the arrangement order of the graphic elements and their corresponding text. If they are arranged in rows, then the left-right matching method is used; conversely, if they are arranged in columns, then the top-bottom matching method is used.
[0089] In summary, following the methods described above, this embodiment constructs an Electrical CAD Element Dataset (ECED). This dataset includes 32 major categories of elements and over 600 element instances.
[0090] like Figure 5The graph shows the sample size distribution of the ECED dataset, where the horizontal axis represents the primitive name and the vertical axis represents the number of primitives. As can be seen, this dataset exhibits a classic long-tail distribution. For example, some categories (such as transistors) appear very frequently in the dataset, dominating in terms of quantity. Conversely, some categories appear only once or very rarely. This long-tail distribution presents challenges for primitive recognition.
[0091] Figure 6 The CAD element recognition process in this embodiment is as follows: Figure 6 As shown, the first step is to input the image of the primitive to be detected; the second step is to perform multi-scale data augmentation on the input image to enhance the multi-scale target recognition capability of the model; finally, considering both efficiency and accuracy, YOLO V8 is selected as the target detection model in this embodiment to obtain the detection result of the currently input primitive.
[0092] (1) Data Augmentation
[0093] Considering the small data volume and diverse scales of primitive datasets, this embodiment proposes a multi-scale data augmentation method to expand the constructed primitive dataset and improve the model's multi-scale target recognition capability. This method mainly achieves this through image scaling and padding operations. First, this embodiment scales the length and width of the original image proportionally to preserve the spatial and shape features of the original image and avoid distortion of the primitive image. Second, padding is applied around the scaled primitive image to fill it to a fixed image size, which is beneficial for training deep neural networks. This embodiment adjusts the scaled images to a uniform size. For images with a scaled size less than 224×224, this embodiment pads the image with white background pixels to expand it to the specified image size. The primitive dataset constructed in this embodiment mainly consists of white background images; therefore, white background padding is used to preserve the realism and naturalness of the primitive image appearance.
[0094] (2) Target recognition:
[0095] This embodiment selects YOLOv8 (You Only Look Once, version 8) as the object detection model, uses an improved CSPDarknet as the backbone network for feature extraction, performs multi-scale feature fusion through Feature Pyramid Network (FPN) and Path Aggregation Network (PANet), and optimizes the model through localization loss, classification loss and confidence loss.
[0096] This embodiment conducted a series of experiments to verify the proposed primitive dataset and primitive recognition framework.
[0097] (1) Experimental setup
[0098] This embodiment uses ECED for experimental verification. To better train and evaluate the model, the dataset is first randomly divided into disjoint training and test sets. 70% of the data is used for training, and the remaining 30% for testing. In YOLOv8 model training, this embodiment empirically sets the batch size to 16, trains for 200 epochs, and uses a learning rate of 0.01. For data augmentation, primitive images are scaled using ratios of 1.25, 1.5, 1.75, 2.0, and 2.25, respectively.
[0099] (2) Results and Discussion
[0100] On the test set, the precision was 96.10%. Table 1 provides more details about the performance of different categories and uses three different metrics: precision, recall, and F1 score.
[0101] Table 1 shows the training results for different categories on the ECED dataset.
[0102] Category Name Number of images Accuracy Recall rate Mean Precision operate 2 97.70％ 100.00％ 99.50％ capacitance 5 100.00％ 61.80％ 90.00％ triode 7 84.60％ 85.70％ 93.30％ sensor 1 100.00％ 100.00％ 99.50％ straight segment 1 100.00％ 100.00％ 99.50％ grounding wire 3 97.80％ 100.00％ 99.50％ cable 4 98.60％ 100.00％ 99.50％ Switches and sockets 23 95.40％ 90.10％ 98.00％ potentiometers and resistors 7 99.20％ 100.00％ 99.50％ Amplifier 1 100.00％ 100.00％ 99.50％ fuse 4 100.00％ 78.60％ 99.50％ Motor and windings 5 85.50％ 80.00％ 93.80％ illumination 5 88.20％ 80.00％ 84.70％ inductance 3 98.00％ 100.00％ 99.50％ diode 6 85.00％ 100.00％ 97.20％ Coupler 2 96.70％ 100.00％ 99.50％ Crystal Oscillator 1 95.90％ 100.00％ 99.50％ transformer 7 100.00％ 76.80％ 99.50％ substation power station 3 100.00％ 50.20％ 99.50％ electrical equipment 4 98.60％ 100.00％ 99.50％ power supply 6 82.60％ 100.00％ 99.50％ other 5 96.40％ 100.00％ 99.50％ relay 4 100.00％ 78.80％ 99.50％ line 11 100.00％ 82.40％ 93.80％ Connecting piece 1 100.00％ 0.00％ 99.50％ Electricity meter 4 98.30％ 100.00％ 99.50％ average 4.81 96.10％ 87.09％ 97.74％
[0103] Table 1 shows the experimental results on the ECED dataset. As can be seen from the table, the average precision for different categories reached 96.10%. However, the detection performance of some categories was difficult to measure due to their low frequency of occurrence in the dataset. For example, the "fuse" category, despite its high precision, had a recall rate of only 78.60% due to the limited number of instances in the dataset, indicating that some instances were missed during detection. The same situation occurred with the "transformer" and "substation / power station" categories, with recall rates of 76.80% and 50.20% respectively, showing that the detector has certain limitations in identifying these categories.
[0104] Meanwhile, this experiment observed that some categories in the dataset showed significantly improved detection performance due to the larger number of instances. Taking "potentiometers and resistors" as an example, it had 7 instances in the test dataset and its F1 score was near perfect, demonstrating high precision and high recall. Furthermore, the "switch and socket" category also exhibited high detection accuracy due to its 23 instances in the test dataset.
[0105] Next, this embodiment further analyzes the classification performance of the model using a confusion matrix. This embodiment constructs a confusion matrix (such as...). Figure 7 As shown in the diagram, rows represent the true classes and columns represent the predicted classes. This matrix clearly shows the prediction accuracy and misclassification rates for each class.
[0106] Using the confusion matrix, this embodiment found that the model performs exceptionally well in some categories, such as "potentiometers and resistors," showing a near-perfect F1 score. However, for some categories, such as "transformers," the model's recall is low due to the small number of samples in the dataset, indicating that the model's recognition ability in these categories needs improvement.
[0107] (3) Ablation test
[0108] In this embodiment, to gain a deeper understanding of the impact of data preprocessing on the performance of the CAD primitive recognition model, three ablation experiments were designed to test different dataset resizing methods. In the first ablation experiment, all images in the dataset were directly resized to a uniform 224x224 pixel size. This method ensures that all input images have the same dimensions by changing the image size, simplifying the model's input requirements. However, this may lead to the loss and distortion of image information, especially for images with smaller original sizes, as shown in Table 2. In the second ablation experiment, a padding technique was used to maintain the original image size and proportions, and a border (usually a white background) was added around the images to ensure that all images have a uniform input size (224x224). This method preserves the original image proportions and reduces information distortion caused by resizing, but the padding may introduce additional background noise, affecting the recognition of certain image features, as shown in Table 2.
[0109] Table 2 shows the experimental results of resize and padding ablation on the dataset.
[0110] Average accuracy Average recall Average mAP50 Resize 91.56％ 76.78％ 95.73％ Padding 94.01％ 84.91％ 97.31％
[0111] Experimental results (see Table 2) show that the Padding method exhibits better performance, especially in terms of recall, compared to the Resize method alone. This is likely because the Padding method preserves the original aspect ratio of the image, reducing information distortion caused by resizing.
[0112] To further explore the impact of data preprocessing on the performance of the CAD primitive recognition model, this embodiment conducts a third ablation experiment to evaluate the effect of different scaling ratios combined with padding techniques on model performance. Specifically, images smaller than 224x224 pixels in the dataset are scaled up proportionally at ratios of 1.25x, 1.5x, 1.75x, 2.0x, and 2.25x. For images smaller than 224x224 pixels after scaling, padding techniques are used to adjust them to the standard size. For images larger than 224x224 pixels, their original proportions and quality are maintained without further size adjustment. The results are as follows: Figure 8 As shown.
[0113] Figure 8 This section illustrates the impact of different magnification ratios combined with infill techniques on model performance in the third ablation experiment. Analysis of the experimental results shows that different magnification ratios have varying effects on the model's mean precision, mean recall, and mean mAP50. At a magnification ratio of 1.25, the model achieves the highest mean precision (96.60%), mean recall (87.20%), and mean mAP50 (98.40%). At this ratio, the image is moderately magnified, preserving more original details while avoiding information distortion caused by excessive magnification. Therefore, the model achieves a balance between high precision and high recall, resulting in optimal performance. As the magnification ratio increases to 1.50 and 1.75, although the model's recall remains high, the mean precision decreases slightly. This may be because details begin to be lost or distorted during further magnification, making the recognition of certain primitives more difficult. At a magnification ratio of 2.00, the model's performance (especially mAP50) reaches its peak. At this point, the image resolution is significantly improved, preserving more details and thus enhancing the model's recognition ability. However, further increasing the magnification ratio to 2.25 did not significantly improve model performance. Excessive magnification may introduce additional noise and background information, negatively impacting recognition.
[0114] This embodiment constructs a novel Electrical CAD Primitive Dataset (ECED) and proposes a baseline method for intelligent analysis of CAD drawings. First, the proposed ECED dataset contains a rich set of primitive symbols and instances, providing learning data and recognition benchmarks for intelligent analysis of CAD drawings, effectively alleviating the challenge of data scarcity in the electrical engineering field. Second, this embodiment utilizes YOLO object detection technology to propose an intelligent CAD drawing recognition model, enabling automatic parsing of CAD drawings and effectively improving the efficiency of engineering drawing review and retrieval.
[0115] To achieve the above embodiments, this application also proposes an intelligent CAD drawing analysis device for robot services.
[0116] Figure 9 This is a schematic diagram of a CAD drawing intelligent analysis device for robot services provided in an embodiment of this application.
[0117] like Figure 9 As shown, the intelligent CAD drawing analysis device for robot services includes:
[0118] The data acquisition module is used to acquire the original primitive dataset;
[0119] The text detection module is used to perform text detection on primitive images in the original image data set and obtain text detection results, which include text location and text label;
[0120] The image detection module is used to preprocess the primitive images in the original image data set based on the text detection results, and to perform primitive detection on the preprocessed images to obtain the image detection results, which include image location and primitive images.
[0121] The pairing module is used to pair text labels and primitive images based on text location and image location, and to construct a primitive dataset based on the paired primitive images and text labels;
[0122] The model training module is used to perform data augmentation on primitive images in the primitive data set, and to train the target detection model using the processed primitive images to obtain the trained target detection model.
[0123] The primitive recognition module is used to perform data augmentation on the primitive image to be detected, and input the processed primitive image into the trained target detection model to output the primitive detection result.
[0124] Optionally, in one embodiment of this application, text detection is performed on primitive images in the original primitive data set, including:
[0125] Text detection is performed using an OCR detection model to identify and locate the position information and text content of text boxes in an image, and to determine text labels based on the text content.
[0126] Optionally, in one embodiment of this application, preprocessing of the primitive image based on the text detection result includes:
[0127] The text region is determined based on the text location, the text region is covered with the background color, and the straight line part is removed using Hough transform.
[0128] The preprocessed image is subjected to primitive detection to obtain the image detection results, including:
[0129] The image is converted to grayscale, and the edges of primitives are located using the Canny edge detection algorithm. Neighboring primitives are then merged using morphological dilation.
[0130] Contour detection is performed using binarized images, and valid primitive contours are selected based on the hierarchical structure and size of the contours.
[0131] The primitive image is cut based on the primitive contour, and the position information of the primitive image is recorded.
[0132] Optionally, in one embodiment of this application, pairing text labels and primitive images based on text location and image location includes:
[0133] The distance between text and primitive images is determined based on text and image locations, and the text label of the nearest text is paired with the primitive image.
[0134] Optionally, in one embodiment of this application, data augmentation of the primitive image includes:
[0135] The primitive image is scaled proportionally, and a solid color background is used to fill the area around the scaled primitive image to make it fill a fixed image size.
[0136] The target detection model is YOLOv8. The target detection model uses an improved CSPDarknet as the backbone network for feature extraction and performs multi-scale feature fusion through a feature pyramid network and a path aggregation network.
[0137] Training the object detection model includes:
[0138] The target detection model is optimized by using localization loss, classification loss, and confidence loss.
[0139] It should be noted that the foregoing explanation of the embodiment of the intelligent analysis method for CAD drawings for robot services also applies to the intelligent analysis device for CAD drawings for robot services in this embodiment, and will not be repeated here.
[0140] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0141] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0142] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0143] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0144] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0145] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0146] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0147] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.
Claims
1. A method for intelligent analysis of CAD drawings for robot services, characterized in that, include: Obtain the original primitive dataset; Text detection is performed on the primitive images in the original primitive data set to obtain text detection results, wherein the text detection results include text location and text label; Based on the text detection results, the primitive images in the original primitive data set are preprocessed, and primitive detection is performed on the preprocessed images to obtain image detection results, wherein the image detection results include image location and primitive images; Text labels and primitive images are paired based on text location and image location, and a primitive dataset is constructed based on the paired primitive images and text labels; Data augmentation is performed on the primitive images in the primitive data set, and the processed primitive images are used to train the target detection model to obtain the trained target detection model. Data augmentation is performed on the primitive image to be detected, and the processed primitive image is input into the trained target detection model to output the primitive detection result; The preprocessing of primitive images based on text detection results includes: The text region is determined based on the text location, the text region is covered with the background color, and the straight line part is removed using Hough transform. The step of performing primitive detection on the preprocessed image to obtain image detection results includes: The image is converted to grayscale, and the edges of primitives are located using the Canny edge detection algorithm. Neighboring primitives are then merged using morphological dilation. Contour detection is performed using binarized images, and valid primitive contours are selected based on the hierarchical structure and size of the contours. The primitive image is cut based on the primitive contour, and the position information of the primitive image is recorded; The process of pairing text labels and primitive images based on text location and image location includes: Determine the order of graphic elements and their corresponding text. If they are arranged by row, use left-right matching; if they are arranged by column, use top-bottom matching. Data augmentation of primitive images includes: The primitive image is scaled proportionally, and a solid color background is used to fill the area around the scaled primitive image to make it fill a fixed image size. The target detection model is YOLOv8, which uses an improved CSPDarknet as the backbone network for feature extraction and performs multi-scale feature fusion through a feature pyramid network and a path aggregation network. Training the object detection model includes: The target detection model is optimized using localization loss, classification loss, and confidence loss.
2. The method as described in claim 1, characterized in that, The text detection of primitive images in the original primitive data set includes: Text detection is performed using an OCR detection model to identify and locate the position information and text content of text boxes in an image, and to determine text labels based on the text content.
3. The method as described in claim 1, characterized in that, The method of pairing text labels and primitive images based on text location and image location also includes: The distance between text and primitive images is determined based on text and image locations, and the text label of the nearest text is paired with the primitive image.
4. A CAD drawing intelligent analysis device for robot services, characterized in that, include: The data acquisition module is used to acquire the original primitive dataset; The text detection module is used to perform text detection on the primitive images in the original image data set and obtain text detection results, wherein the text detection results include text location and text label; The image detection module is used to preprocess the primitive images in the original image data set based on the text detection results, and to perform primitive detection on the preprocessed images to obtain image detection results, wherein the image detection results include image location and primitive images; The pairing module is used to pair text labels and primitive images based on text location and image location, and to construct a primitive dataset based on the paired primitive images and text labels; The model training module is used to perform data augmentation on the primitive images in the primitive data set, and use the processed primitive images to train the target detection model to obtain the trained target detection model. The primitive recognition module is used to perform data augmentation on the primitive image to be detected, and input the processed primitive image into the trained target detection model to output the primitive detection result. The preprocessing of primitive images based on text detection results includes: The text region is determined based on the text location, the text region is covered with the background color, and the straight line part is removed using Hough transform. The step of performing primitive detection on the preprocessed image to obtain image detection results includes: The image is converted to grayscale, and the edges of primitives are located using the Canny edge detection algorithm. Neighboring primitives are then merged using morphological dilation. Contour detection is performed using binarized images, and valid primitive contours are selected based on the hierarchical structure and size of the contours. The primitive image is cut based on the primitive contour, and the position information of the primitive image is recorded; The process of pairing text labels and primitive images based on text location and image location includes: Determine the order of graphic elements and their corresponding text. If they are arranged by row, use left-right matching; if they are arranged by column, use top-bottom matching. Data augmentation of primitive images includes: The primitive image is scaled proportionally, and a solid color background is used to fill the area around the scaled primitive image to make it fill a fixed image size. The target detection model is YOLOv8, which uses an improved CSPDarknet as the backbone network for feature extraction and performs multi-scale feature fusion through a feature pyramid network and a path aggregation network. Training the object detection model includes: The target detection model is optimized using localization loss, classification loss, and confidence loss.
5. The apparatus as described in claim 4, characterized in that, The text detection of primitive images in the original primitive data set includes: Text detection is performed using an OCR detection model to identify and locate the position information and text content of text boxes in an image, and to determine text labels based on the text content.
6. The apparatus as claimed in claim 4, characterized in that, The method of pairing text labels and primitive images based on text location and image location also includes: The distance between text and primitive images is determined based on text and image locations, and the text label of the nearest text is paired with the primitive image.