Method and system for the automatic verification of compliance with a set of rules for two-dimensional drawings

Abstract

The invention presented is a cutting-edge tool based on artificial intelligence, developed for the purpose of detecting and verifying the rules outlined in the design drawings of brake calipers. In particular, the method employs object detection technology to accurately identify and isolate the desired rule to be verified. Moreover, optical character recognition technology is used to accurately analyze and characterize the values associated with the detected rules. The artificial intelligence tool is accessible by means of general devices such as laptops, desktops and smartphones through a graphical interface which allows designers to self-verify compliance with design rules in their technical drawings, increasing the accuracy and quality of the technical drawing and reducing design time.

Description

[0001] METHOD AND SYSTEM FOR THEAUTOMATIC VERIFICATION OFCOMPLIANCE WITH A SET OFRULES FOR TWO-DIMENSIONALDRAWINGS

[0002] The present invention relates to a method and a system for the automatic verification of compliance with a set of two-dimensional drawing rules, in particular CAD drawings.

[0003] Background art

[0004] In the field of mechanical engineering, technical drawings are essential for conveying design information from the engineers to the producers. These drawings contain information on dimensions, tolerances and materials to be used to manufacture a product, so that it meets the required specifications and functions optimally in real scenarios. However, violations of design rules are not uncommon and can lead to delays in the design cycle, due to the time necessary to redesign the product after feedback from other engineers or customers, or, in the worst cases, to manufacturing defects and costly reworking. It is therefore necessary to have an automatic system capable of verifying the compliance of technical drawings with design rules.

[0005] One of the main difficulties in creating an automated system for the verification of compliance of technical drawings with design rules is the implementation of a formally defined business logic which interacts with the meshes or structures of CAD software models. Meshes are difficult to work with, since they lack the hierarchical structure needed to convey any type of semantic context necessary to implement reusable and reliable rule controls. The representation of CAD software 3D models provides the type of hierarchy missing in pure meshes, but inconsistencies in the design process and representation of 3D models between different CAD tools pose new difficulties in implementing rule controls.

[0006] Many previous attempts to automate design rule controls are based on the 3D model data structure used by CAD software. The problems inherent in this solution are exemplified in WO2020138738A1, where the described system is based on separation into predefined and known components, grouped into a final assembly, neglecting other possible hierarchical representations of the final product. Moreover, the patent document does not provide implementation details which make the effectiveness of the proposed method assessable.

[0007] A possible solution for overcoming the limitation is to require the CAD software user to manually annotate the points (by means of appropriate "tags"), surfaces or volumes of interest of the 3D model for each specific verification of the rules. This is the type of work presented in KR101769433. While on the one hand the dependence on the predefined structure of the 3D model is eliminated, on the other hand this solution offers a lower degree of automation and marginal advantages as compared to manual controls in terms of violating the captured rules, since the execution of annotations can be as error-prone as manual self-controls.

[0008] Object and subject-matter of the invention

[0009] It is the object of the present invention to provide a method and a system for the automatic verification of compliance with a set of two-dimensional drawing rules, in particular CAD drawings, which solves the problems and overcomes the drawbacks of the prior art.

[0010] The present invention relates to a method and a system according to the appended claims.

[0011] Detailed description of embodiments of the invention

[0012] List of figures

[0013] The invention will now be described by way of nonlimiting illustration, with particular reference to the figures in the accompanying drawings, in which:

[0014] Figure 1 shows an example of a rule for technical drawings, according to an embodiment of the invention;

[0015] Figure 2 shows a block diagram of a preferred embodiment of the invention;

[0016] Figure 3 shows a flow diagram illustrating the logical flow of user interaction with the method, according to an embodiment of the invention;

[0017] Figure 4 shows a flow diagram of the processing of a new technical drawing, according to an aspect of the invention;

[0018] Figure 5 shows a different embodiment of the method according to the invention; Figure 6 shows an example of a result of an object detection according to the method of the invention;

[0019] Figure 7 shows an example of a result of a text localization in an image according to the method of the invention; and

[0020] Figure 8 shows an implementation example of the radius according to the invention.

[0021] It is here specified that elements of different embodiments can be combined to provide further embodiments, without restrictions, while respecting the technical concept of the invention, as those skilled in the art will effortlessly understand from the description .

[0022] The present description also refers to the prior art for the implementation thereof in relation to the detail features not described, such as elements of minor importance usually used in the prior art in solutions of the same type, for example.

[0023] When an element is introduced, it is always understood that there can be "at least one" or "one or more ".

[0024] When a list of elements or features is given in this description, it is understood that the finding according to the invention "comprises" or alternatively "consists of" such elements.

[0025] When listing features within the same sentence or bullet list, one or more of the single features can be included in the invention without connection with the other features on the list.

[0026] Embodiments This invention aims to automate the verification of predefined rules (related to technical design drawings) on specific (i.e., of a certain type, such as brake caliper drawings, generally of one or more general devices) design drawings (or, indifferently, "technical drawings"); an example of a rule for the design drawing is Figure 1, where an exemplary portion of the technical drawing is shown together with the limit values (minimum and / or maximum) with which the quotas of interest must comply.

[0027] In general, as a rule, this document refers to a geometric constraint with which the technical drawing must comply. This translates into one or more constraints of maximum and / or minimum values on one or more quotas.

[0028] Each rule is associated with a region of relevance, to which the rule itself applies on the technical drawing.

[0029] The invention can be implemented in a software system consisting of several components running on different servers: a web server, a data server and an artificial intelligence (Al) server. The latter performs a series of automated steps which control the rules of a technical drawing and indicate which design rules have not been respected or have not been depicted in the drawing.

[0030] The automated steps can comprise the following steps.

[0031] As a first step, there ca be the detection of geometric (non-textual) objects in the technical drawings to be verified. An artificial intelligence based algorithm is used which is capable of identifying and locating predefined objects on the images which are composed by the technical drawings. This component is used to identify the regions of relevance of each rule on the design drawings. Deep Neural Network-based object detection was used for this task, which required training a neural network model on images of specific design drawings for some use cases.

[0032] As a second step, an optical recognition (OCR) of the characters in the technical drawings can be performed: this method allows automatically identifying and encoding the text present in an image. The values encoded by the machine are then compared with a set or range of allowed values (quotas) predefined in the rule prescription, in the fourth step.

[0033] As a third step, a process can be carried out to associate the values extracted from the OCR with the regions of relevance of the detected rules: both the above methods locate the region of interest and the values; to identify which values belong to each region of interest, an algorithm was designed according to an aspect of the invention. This algorithm associates the values detected in the second step with each region of interest detected in the first step, based on the position of the first with respect to the second and further graphical elements (by way of example, mainly two quoting lines) as observed in the data set used for object detection and OCR training. According to a preferred embodiment of the invention, the association is carried out based on the distance of the values from the region of interest, associating the first with the second if they are within the ninety-ninth percentile of the distribution of the distances from the center of the region of interest at the related quotas observed in the example samples used for training (both Al and OCR. The training dataset without considering the output labels is the same). Such a procedure is exemplified in Figure

[0034] 8 where the radius within which the values identified by the OCR system are associated is depicted for a given region of interest.

[0035] As a fourth step, an association can be performed of the values detected in the second step with the related prescription of predefined rules: after associating each detected region with the values thereof, the quotas on the technical drawing are associated and then compared with those prescribed in the definition of the rule, taking into account predefined tolerances.

[0036] In the prior art, no attempt has been found to take advantage of the potential of these drawings in 2D rather than 3D. The use of deep learning-based object detection on 2D mechanical drawings requires several data for each specific rule definition, but since engineering drawings are sensitive information, in most cases classified, the images needed to train such a model are very rare.

[0037] To carry out the object detection step, a model was trained on images made according to design rules: from the very definition of the rule and the region of interest of the rule situated in a set of existing engineering drawings of brake calipers in a specific case of interest. In order to provide these images to our model, a pre-processing step was required, since the data collected for this specific project were generated by CATIA™ V5 design software. The object detection model chosen for this project is

[0038] Faster-RCNN (in particular the ResNet-50 backbone and the faster_rcnn_R_50_C4_1x implementation of detectron2 v0.6 were used for testing, see also https: / / debuggercafe.com / object-detection-using pytorch-faster-rcnn-resnet50-fpn-v2 / ), which requires images as data input for model training. The data

[0039] (drawings) acquired for this invention were rasterized into bitmap images.

[0040] After transforming the data into images, the rasterized drawings were separated into blocks; since these drawings consist of different views of the same caliper (or other chosen object), it is important for the performance of the model that the views are separated and processed one by one. The image data was then processed to train the model, resizing it into a suitable format. Moreover, in order to improve model performance, techniques were used to increase the data on the available dataset, such as image rotation, tilting and translation.

[0041] Deep learning-based object detection learns based on the data provided thereto. Faster-RCNN was trained on general data, in particular on the COCO ("Common Objects in Context") dataset, which consists of thousands of images with more than 80 common objects in everyday contexts. Since this system aims to detect specific design rules, the model was requalified on internally collected data specifically for this use case to overcome generalization issues. The object detection step in this system was re-qualified to 5 classes of two different rules. As can be seen in Figure 6, the object detection identified the region of interest of a rule and the confidence score of the prediction.

[0042] In order to achieve the goal of complete automation, a second recognition step was necessary, in this case

[0043] OCR, to compare the values on the design drawings with the rule definitions; the input of this step is also an image of a view extracted from the design drawing. OCR is a subfield of computer vision which extracts and locates characters in images; this tool is very sensitive to the type of character, clarity and size of the characters with respect to those on which it was trained.

[0044] Moreover, a very common problem with existing OCR tools is the dependence thereof on the horizontal orientation of text. In the drawings, the text is not only oriented horizontally, but in most cases it is inclined with an angle ranging from 0 to 360 degrees. This feature and the relatively rare presence of special symbols make it more difficult for common OCR tools to automatically encode the necessary values. Although OCR models are available which are capable of managing all the problems indicated, unlike other ready-made solutions, a fine- tuning of these models was preferred to take into account the variability of the characters with respect to the training set on which they were initially built.

[0045] Therefore, a CRAFT (Character Region Awareness for Text

[0046] Detection)-based OCR model was trained using the business drawing dataset described above. Figure 7 demonstrates how this invention is capable of locating texts in a technical drawing view regardless of the orientation of the text. The boxes indicate the identification of the text by the method.

[0047] The results of the two previous steps were combined using a specific process defined for the preferred use case, as described above in the third and fourth steps of the method, also with reference to Fig. 8.

[0048] Practical experiments of this invention were carried out with an initial dataset of 15 design drawing documents available in PDF, CATDrawing and DXF formats.

[0049] The PDF format was chosen as the preferred data format to use for this invention, as it was the best data format to work with for the use cases of greatest interest.

[0050] The object detection step provided an overall accuracy of 52% correct predictions and 33% false positives, a result considered good considering the low amount of data available for the model training. In particular, for one of the test rules, 53% of true prediction was documented, while 18% false positives and

[0051] 29% false negatives, while for a different test rule 36% true prediction was documented, while 64% false positives. For a third test rule, 50% true predictions and 50% false positives were recorded and for a fourth test rule 33% true predictions and 67% false negatives.

[0052] The OCR instead gave a result of 56% correct predictions and 0% false positives, but 23% false negatives and 20% characters detected with incorrect values.

[0053] The user's interaction with the system described in this section occurs through a graphical interface component which allows the user to interact with the system, providing a design drawing document as input.

[0054] With the steps described above, the system produces a result if the region of interest located on the design drawing and the values encoded by the machine comply with the set of predefined design rules, as shown in the flow diagram 300 in Figure 3. The graphical interface allows the user to upload the document of interest and perform the calculation in 310 . In a specific implementation, the document of interest was prepared with Catia software 320.

[0055] This GUI allows the user to load a design drawing document, define the type thereof, and returns as output if all necessary rules for such a type (e.g., a brake caliper) are depicted in the drawings and comply with the rule definitions. In fact, in 330 a first verification of compliance of the document is performed by virtue of the present invention. In step 340, the process goes on according to the verification result: if not all the rules have been respected, the process returns in 341 to the starting point 310 for (manual) correction of the document, otherwise the flow 342 leads to a second manual, optional verification.

[0056] In fact, the user interaction process allows an optional minimum human involvement, by virtue of a step

[0057] 350 of verifying the design drawing after the system has already provided an output. This intervention allows reducing the time necessary for design engineers to obtain feedback on the correctness of their drawing, allowing them to redesign more quickly if necessary. Such a second verification is all the less important the more the algorithm of the first verification is accurate.

[0058] Here too, in step 360, the process goes on according to the result of the verification: if not all the rules have been respected, the process in 361 returns to the starting point 310 for document correction, otherwise the flow leads to the end 370 of the process.

[0059] Each new document passes through the system and goes through all the processes described in this section

[0060] (except possibly the optional ones); to further simplify, Figure 4 shows the steps that each new upload goes through to serve the user with an output. In step

[0061] 410, a design drawing is obtained, in step 420 it is transformed into an image by separating the different views, in step 430 the image analysis is carried out, which is divided into two branches. In the first branch

[0062] 440 an object detection related to the various rules is carried out, in the second branch 450 an OCR process is carried out. The results of the two branches are used for the verification, which provides a verification result in 460.

[0063] As also described in Figure 2, an implementation of the system 200 of the present invention can include an

[0064] Al server 210 which detects objects in 211, recognizes text objects in 212, and verifies compliance with the rules in 213.

[0065] It is also possible to display a graphical interface

[0066] 220 to the user with two connections, one with the Al server and one with the data server 230, in order to carry out the process on a new document by passing it to the Al server and see the previously loaded design documents and the results thereof, respectively. The graphical interface comprises a frontend 221 and a backend 222. The data server 230 comprises an image archive 231 and a prediction database 232.

[0067] A different embodiment of the method involves a computer system configured to perform the following steps:

[0068] Input provision: the system is provided with a technical drawing as a series of images. This drawing can include various symbols, dimensions, and other graphical elements essential for engineering or production purposes.

[0069] Rule set definition: a rule set for the technical drawings is provided, which can include tolerances, dimensional requirements, or layout standards specific to the intended application, such as vehicle brake components.

[0070] Al object recognition: the system uses an Al model, specifically a neural network trained on technical drawings, to identify and locate non textual objects within the drawing. The Al model is optimized to distinguish between different components, such as brake calipers, cylinders and pistons. The objects are associated with the relevant rules for compliance verification.

[0071] OCR character recognition: the system uses an

[0072] OCR algorithm adapted for technical drawings to recognize and extract numerical characters

[0073] (e.g., dimensions) from the drawing. The OCR can be optimized to manage character sets and sizes commonly used in automotive engineering documents.

[0074] Character association with objects: each recognized numerical character is associated with the position of a respective object identified by the Al. For example, a dimension associated with a hole of the caliper or the distance of the mounting points is connected accordingly.

[0075] Compliance verification: the system compares the extracted numerical values with the values of the rules, considering the specific predefined tolerances for the brake components of the vehicles. For example, it verifies if the caliper dimensions fall within the tolerance levels required for safe and efficient brake operation.

[0076] Correction of errors and iteration: if discrepancies are identified, the system automatically corrects (for example based on a second Al algorithm appropriately trained based on the result of the above comparison, or based on the numerical difference between the compared values) the technical drawing based on the violations of the rules (how large the violation is, or how far it deviates numerically from the rules). Corrections can include adjustment of dimensions, such as caliper thickness or hole diameter. The correction process uses trained machine learning algorithms which adapt to past errors, improving accuracy with repeated iterations .

[0077] Alternatively to the last step, there can be a manual correction of the drawings, then repeating the steps listed to verify if the corrected drawing now meets all or almost all of the requirements. The system iteratively processes the drawing until compliance is reached or the drawing meets a predefined accuracy threshold. This can apply to all the embodiments of the present description.

[0078] Such a threshold can be predefined by the user.

[0079] As an optional further step, once compliance has been achieved, the final technical drawing can be transmitted directly to the electronic devices of a production plant of the device referred to in the technical drawing (s), and the device can be produced by means of the plant.

[0080] Alternatively, the device can be produced in any possible manner based on the technical drawings.

[0081] In the case of a system applied specifically to the production of brake calipers for vehicles, by means of this option of the invention, it is ensured that the technical drawing meets all safety and performance standards. The process includes the transmission of the compliant technical drawing (s) to the related production equipment, such as a CNC machine used for the production of the caliper mold. The equipment receives the compliant drawing, ensuring the precise execution of the specifications .

[0082] In an optional specific aspect of the invention, the system can record data from the caliper production process, including any deviations. These data are fed back into the Al algorithm to improve future compliance verifications, allowing the system to continuously improve.

[0083] Such aspects can conveniently be applied to devices other than brake calipers.

[0084] These further embodiments offer several technical advantages, including:

[0085] Greater accuracy: Al-based object recognition methods and die adjustment improve precision in brake caliper production, ensuring components meet safety standards.

[0086] Seamless integration with production systems: the method's ability to interface directly with

[0087] CNC equipment and other production machinery ensures the accuracy and efficiency of the entire production process.

[0088] As an alternative to the preferred implementation, the following solutions can also be utilized.

[0089] A first possible different implementation consists of a different architecture which can effectively act as a plugin for the 3D drawing software. This implementation strategy is based on the use of computational resources, both on-site and in the cloud, already used for 3D drawing software. The primary goal is to provide users with near real-time functionality, significantly reducing the time taken to correct design rule violations. By taking advantage of data and features derived from 3D drawing software, the Al-driven plugin system cannot only validate existing design rules, but also improve optical character recognition (OCR) results in real time. In a deeper integration scenario, it can even obviate the need for a dedicated OCR module, thus simplifying the architecture of the Al system.

[0090] A different approach for the modelling part could be to attempt a holistic view of image analysis. To improve the model's performance, it could be trained to recognize blocks containing both text characters and design elements. This approach offers several advantages, such as reducing model dependence on OCR (Optical Character

[0091] Recognition) components and a more streamlined software architecture. It is widely recognized in the literature that attributing correct context to an image significantly improves the performance of artificial intelligence models.

[0092] Still on the modelling front, an alternative approach can involve the use of state-of-the-art data augmentation techniques for the entire package of design drawings. Given the advances in generative Al, model performance can be improved by generating synthetic data drawings and then applying data augmentation to solve the inherent limitations of the dataset. These limitations mainly stem from various constraints, such as the limited number of drawings produced by the company during the year and the difficulty of finding freely available public datasets online. By adopting this approach based on "synthetic data", the model can be instructed on various borderline cases which it could encounter and understand, thus strengthening the capabilities thereof. From further tests carried out, it is possible to extend the invention to verify the rules also on 3D CAD drawings and 2D design documents.

[0093] Expanding the capabilities of this innovation involves implementing a system which can effectively validate design rules not only within 2D design documents, but also in the complex domain of 3D CAD drawings. To achieve this extension, it would probably be necessary to develop a specialized object detection model, rigorously trained to recognize and evaluate elements within the 3D models.

[0094] Since design documentation is often available in both 3D and 2D formats, employing a comprehensive approach which comprises both representations can significantly raise quality control standards in the field. By ensuring that design rules are strictly respected in both 2D and 3D design document s, this extended functionality can contribute to greater accuracy and reliability of engineering processes.

[0095] The artificial intelligence tool of the invention can be made accessible by means of general devices such as laptops, desktops and smartphones through a graphical interface which allows designers to self-verify compliance with design rules in their technical drawings, increasing the accuracy and quality of the technical drawing and reducing design time.

[0096] Two or more of the parts (elements, devices, systems) described above can be freely associated and considered as a part kit according to the invention. Preferred embodiments have been described above and variations of the present invention have been suggested, but it should be understood that those skilled in the art may make modifications and changes without departing from the related scope of protection, as defined by the appended claims.

Claims

AMENDED CLAIMS received bythe InternationalBureauonApril1st,2025 (01.04.2025)CLAIMS1. A computer-implemented method (400) for the automatic verification of compliance of a technical drawing of one or more devices and the subsequent production of the one or more devices, comprising the execution of the following steps:A. obtaining (410) a technical drawing as a series of images (420);B. obtaining a set of technical drawing rules;C. recognizing and locating (440) one or more non- textual objects within said technical drawing by means of a first trained artificial intelligence algorithm, each non-textual object being associated with a respective technical drawing rule;D. recognizing (450), by means of an OCR algorithm, one or more numerical characters within said technical drawing;E. associating (450) each of said one or more numerical characters to a position of a respective object of said one or more non-textual objects;F. verifying (460) the compliance of the technical drawing by calculating a difference between values of said one or more numerical characters of stepD and the corresponding values in said set of technical drawing rules , taking into account predefined tolerances;G. providing (410) a correct technical drawing based on the difference of step F;H. repeating steps A to G until said difference of step F is below one or more predefined thresholds, finally providing a compliant technical drawing;I. producing said device in a production line according to the compliant technical drawing of step H.

2. A method according to claim 1, wherein in step G a second artificial intelligence algorithm trained for the correction of technical drawings based on said difference is used.

3. A method according to claim 2, wherein said second trained artificial intelligence algorithm is also trained based on data from said production line acquired during step I.

4. A method according to one of claims 1 to 3, wherein in step I the compliant technical drawing is digitally received by a processing unit of said production line.

5. A method according to one of claims 1 to 4, wherein step E is carried out, for each object from said one or more non-textual objects, by: selecting an object from said one or more non- textual objects; associating to said object a subset of said one or more numerical characters whose respective distance from a center of said object is within a predefined threshold distance.

6. A method according to claim 5, wherein said predefined threshold distance is equal to an average of distances falling within a ninety-ninth percentile of a distribution of distances from centers of additional one or more non-textual objects to respective numerical characters pre-associated by a user in a set of training technical drawings.

7. A method according to one of claims 1 to 6, wherein the artificial intelligence algorithm is a convolutional neural network, wherein the technical drawings in stepA have been rasterized (420) into a series of bitmap images.

8. A method according to one of claims 1 to 7, wherein steps C and D are carried out simultaneously, by means of an artificial intelligence algorithm trained to recognize textual objects and numerical characters at the same time.

9. A method according to one of claims 1 to 8, wherein the technical drawing is two-dimensional.

10. A method according to one of claims 1 to 9, wherein the technical drawing is a technical drawing of an element of a wheeled-vehicle braking system.

11. A computerized system (200) for the automatic verification of compliance of a technical drawing of one or more devices and the subsequent production of the oneor more devices, comprising: a data server (230) configured to receive the technical drawing of step A and comprising a database of technical drawing rules according to step B of the method according to one of claims 1 to 10, and an artificial intelligence server (210) configured to carry out steps C to G of the method according to one of claims 1 to 10; a graphical user interface (221,222) connected to data server (230) and the artificial intelligence server (210); a production line configured to carry out step I of the method according to one of claims 1 to 10.

Images