Method, device, apparatus and medium for generating digital models from drawings

Abstract

Embodiments of the present disclosure relate to methods, devices, equipment and computer-readable storage media for generating digital models from architectural or non-architectural design drawings. The method includes: extracting a first data set from a first image of a drawing of a design target, the first data set at least including the position, geometry and type of one or more types of components of the design target; extracting a second data set from a second image of the drawing with a higher resolution, the second data set including the position and geometry of one or more types of components of the design target, and the position and geometry of the corresponding components included in the second data set have higher accuracy than the position and geometry of the corresponding components included in the first data set; correcting the first data set at least based on the second data set; and generating a digital model of the design target based at least on the corrected first data set. The scheme of the present disclosure provides a mold remodeling method combining low-resolution and high-resolution images and introduces a feedback mechanism, thereby reducing the computing resources and manpower consumption used for mold remodeling while improving the accuracy of the design drawing remodeling.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of engineering design, and more particularly, to a method for generating a digital model from architectural or non-architectural design drawings. Background Art

[0002] Digital tools represented by Building Information Modeling (BIM) are increasingly being used in the construction industry, and are expected to become standard features of the industry in the future. Compared to traditional two-dimensional design drawings, BIM is more intuitive, contains a variety of processable information (such as building components, plumbing pipes, and electromechanical, etc.), and can be used for simulation and simulation, making the entire design process more centralized, standardized, and traceable.

[0003] However, since designers are not yet fully accustomed to adopting BIM, there are only a few drawings generated by forward design using BIM software in architectural design. The vast majority of BIMs are to draw design drawings first, and then convert the drawings into BIM in the modeling software based on the understanding of the drawings. This process is called BIM re-molding. Conventional BIM re-molding is usually performed manually based on the understanding of the drawings. Manual re-molding takes a lot of time (for example, for a project of 50,000 square meters, the re-molding time may take up to 2 months), and it will also bring a high economic burden to related companies (in some cases, it can be as high as 1 to 2 yuan per square meter). Therefore, automated re-molding of design drawings is of great significance and value to the construction industry. Summary of the invention

[0004] In a first aspect of the present disclosure, a method for generating a digital model from an architectural or non-architectural design drawing is provided, comprising: extracting a first data set from a first image of a drawing of a design target, the first data set comprising at least positions, geometries, and types of components of one or more types of the design target; extracting a second data set from a second image of the drawing with a higher resolution, the second data set comprising positions and geometries of components of one or more types of the design target, the positions and geometries of the corresponding components included in the second data set having a higher accuracy than the positions and geometries of the corresponding components included in the first data set; correcting the first data set at least based on the second data set; and generating a digital model of the design target based at least on the corrected first data set.

[0005] In a second aspect of the present disclosure, there is provided an apparatus for generating a digital model from architectural or non-architectural design drawings, comprising: an extraction module configured to extract a first data set from a first image of a drawing of a design target, the first data set comprising at least positions, geometries and types of components of one or more types of the design target, and to extract a second data set from a second image of the drawing having a higher resolution, the second data set comprising positions and geometries of components of one or more types of the design target, the positions and geometries of the corresponding components included in the second data set having higher accuracy than the positions and geometries of the corresponding components included in the first data set; a correction module configured to correct the first data set based at least on the second data set; and a generation module configured to generate a digital model of the design target based at least on the corrected first data set.

[0006] In a third aspect of the present disclosure, an electronic device is provided, comprising one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to the first aspect of the present disclosure.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0008] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, which are used by a processor to implement the method according to the first aspect of the present disclosure.

[0009] Please note that the present summary is provided to introduce a selection of concepts in a simplified form, which concepts will be further described in the following detailed description. The present summary is not intended to identify key features or essential features of the present disclosure, nor is it intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of example embodiments of the present disclosure in conjunction with the accompanying drawings, in which:

[0011] Figure 1 A schematic diagram illustrating an example environment in which embodiments of the present disclosure can be implemented;

[0012] Figure 2 A flowchart of a method for generating a digital model from a design drawing according to some embodiments of the present disclosure is illustrated;

[0013] Figure 3A flowchart illustrating a method of extracting a data set for generating a data model from an image of a design drawing according to some embodiments of the present disclosure;

[0014] Figure 4 shows a non-limiting example interface for uploading an image of a design drawing according to some embodiments of the present disclosure;

[0015] Figure 5 An exemplary process of generating a digital model from images of a plurality of design drawings according to some embodiments of the present disclosure is shown;

[0016] Figure 6 A schematic block diagram showing an apparatus for generating a digital model from a design drawing according to some embodiments of the present disclosure; and

[0017] Figure 7 A schematic block diagram of a device that can be used to implement an embodiment of the present disclosure is shown.

[0018] Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] The term "including" and its variations used in this document are open-ended, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment". The relevant definitions of other terms will be given in the following description.

[0021] The term "re-modeling" used in this article refers to the process of converting the drawn engineering design drawings into digital models. Taking BIM as an example, the architectural design institute can convert the two-dimensional drawings into three-dimensional building models through BIM re-modeling, which is used to visually check and optimize the design and improve work efficiency. The construction party can also timely re-model the latest two-dimensional drawings of the design institute into three-dimensional building models to avoid information deviation caused by inconsistent drawings and models, and use the rich information in the raw model to optimize the construction process management, including but not limited to timely adjusting the quantity of materials, adjusting the project schedule and construction sequence, controlling construction costs, and analyzing cash usage. The building operation and maintenance party can also generate BIM models based on the drawings, and combine sensors and Internet of Things technologies to achieve real-time and visual monitoring of the building status to reduce operation and maintenance costs and improve operation and maintenance efficiency.

[0022] Automatic mold remodeling requires extracting different types of components and annotation text from drawings. However, since most drawings do not name layers or arrange layer contents in a standard way, the same layer may contain a variety of different components or annotation text, so it is impossible to separate different types of components and annotation text by simple layer matching. Some modeling software attempts to use artificial intelligence (AI) methods such as deep learning and convolutional neural networks to extract component information from drawings to automate the mold remodeling process (such as BIM mold remodeling). However, training AI models for mold remodeling requires manual annotation of a large number of drawings, which consumes a lot of manpower. In addition, due to the large size of engineering drawings, it takes a long time to remodel them using computationally intensive AI methods and consumes a lot of GPU resources. In addition, due to the noise of engineering drawings and the limitations of automatic recognition models, the accuracy of the models converted by AI models often cannot meet engineering requirements.

[0023] To at least partially solve the above problems and other potential problems, the embodiments of the present disclosure propose a method for generating digital models from design drawings. The scheme selects lower resolution images for computationally intensive drawing recognition by the AI ​​model, thereby increasing the speed of AI component extraction. Then, pixel-level detection of super-resolution restored images or high-scale images is used to interactively adjust the identified data to improve the accuracy of the identified data, thereby generating a high-precision digital model.

[0024] In addition, the solution also introduces a feedback mechanism when training the AI ​​model for drawing recognition. The feedback mechanism uses less annotated data in the initial training set of the AI ​​model, and iteratively uses the data with improved accuracy in the remodeling process using the solution to improve the training data set in order to retrain the AI ​​model. In this way, the workload of manual annotation can be reduced, and the noise impact in the drawing annotation can be reduced and the quality of the AI ​​model can be improved during the iterative training process, so that the model can ultimately effectively meet the requirements of engineering use.

[0025] For illustrative purposes, the solution will be described in the context of architectural drawing mold making in this disclosure, but it should be understood that the same technology is also applicable to other similar mold making scenarios, such as industrial design mold making, etc.

[0026] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Environment 100 includes computing device 101. Although shown as a single device, computing device 101 may also be multiple devices, a virtual device, or any other form of device suitable for implementing embodiments of the present disclosure.

[0027] For the sake of illustration, Figure 1 , different images in the computing device 101 are also shown, namely, the first image 110 and the second image 130. The image 110 and the second image 130 may be images of the same design drawing with different resolutions. In some embodiments, as will be described later, the second image 130 may be converted from the first image 110. According to some embodiments of the present disclosure, the computing device 101 may extract the first data set 120 from the first image 110 and the second data set 140 from the second image 130. In addition, using the method according to the embodiment of the present disclosure, the computing device may also generate a digital model 150 of the design target using the first data set 120 and the second data set 140. This will be described in more detail below in conjunction with other drawings.

[0028] It should be understood that the environment 100 is described for illustrative purposes only and does not imply any limitation on the scope of the present disclosure. The environment 100 may also include Figure 1 Furthermore, the embodiments of the present disclosure may also be applied in environments different from environment 100.

[0029] Figure 2 1 is a flowchart of a method 200 for generating a digital model from a design drawing according to some embodiments of the present disclosure. The method 200 may be executed, for example, by the computing device 101 (more specifically, a processor of the computing device 101). Figure 1 The method 200 will be described in detail.

[0030] At block 210, the computing device 101 extracts a first data set 120 from the first image 110. The first image 110 is an image of a drawing of a design target (such as an architectural design drawing), and the first data set 120 includes at least the location, geometry, and type of one or more types of components of the design target. Taking the architectural design drawing as an example, the computing device 101 can extract the geometry, location, and associated annotation information of components of types such as walls, doors, and windows from the architectural drawing of the design target, and can extract the geometry, location, and associated annotation information of components such as walls, beams, and columns from the structural drawing of the design target.

[0031] In various embodiments, the first image 110 may have any suitable format, including but not limited to the image format DWG format used by computer-aided design (CAD), portable document format (PDF), portable network graphics (PNG) format, joint photographic experts group (JPG) format, and bitmap (BMP) format, etc. In various embodiments, taking architecture as an example, the design drawing of the first image 110 may be a plan view, an elevation view, or a section view, etc.

[0032] When the first image 110 is, for example, an image of a non-standardized drawing containing a variety of different components or annotated text, the computing device 101 can use a trained recognition model (such as an AI model) to extract the first data set 120 from the first image 110. In some embodiments, the computing device 101 uses a training set obtained by annotating the component information in the design drawing to train the recognition model. Data extraction based on AI models and the like usually requires a lot of resources and time. Therefore, in some such computationally intensive embodiments, the first image 110 used to extract the first data set 120 can have a suitable lower resolution. The resulting first data set 120 thus includes less accurate geometric shapes and position information for further processing.

[0033] Combined with the following Figure 3 The extraction of the first data set 120 is described in more detail. Figure 3 1 is a flowchart of a method 300 for extracting a data set for generating a digital model from an image of a design drawing according to some embodiments of the present disclosure. The method 300 may be executed, for example, by the computing device 101. Figure 1 The method 300 will be described in detail.

[0034] At block 310, the computing device 101 may extract a component data set from the first image 110, wherein the component entries in the component data set include at least the location, geometry, and type of the component. In some embodiments, the computing device 101 may utilize a trained AI model to holistically identify and extract the locations, geometry information, and types of corresponding instances (e.g., walls) of multiple types of component instances included in the drawing.

[0035] At block 320, computing device 101 may extract an annotation data set from first image 110, wherein the annotation entry in the annotation data set includes at least the location and text content of the annotation. For example, the text content may include the dimension value of the component and additional information about various properties of the component. In some embodiments, computing device 101 may utilize a trained AI model to holistically identify and extract the annotation data included in the drawing.

[0036] At block 330, the computing device 101 may associate each component entry in the component data set with an annotation entry corresponding to the same component in the annotation data set, thereby generating a first data set 120. In some embodiments, the computing device 101 may match the component with the corresponding annotation text based on a relative position relationship and a connection line between them, thereby obtaining a first data set 120 including component position, range, size, type, and / or other information (such as reinforcement).

[0037] It should be understood that the present disclosure describes the component data set and the annotation data set only for the purpose of convenience of explanation, and is not intended to limit the extracted data to a specific format or form. For example, in some implementations, the component data set and the annotation data set can be stored in the same data set for further processing. It should also be understood that the method of the present disclosure is not limited by the specific order of extracting the component data and the annotation data. Figure 2 The component data and annotation data may be extracted in the reverse order shown, in parallel, or in any other suitable order.

[0038] Continue to refer Figure 2 At block 220, the computing device 101 extracts a second data set 140 from the second image 130 of the drawing, the second data set 140 including positions and geometries of one or more types of components of the design target. The resolution of the second image 130 is higher than that of the first image 110. Thus, the positions and geometries of the corresponding components included in the second data set 140 have higher accuracy than the positions and geometries of the corresponding components included in the first data set 120. Therefore, the correction data set 140 can be used as base data to correct the less accurate geometry and position information in the first data set.

[0039] In some embodiments, the computing device 101 may use different corresponding methods to improve the image quality of the first image 110 in different formats to obtain the second image 130. In some embodiments, for example, when the first image 110 is in a format such as PDF or DWG, the computing device 101 may obtain the second image 130 by increasing the output image magnification ratio of the first image 110 or performing super-resolution restoration on the first image 110. In some embodiments, for example, when the first image 110 is in a PNG format, the computing device 101 may perform super-resolution restoration on the first image 110 to obtain the second image 130.

[0040] In some embodiments, the computing device 101 may extract the second data set 140 from the second image 130 using a pixel-level shape recognition method. For example, the computing device 101 may use methods such as straight line detection and arc detection at the pixel level to detect some unique features of a certain type of component (for example, in an architectural drawing, a wall may be a double straight line, and a door may have an arc, etc.), thereby extracting the accurate position and geometric shape of the component. In some embodiments, the computing device 101 may use the component information in the first data set 120 to find the approximate area range of the corresponding component in the second image 130. Then, the computing device 101 may perform shape recognition detection on the pixels within the component area range, thereby obtaining the position and geometric shape information of the component with higher accuracy.

[0041] At block 230, the computing device 101 corrects the first data set 120 based on at least the second data set 140. The computing device 101 may use the information in the second data set 140 acquired at block 220 as reference information for correction. In some embodiments, the computing device 101 may align the position and geometry of the component in the first data set 120 with the position and geometry of the corresponding component in the second data set 140, that is, modify the position and geometry of the component to the geometry of the corresponding more accurate position in the second data set 140.

[0042] In some embodiments, after correcting the first data set 120 using the second data set 140, the computing device 101 may use information such as the drawing scale to convert the values ​​representing the position and geometry of the component in the first data set 120 into corresponding values ​​in the physical coordinate system to obtain the physical size of the component. The value of the physical size obtained in this way is usually not an integer, which is usually inconsistent with the actual situation of the original design drawing. In some embodiments, the computing device 101 may perform modulus rounding on the converted physical size based on one or more specified moduli, and use the rounded size value as a further correction result. For example, when the design target is a building, the components in the drawings are usually designed as multiples of certain standard size units (i.e., building moduli). In this case, the computing device 101 may modify (i.e., round) the size value (e.g., side length) of the component to a multiple of the corresponding building modulus closest to the size value.

[0043] Unlike AI model-based extraction, which often requires GPU resources, the steps of box 220 and box 230 usually use CPU resources, thereby achieving improved accuracy while saving GPU resources.

[0044] In some embodiments, during the correction of the first data set 120, the computing device 101 may visualize at least one of the first data set 120 and the second data set 140 in an interactive manner on the user interface to achieve interactive adjustment. For example, the computing device 101 may present the automatically identified and / or matched component instances in the form of straight line segments, polyline segments, rectangular boxes, polygonal boxes, and circles on the interactive interface, so that the user can manually edit the displayed shapes to fine-tune these shapes. This can ensure the accuracy of information extraction and avoid the problem of inaccurate automatic identification and matching of AI models.

[0045] At block 240, the computing device 101 generates a digital model of the design target based at least on the corrected first data set 120. For example, the computing device 101 may convert the corrected first data set 120 into a model file format supported by common 3D modeling software, such as RVT, OBJ, and FBX, as a digital model or a portion thereof.

[0046] It should be understood that the method 200 may also include additional actions not shown. For example, the system 101 may convert the first image 110 into a processable file format before extracting data using the first image 110. In some embodiments, the system 101 may convert the first image 110 into a picture format that the AI ​​model can process based on the file format indicated by the user or identified by the file extension or file header. In some embodiments, the system 101 may output the generated model file to the user side for the user to browse or adjust using general modeling software (such as BIM modeling software).

[0047] Figure 4 A non-limiting example interface 400 according to some embodiments of the present disclosure is shown, and the example interface 400 is used to select an image of a design drawing, which can then be used to generate a digital model of the design drawing according to a method (e.g., method 200) of an embodiment of the present disclosure. For example, the example interface 400 can be Figure 1 The display of the computing device 101 of FIG. 100 is displayed to guide the user to upload an image of the two-dimensional drawing. The image can then be uploaded by the computing device 101, for example. Figure 2 The method 200 (eg, as the first image 110) is used. Figure 1 The example interface 400 is described in detail.

[0048] The example interface 400 includes a button 410. The computing device 101 can enable (e.g., a user of the computing device 101) to select an image of a design drawing to be used to generate a digital model in response to clicking the button 410, and the path of the selected image can be displayed in the text box 405. The image can be a common file format such as DWG, PDF, PNG, JPG, or BMP. In addition, the design drawing can be a drawing of a certain type in a plurality of types. Taking architectural drawings as an example, the design drawings can be floor plans, elevations, or sections from different perspectives. In some embodiments, in response to clicking the button 410, the computing device 101 can support batch uploading of a group of images. It should be understood that using a group of images (e.g., images of different parts of the design drawing) to generate a digital model (e.g., as the first image 120)) does not deviate from the principles and scope of the present disclosure.

[0049] The example interface 400 also includes a drop-down box 420 and a drop-down box 430. In some embodiments, during the selection of an image using the example interface 400, the computing device 101 may use the format (e.g., DWG, PDF, PNG, JPG, BMP, etc.) selected in the drop-down box 420 (e.g., by the user) as the format of the selected image for subsequent operations (e.g., converting the selected image into a format that can be processed by the AI ​​model, etc.). In some embodiments, during the selection of an image using the example interface 400, the computing device 101 may use the type (e.g., plan view, elevation view, and section view, etc.) selected in the drop-down box 430 (e.g., by the user) as the type of the selected image for subsequent operations (e.g., selecting a suitable extraction method, etc.). In this way, the problem of the file extension not being consistent with the actual format or the situation where the annotation text in the drawing does not match the perspective taken by the drawing can be avoided.

[0050] The method (eg, method 200 ) according to an embodiment of the present disclosure may be used to generate a digital model from a plurality of different types of design drawings (eg, drawings taken from different perspectives) of a design target. Figure 5 An example process 500 for generating a digital model from a plurality of design drawings according to some embodiments of the present disclosure is shown. The example process 500 may be performed by a computing device 101, and will be referred to below. Figure 1 An example process 500 is described.

[0051] exist Figure 5 , image 510-1, image 510-2, and image 510-3 (collectively referred to as image 510) may be different types of drawings of a design target (e.g., they may be plan views, elevation views, and cross-section views, respectively). In the example process 500, the computing device 101 acquires a data set 520-1 from the image 510-1 in an extraction and correction subprocess 505-1. For example, the computing device 101 may use the image 510-1 as the first image 110, and use the method 200 to extract and correct the first data set 120-1 as the data set 520-1. In a similar manner, the computing device 101 may acquire a data set 520-2 from the image 510-1 in an extraction and correction subprocess 505-2, and acquire a data set 520-3 from the image 510-3 in an extraction and correction subprocess 505-3.

[0052] The computing device 101 then integrates the extracted and corrected data sets 520-1, 520-2, and 520-3 in the data integration subprocess 515 to obtain integrated data 525. For example, the computing device 101 may perform mutual verification on duplicate data and remove redundant data to obtain the integrated data 525. The integrated data 525 is then used by the computing device 101 to generate a digital model 550 in the model generation subprocess 535. For example, the computing device 101 may convert the integrated data 525 into a model file format (such as RVT, OBJ, and FBX, etc.) supported by the modeling software as the digital model 550.

[0053] It should be understood that without departing from the scope of the present disclosure, at least some of the sub-processes in process 500 (e.g., extraction and correction sub-process 505-1, extraction and correction sub-process 505-2, and extraction and correction sub-process 505-3) can be performed sequentially, in parallel, or in other suitable sequences. In addition, more or fewer images 510 can also be used in the example process 500, depending on the specific application scenario.

[0054] The methods and processes described above can achieve automatic or semi-automatic demolding from design drawings of any format (standardized and non-standardized). On the basis of automatically extracting and correcting the drawing information, very accurate data can be obtained to automatically generate a digital model with only a small amount of manpower. Compared with conventional automatic demolding solutions, the solution disclosed in the present invention has a wider scope of application and can ensure high demolding accuracy even when the automatically extracted building information is not completely accurate. Taking BIM demolding as an example, architectural design institutes, construction parties, and operation and maintenance parties can use the solution disclosed in the present invention to automate the BIM demolding process so as to use fewer resources and obtain the BIM they need in a more timely manner, thereby improving their work efficiency.

[0055] After obtaining data for generating a digital model using the method described above, some embodiments of the present disclosure further include a process for providing feedback on the automatic extraction of drawing data. The feedback process may be performed by the computing device 101, as described below. Figure 1 Let’s describe the feedback process in detail.

[0056] In some embodiments, the computing device 101 may use a first recognition model to extract a first data set 120 from a first image 110. In some embodiments, the first recognition model is determined using a training set obtained by manually annotating component information in a design drawing. For example, information suitable for training a first recognition model in a first image 110 of a drawing may be manually annotated, and the information may be stored as a training set. The type of information that needs to be annotated depends on the specific application, and the scheme of the present disclosure is not limited thereto. The training set may then be used to train the first recognition model. When training the first recognition model, the system 101 may use a smaller training set that only annotates a portion of the component information in the drawing in order to determine the first recognition model more quickly.

[0057] After extracting the first data set 120 using the first recognition model, the computing device 101 may use the steps in the method according to an embodiment of the present disclosure (e.g., box 220 and box 230 of method 200) to automatically or semi-automatically correct the first data set 120, and use the corrected first data set 120 to determine a second recognition model for extracting components from the image. In some embodiments, the computing device 101 may use the corrected first data set 120 as a training set to retrain the recognition model, thereby obtaining a second recognition model. In some other embodiments, the computing device 101 may use the corrected first data set 120 and an automatic annotation algorithm to automatically generate new annotation data for training from the first image 110. Then, the computing device 101 may use the new annotation data as a training set to retrain the recognition model after optionally making a small amount of manual adjustments, thereby obtaining a second recognition model.

[0058] Since the quality of the training set used to train the second recognition model is improved during the feedback process, the quality of the second recognition model trained using the higher quality training set will be higher than that of the first recognition model. It should be understood that the computing device 101 can iteratively perform the above feedback process to obtain a recognition model with desired quality for use in digital model generation (e.g., BIM remodeling) according to an embodiment of the present disclosure.

[0059] Figure 6A schematic block diagram of an apparatus 600 for generating a digital model from a design drawing according to some embodiments of the present disclosure is shown. The apparatus 600 may include an extraction module 610, a correction module 620, and a generation module 630, wherein: the extraction module 610 is configured to extract a first data set from a first image of a drawing of a design target, the first data set at least including the position, geometry, and type of one or more types of components of the design target, and extract a second data set from a second image of the drawing with a higher resolution, the second data set including the position and geometry of one or more types of components of the design target, and the position and geometry of the corresponding components included in the second data set have higher accuracy than the position and geometry of the corresponding components included in the first data set; the correction module 620 is configured to correct the first data set based on at least the second data set; and the generation module 630 is configured to generate a digital model of the design target based on at least the corrected first data set.

[0060] In some embodiments, the extraction module 610 extracts the first data set including: extracting a component data set from the first image, the component entries in the component data set including at least the location, geometry and type of the component; extracting an annotation data set from the first image, the annotation entries in the annotation data set including at least the location and text content of the annotation; and generating the first data set by associating each of the component entries in the component data set with an annotation entry corresponding to the same component in the annotation data set.

[0061] In some embodiments, the correction module 620 corrects the first data set including aligning the position and geometry of the corresponding components in the first data set with the position and geometry of the corresponding components in the second data set.

[0062] In some embodiments, the correction module 620 correcting the first data set also includes: converting the values ​​representing the position and geometry of the components in the aligned first data set into corresponding values ​​in the physical coordinate system to obtain the physical dimensions of the components; and performing modulus rounding on the physical dimensions of the components in the first data set based on one or more specified moduli.

[0063] In some embodiments, the extraction module 610 is further configured to acquire the second image by performing at least one of the following operations: increasing the output image magnification ratio of the first image; or performing super-resolution restoration on the first image.

[0064] In some embodiments, the extraction module 610 is further configured to use a pixel-level shape recognition method to obtain the position and geometry of the component in the correction image set from the second image.

[0065] In some embodiments, the correction module 620 is further configured to: during correction of the first data set, visualize at least one of the first data set and the second data set on a user interface in an interactive manner.

[0066] In some embodiments, the extraction module 610 is further configured to extract a first dataset of components of the design object from the first image of the design object using the first recognition model.

[0067] In some embodiments, the apparatus 600 further includes a training module 510 , and the training module 640 is configured to: use a training set obtained by annotating component information in a design drawing to determine a first recognition model therein.

[0068] In some embodiments, the training module 640 is further configured to use the corrected first data set to determine a second recognition model for extracting a data set of components from an image.

[0069] In some embodiments, the generation module 630 can convert the corrected first data set into a model file format supported by common 3D modeling software, such as RVT, OBJ, and FBX, as a digital model of the design target or a portion thereof. In some embodiments, the generation module 630 can integrate multiple corrected first data sets and convert them into a common model file format.

[0070] It should be understood that, depending on the specific implementation, the operations described above as being performed by a module of the apparatus 600 may also be performed by another module or multiple modules, and the operations described as being performed by multiple modules may also be performed by a single module, and the scope of the present disclosure is not limited thereto. It should also be understood that the apparatus 600 may also include Figure 6 Modules not shown in the figure, such as modules that perform common modeling operations such as data integration.

[0071] Figure 7 A block diagram of a computing device 700 capable of implementing multiple embodiments of the present disclosure is shown. Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0072] like Figure 7As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0073] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0074] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods described above (e.g., method 200). For example, in some embodiments, method 200 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the various methods described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the various methods described above in any other appropriate manner (e.g., by means of firmware).

[0075] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0076] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0077] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0078] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0079] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0080] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0081] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0082] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating a digital model from an architectural or non-architectural design drawing, comprising: Extracting a first data set from a first image of a drawing of a design object, the first data set being extracted by a trained first recognition model and comprising at least positions, geometric shapes and types of components of one or more types of the design object; extracting a second data set from a second image of the drawing having a higher resolution, the second data set including positions and geometries of the one or more types of components of the design target, the positions and geometries of the corresponding components included in the second data set having a higher accuracy than the positions and geometries of the corresponding components included in the first data set; correcting the first data set based at least on the second data set; generating a digital model of the design object based at least on the corrected first data set; as well as A second recognition model for extracting a dataset of components from an image is determined based on the corrected first dataset, the second recognition model being different from the first recognition model.

2. The method of claim 1, wherein extracting the first data set comprises: Extracting a component data set from the first image, the component entries in the component data set at least including a location, a geometric shape and a type of the component; Extracting a labeling data set from the first image, wherein the labeling items in the labeling data set at least include a labeling position and text content; as well as The first data set is generated by associating each of the component entries in the component data set with an annotation entry corresponding to the same component in the annotation data set.

3. The method of claim 1 , wherein correcting the first data set comprises: The position and geometry of the corresponding component in the first data set are aligned with the position and geometry of the corresponding component in the second data set.

4. The method of claim 3, wherein correcting the first data set further comprises: Converting the values ​​representing the position and geometric shape of the component in the aligned first data set into corresponding values ​​in a physical coordinate system to obtain the physical size of the component; as well as The physical dimensions of the components in the first data set are modulo-rounded based on one or more specified moduli.

5. The method according to claim 1, wherein the second image is acquired by performing at least one of the following operations: increasing the output image magnification ratio of the first image; or Super-resolution restoration is performed on the first image.

6. The method of claim 1, wherein the position and geometry of the components in the second data set are obtained from the second image using a pixel-level shape recognition method.

7. The method according to claim 1, further comprising: During correction of the first data set, at least one of the first data set and the second data set is visualized on a user interface in an interactive manner. 8 . The method according to claim 1 , wherein the first recognition model is determined using a training set obtained by annotating component information in a design drawing.

9. A device for generating a digital model from architectural or non-architectural design drawings, comprising: Extraction module, configured as extracting a first data set from a first image of a drawing of a design object, the first data set being extracted by a trained first recognition model and comprising at least positions, geometric shapes and types of components of one or more types of the design object, and extracting a second data set from a second image of the drawing having a higher resolution, the second data set including positions and geometries of the one or more types of components of the design target, the positions and geometries of the corresponding components included in the second data set having a higher accuracy than the positions and geometries of the corresponding components included in the first data set; a correction module configured to correct the first data set based at least on the second data set; A generating module, configured to generate a digital model of the design target based at least on the corrected first data set; as well as A training module is configured to determine a second recognition model of a dataset for extracting components from an image based on the corrected first dataset, wherein the second recognition model is different from the first recognition model.

10. The apparatus of claim 9, wherein extracting the first data set comprises: Extracting a component data set from the first image, the component entries in the component data set at least including a location, a geometric shape and a type of the component; Extracting a labeling data set from the first image, wherein the labeling items in the labeling data set at least include a labeling position and text content; as well as The first data set is generated by associating each of the component entries in the component data set with an annotation entry corresponding to the same component in the annotation data set.

11. The apparatus of claim 9, wherein correcting the first data set comprises: The position and geometry of the corresponding component in the first data set are aligned with the position and geometry of the corresponding component in the second data set.

12. The apparatus of claim 11, wherein correcting the first data set further comprises: Converting the values ​​representing the position and geometric shape of the component in the aligned first data set into corresponding values ​​in a physical coordinate system to obtain the physical size of the component; as well as The physical dimensions of the components in the first data set are modulo-rounded based on one or more specified moduli.

13. The apparatus according to claim 9, wherein the extraction module is further configured to obtain the second image by performing at least one of the following operations: increasing the output image magnification ratio of the first image; or Super-resolution restoration is performed on the first image.

14. The apparatus according to claim 9, wherein the extraction module is further configured to: A pixel-level shape recognition method is used to obtain the position and geometry of the components in the second data set from the second image.

15. The apparatus according to claim 9, wherein the correction module is further configured to: During correction of the first data set, at least one of the first data set and the second data set is visualized on a user interface in an interactive manner.

16. The apparatus according to claim 9, further comprising a training module, wherein the training module is configured to: The first recognition model is determined using a training set obtained by annotating component information in a design drawing.

17. An electronic device comprising: Memory and processor; The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1 to 8.

18. A computer-readable storage medium having one or more computer instructions stored thereon, wherein the one or more computer instructions are executed by a processor to implement the method according to any one of claims 1 to 8.

19. A computer program product comprising one or more computer instructions, wherein the one or more computer instructions are executed by a processor to implement the method according to any one of claims 1 to 8.

Images