Three-dimensional modeling generation method and related device

Abstract

The invention discloses a three-dimensional modeling generation method and a related device, and the method comprises the steps: firstly obtaining a building design drawing of a building structure of a target building in a two-dimensional plane through building elements, and then determining a plane image which reflects the graphic features of the building elements; according to the plane image, a type image comprising a type pixel area is obtained through a type recognition model, the type pixel area has a corresponding relation with building elements in the building design drawing based on the spatial relation between the type image and the building design drawing, and the element types of the building elements are identified through pixel colors. And finally, performing element type labeling on the building elements in the building design drawing based on the type drawing to obtain a labeled design drawing, and performing rollover processing on the labeled design drawing to generate three-dimensional modeling. According to the invention, element type identification of a graph level can be carried out based on the graph characteristics of the building elements in the plane image through the type identification model, the marking efficiency and accuracy of the building elements in the building design drawing are improved, and the turnover efficiency and precision are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, and in particular to a three-dimensional modeling generation method and related device. BACKGROUND

[0002] With the development of technology, the traditional two-dimensional plane building design drawing cannot meet the actual demand, and it is expected to adapt to new technical requirements by converting the two-dimensional building design drawing into three-dimensional modeling.

[0003] In the process of converting the two-dimensional building design drawing into three-dimensional modeling, the building elements involved in the building design drawing need to be determined, and the types of the building elements in the building plan are labeled to realize correct modeling. In related technologies, the types of building elements are labeled by manual methods, which is low in efficiency and difficult to ensure accuracy.

[0004] In order to improve the efficiency of converting the two-dimensional building design drawing into three-dimensional modeling, there is an urgent need for a three-dimensional modeling generation method that can accurately and efficiently complete the element type labeling of building elements. SUMMARY

[0005] To solve the above technical problems, the present application provides a three-dimensional modeling generation method and related device, which can identify the element types of the building elements in the plan image based on the graphic features of the building elements through a type identification model, improve the labeling efficiency and accuracy of the building elements in the building design drawing, and further ensure the modeling efficiency and accuracy.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In one aspect, the embodiments of the present application provide a three-dimensional modeling generation method, which comprises:

[0008] Obtaining a building design drawing of a target building, the building design drawing being used to identify the building structure of the target building in a two-dimensional plane through building elements;

[0009] Determining a plan image of the building design drawing, the plan image being used to reflect the graphic features of the building elements in the building design drawing;

[0010] According to the plan image, a type image corresponding to the building design drawing is obtained through a type identification model, the type image comprising a type pixel region, the type pixel region having a corresponding relationship with the building elements in the building design drawing based on the spatial relationship between the type image and the building design drawing, the type pixel region identifying the element type of the corresponding building element through pixel color, and different pixel colors corresponding to different element types;

[0011] Based on the type graph, the building elements in the building design graph are marked with element types to obtain a marked design graph;

[0012] Through the marked design graph, a three-dimensional modeling corresponding to the target building is generated.

[0013] In another aspect, the embodiments of the present application provide a three-dimensional modeling generation device, the device comprising: an acquisition module, a determination module, a marking module, a marking module and a generation module;

[0014] The acquisition module is configured to acquire a building design graph of a target building, the building design graph being used to identify the building structure of the target building in a two-dimensional plane through building elements;

[0015] The determination module is configured to determine a planar image of the building design graph, the planar image being used to embody the graphical features of the building elements in the building design graph;

[0016] The identification module is configured to obtain a type graph corresponding to the building design graph according to the planar image through a type identification model, the type graph comprising a type pixel region, the type pixel region having a corresponding relationship with the building elements in the building design graph based on the spatial relationship between the type graph and the building design graph, the type pixel region identifying the element type of the corresponding building elements through pixel color, and different pixel colors corresponding to different element types;

[0017] The marking module is configured to mark the building elements in the building design graph with element types based on the type graph to obtain a marked design graph;

[0018] The generation module is configured to generate a three-dimensional modeling corresponding to the target building through the marked design graph.

[0019] In yet another aspect, the embodiments of the present application provide a computer device, the computer device comprising a processor and a memory:

[0020] The memory is configured to store a computer program;

[0021] The processor is configured to execute the method according to the above aspects according to the computer program.

[0022] In yet another aspect, the embodiments of the present application provide a computer readable storage medium for storing a computer program, the computer program being used to execute the method according to the above aspects.

[0023] In yet another aspect, the embodiments of the present application provide a computer program product comprising a computer program, which, when running on a computer device, causes the computer device to execute the method according to the above aspects.

[0024] It can be seen from the above technical solutions that the building design scheme of the target building can be embodied by the building design drawing, and the building design drawing identifies the building structure of the target building in a two-dimensional plane through the building element. Since different building elements have different graphic features in the building design drawing, the building element can be accurately identified in the graphic dimension according to the plan image mainly embodying the graphic features of the building element. In order to quickly and accurately label the element type of the building element, the plan image of the building design drawing needs to be determined, and then the plan image is identified in the graphic dimension by the category recognition model to obtain the type image corresponding to the building design drawing. The type image includes a type pixel area, the type pixel area has a corresponding relationship with the building element in the building design drawing based on the spatial relationship between the type image and the building design drawing, and different element types are identified by different pixel colors. Since the type pixel area accurately identifies the corresponding building element by color and area size, the type image can quickly and conveniently label the element type of the building element in the building design drawing to obtain a labeled design drawing, and the three-dimensional modeling of the target building is obtained based on the labeled design drawing. The element type of the plan image is identified in the graphic dimension by the type recognition model, without considering other complex parameters in the building design drawing, and the element type can be automatically identified with high quality only by the model understanding the graphic features of the building element, which greatly improves the labeling efficiency and accuracy of the building element in the building design drawing and ensures the efficiency and precision of the modeling. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0026] Figure 1 A scene schematic diagram of a three-dimensional modeling generation method provided by an embodiment of the present application;

[0027] Figure 2 A flowchart of a three-dimensional modeling generation method provided by an embodiment of the present application;

[0028] Figure 3 A schematic diagram of a plan image generation provided by an embodiment of the present application;

[0029] Figure 4 A schematic diagram of obtaining a labeled design drawing provided by an embodiment of the present application;

[0030] Figure 5Another schematic diagram of generating a planar image provided by an embodiment of the present application;

[0031] Figure 6 A schematic diagram of a design drawing to be labeled provided by an embodiment of the present application;

[0032] Figure 7 A schematic diagram of three-dimensional modeling based on a second planar image generation method provided by an embodiment of the present application;

[0033] Figure 8 A schematic diagram of three-dimensional modeling provided by an embodiment of the present application;

[0034] Figure 9 A schematic diagram of training a type recognition model provided by an embodiment of the present application;

[0035] Figure 10 A schematic diagram of building element translation provided by an embodiment of the present application;

[0036] Figure 11 A schematic diagram of obtaining a sample label provided by an embodiment of the present application;

[0037] Figure 12 A schematic diagram of a method for determining a sample label provided by an embodiment of the present application;

[0038] Figure 13 A schematic diagram of a data format provided by an embodiment of the present application;

[0039] Figure 14 A schematic diagram of generating three-dimensional modeling provided by an embodiment of the present application;

[0040] Figure 15 A device schematic diagram of a three-dimensional modeling generation device provided by an embodiment of the present application;

[0041] Figure 16 A structural diagram of a terminal device provided by an embodiment of the present application;

[0042] Figure 17 A structural diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0043] Embodiments of the present application are described below with reference to the accompanying drawings.

[0044] The architectural design scheme for the target building can be embodied by an architectural design drawing, which includes various types of architectural elements. With the development of technology, the traditional architectural design drawing for identifying the architectural structure in a two-dimensional plane cannot meet the actual needs, and therefore it is expected to realize intuitive and three-dimensional display of the architectural elements by converting the two-dimensional architectural design drawing into three-dimensional modeling to adapt to new technical needs.

[0045] In the process of converting the two-dimensional architectural design drawing into three-dimensional modeling, the architectural elements in the architectural design drawing need to be obtained and the element types corresponding to the architectural elements are determined, and the determined element types are used as a basis to accurately model the architectural design drawing to obtain the three-dimensional modeling of the target building.

[0046] In the related art, when determining the element types of the architectural elements in the architectural design drawing, it is inevitable to realize the determination by a manual manner, i.e., manually labeling the element types of the architectural elements in the architectural design drawing. This manual manner has low efficiency and low accuracy, and adversely affects the result of the three-dimensional modeling of the target building obtained by subsequent modeling.

[0047] Therefore, the embodiments of the present application provide a three-dimensional modeling generation method and related device, which can perform element type recognition of a planar image at a graphic level by using a type recognition model. The element types can be automatically recognized with high quality only by understanding the graphic features of the architectural elements, which greatly improves the labeling efficiency and accuracy of the architectural elements in the architectural design drawing and ensures the modeling efficiency and accuracy.

[0048] The three-dimensional modeling generation method provided by the embodiments of the present application can be implemented by a computer device, which can be a terminal device or a server. The server can be a physical server, a server cluster composed of multiple physical servers, or a distributed system, or a cloud server providing cloud computing services. The terminal device includes but is not limited to a mobile phone, a computer, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, etc. The terminal device and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the present application.

[0049] First, some terms possibly involved in the embodiments of the present application are explained.

[0050] Computer-aided design (CAD): It is a method of using computer technology to design and draw drawings, which can be used to create two-dimensional architectural design drawings and three-dimensional modeling.

[0051] Building Information Modeling (BIM): It is a digital modeling method aimed at integrating information in the design, construction, and management processes of a building. BIM models not only contain geometric information, but also various attributes and parameters of the target building, such as materials, dimensions, costs, etc. Among them, BIM elements refer to architectural elements such as walls, doors, and windows, etc.

[0052] Revit: Revit is a software developed by Autodesk that is widely used in the field of BIM. It provides a platform for architects, engineers, and construction professionals to create, manage, and visualize building information models. Revit supports the creation of 3D models, 2D drawings, and schedules, and allows users to collaborate and share information with others.

[0053] GeoJSON: It is a lightweight data interchange format based on JSON (JavaScript Object Notation) that is used to represent geospatial data such as points, lines, and polygons. It is widely used in the field of maps and geographic information systems (GIS). In the embodiments of the present application, GeoJSON is used to generate geospatial graphs to represent the geospatial data of architectural elements in architectural design drawings.

[0054] Artificial Intelligence (AI): It is the use of digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence. In other words, artificial intelligence is a comprehensive technology in computer science that aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design and implementation of various intelligent machines that can perceive, reason, and make decisions.

[0055] Artificial intelligence technology is a comprehensive discipline that involves a wide range of fields, both hardware and software. Artificial intelligence basic technologies generally include sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-training model technology, operation / interaction system, mechatronics, etc. Among them, the pre-training model is also called the large model or the basic model, which can be widely applied to downstream tasks in various directions of artificial intelligence after fine-tuning. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0056] Computer Vision (CV) Computer vision is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes to identify, follow and measure targets, etc. Machine vision, and further processing of images to make them more suitable for human observation or transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, trying to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc. It also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0057] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning. Pre-trained models are the latest development in deep learning, integrating the above technologies.

[0058] With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned vehicles, autonomous vehicles, drones, digital twins, virtual humans, robots, artificial intelligence generated content (AIGC), conversational interaction, intelligent medical care, intelligent customer service, game AI, etc. With the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0059] The scheme provided in the embodiments of the present application relates to machine learning and computer vision technology of artificial intelligence, for example, before a type recognition model is used to obtain a type graph corresponding to an architectural design graph from a planar image, the type recognition model needs to be obtained by training an initial type recognition model, and the initial type recognition model is enabled to determine a type graph from a planar image through learning in the training process. In the embodiments of the present application, the computer vision recognition technology is related to adjusting the size of an architectural design graph and a geographic space graph, and identifying and determining the overlapping degree of an architectural element in a design graph to be labeled and a type pixel region in a type graph. The specific implementation is described as follows:

[0060] Figure 1 A scene schematic diagram of a three-dimensional modeling generation method provided in the embodiments of the present application, wherein the computer device is a server.

[0061] When it is necessary to generate a corresponding three-dimensional modeling by performing a mold turning process on a two-dimensional architectural design graph (in which a target building in a two-dimensional plane is identified by an architectural element), the server first needs to complete the element type labeling of the architectural element in the architectural design graph to obtain a labeled design graph. Then, the mold turning process is performed based on the architectural element in the labeled design graph to obtain the three-dimensional modeling corresponding to the target building. In order to improve the labeling efficiency of the element type of the architectural element in the architectural design graph, the type graph is used as a reference for the architectural design graph in the embodiments of the present application, so as to realize the fast and accurate labeling of the element type of the architectural design graph and obtain the labeled design graph (only part of the architectural elements are exemplarily labeled in the graph).

[0062] The type graph includes a type pixel region, the type pixel region has a corresponding relationship with the architectural element in the architectural design graph based on the spatial relationship between the type graph and the architectural design graph, and different element types are identified by different pixel colors. The type pixel region can identify the corresponding architectural element through color and region size, and the architectural element can include doors, windows, walls and the like. As shown in the figure, black lines are used to identify walls, white regions are used to identify windows, and gray regions are used to identify doors.

[0063] The following is an introduction to the process of obtaining the type diagram: First, it is necessary to determine the corresponding plane image based on the architectural design drawing. Different architectural elements will present different graphic characteristics in the architectural design drawing, and the graphic characteristics of the architectural elements in the architectural design drawing can be reflected in the plane image. Then, based on the obtained plane image, the type recognition model identifies the graphic dimension of the plane image to obtain the type diagram corresponding to the architectural design drawing. The type recognition model learns and understands the graphic characteristics of the architectural elements to realize the recognition of the element type of the architectural elements in the graphic dimension of the plane image. Without considering and understanding other parameters in the architectural design drawing, the element type in the architectural design drawing can be determined to obtain the corresponding type diagram. It is more convenient and efficient to label the element type of the architectural design drawing according to the type diagram, which can improve the efficiency of mold making and generating three-dimensional modeling.

[0064] Figure 2 This is a flowchart of a three-dimensional modeling generation method provided in an embodiment of the present application. The method can be executed by a computer device. In this embodiment, the computer device is described as a server as an example.

[0065] S201: Obtain architectural design drawings of a target building.

[0066] An architectural design drawing includes architectural elements and their spatial location information, which are used to identify the target building's two-dimensional architectural structure using the architectural elements. The target building is the building to be three-dimensionally modeled. The building itself can have multiple floors, such as a building or a single room. In the embodiments of the present application, it is intended to generate a corresponding three-dimensional model based on the target building's two-dimensional architectural structure, thereby providing a three-dimensional and intuitive representation of the target building's architectural structure.

[0067] Architectural elements can be understood as the two-dimensional representation of the building entities in the target building in the architectural design drawings. For example, architectural elements can include but are not limited to the following element types: walls, doors, and windows. It should be noted that different types of architectural elements in architectural design drawings have different graphic characteristics. For example, walls are generally described by straight line graphics, and doors are generally described by a combination of straight lines, dashed lines, and arcs (see Figure 1Architectural design drawings combine and place architectural elements in different spatial locations to create a two-dimensional representation of the target building's structure. Architectural design drawings include a wide range of information about these elements, including their graphics, dimensions, layers, and materials. A two-dimensional plane is defined by length and width; graphics in such a space only have these two dimensions, without depth. Three-dimensional modeling can be understood as creating a representation of the target building based on a two-dimensional architectural design drawing and incorporating data about depth.

[0068] S202: Determine a plan image of the architectural design drawing.

[0069] As mentioned above, different architectural elements have different graphic characteristics in architectural design drawings. Therefore, we can consider the differences in the graphic characteristics of architectural elements and implement the annotation of the element types of architectural elements in the architectural design drawings so as to perform efficient mold processing and generate three-dimensional modeling later.

[0070] In order to fully utilize the differences in the graphic features of architectural elements, it is necessary to obtain the plane image corresponding to the architectural design drawing, and reflect the graphic features of the architectural elements in the architectural design drawing through the plane image. As mentioned above, the architectural design drawing includes a lot of information about architectural elements, and different information corresponds to different types. The graphic features of the architectural elements in the plane image need to be accurately expressed. As for whether the plane image includes other types of information, it is not limited here. In other words, the plane image can include information related to the architectural design. Figure 1 The types of information of architectural elements that are consistent may also include fewer types of information of architectural elements than in the architectural design drawings. If one type of information is regarded as an expression dimension of one architectural element, then the number of expression dimensions of architectural elements may be different between the architectural design drawings and the plan images.

[0071] The spatial relationship between the plane image and the architectural design drawing can include a relative spatial relationship and an absolute spatial relationship. The relative position relationship (i.e., the absolute spatial relationship) between the various architectural elements in the plane image and the architectural design drawing is consistent. For example, assuming that the architectural element "door" is located to the left of the architectural element "window" in the architectural design drawing, the architectural element "door" in the plane image generated based on the architectural design drawing is also located to the left of the architectural element "window". At the same time, from the perspective of the spatial position relationship (i.e., the relative spatial relationship) between the architectural elements in the plane image and the architectural design drawing, assuming that the architectural element "door" is located in the middle position in the architectural design drawing, the architectural element "door" in the corresponding plane image will be located in the middle position in the plane image, that is, the relative position relationship between the architectural elements in the plane image and the architectural design drawing is also consistent.

[0072] S203: obtaining a type graph corresponding to the architectural design drawing through a type recognition model according to the planar image.

[0073] When the planar image is obtained, the type recognition model is used to recognize the graphic dimension of the planar image, and a type graph corresponding to the architectural design drawing is obtained. The type graph refers to an image in which the architectural elements are identified by color and area size through a type pixel area. The type pixel area identifies the element type of the architectural element through pixel color, and different pixel colors correspond to different element types.

[0074] It is mentioned above that the planar image is obtained based on the architectural design drawing, and the spatial position relationship between the architectural elements is consistent. In addition, the type recognition model does not change the spatial position relationship between the architectural elements in the process of obtaining the type graph corresponding to the architectural design drawing based on the planar image. Therefore, the type pixel area identified in the type graph has a corresponding relationship with the architectural elements in the architectural design drawing based on the spatial relationship between the type graph and the architectural design drawing. It should be noted that the type graph and the planar image have consistent image sizes. It is mentioned above that the image sizes of the planar image and the architectural design image can be different, and the image sizes of the corresponding type graph and the architectural design image can also be different. However, since the architectural design drawing, the planar image and the type graph have a corresponding spatial relationship, the spatial relationship can be understood as the corresponding relationship of the spatial position information between the architectural elements in different images mentioned above. Even if the image sizes are different, the corresponding relationship between the type pixel area in the type graph and the architectural elements in the architectural design drawing can be established.

[0075] Referring to Figure 1 illustrated, Figure 1 The planar image and the type graph are identified in the examples shown in the drawings. The image features of the architectural elements in the planar image are identified by the corresponding type pixel areas in the type graph. For example, the graphic features of the architectural element "window" shown in the planar image are identified by the white area in the type graph, the graphic features of the architectural element "door" shown in the planar image are identified by the gray area in the type graph, and the graphic features of the architectural element "wall" shown in the planar image are identified by the black line area in the type graph.

[0076] S204: annotating the element types of the architectural elements in the architectural design drawing based on the type graph to obtain an annotated design drawing.

[0077] The type pixel region in the type diagram has a corresponding relationship with the building element in the building design diagram, and different pixel colors are used to distinguish element types. Since the type pixel region accurately identifies the corresponding building element through color and area size, using the type diagram can quickly and conveniently label the element types of the building elements in the building design diagram to obtain a labeled design diagram. Referring back to Figure 1 In the labeled design diagram, the element types of different building elements are exemplarily labeled, and the specific labeling positions can be determined by those skilled in the art according to actual application conditions, which are not limited herein.

[0078] S205: performing a mockup process on the labeled design diagram to generate a three-dimensional modeling corresponding to the target building.

[0079] The aforementioned mockup process is a process of three-dimensional modeling of the two-dimensional building design diagram, and the obtained three-dimensional modeling can realize intuitive and three-dimensional display of the building elements. By labeling the element types of the building elements in the building design diagram to obtain a labeled design diagram, the manual labeling of the element types of the building design diagram is avoided in the mockup process, which can save labor costs while improving labeling efficiency and accuracy, thereby improving the efficiency of the mockup process and the effectiveness of the obtained three-dimensional modeling.

[0080] As can be seen from the above technical solution, the building design scheme of the target building can be embodied by the building design diagram, and the building design diagram identifies the building structure of the target building in a two-dimensional plane through building elements. Since different building elements have different graphic features in the building design diagram, the building elements can be accurately identified from the graphic dimension according to the plan image mainly embodying the graphic features of the building elements. In order to quickly and accurately label the element types of the building elements, the plan image of the building design diagram is determined, so the plan image is identified from the graphic dimension by the category recognition model to obtain a type diagram corresponding to the building design diagram. The type diagram includes a type pixel region, the type pixel region has a corresponding relationship with the building elements in the building design diagram based on the spatial relationship between the type diagram and the building design diagram, and different pixel colors are used to identify different element types. Since the type pixel region accurately identifies the corresponding building element through color and area size, using the plan image can quickly and conveniently label the element types of the building elements in the building design diagram to obtain a labeled design diagram, and a three-dimensional modeling of the target building is obtained based on the labeled design diagram. The element type recognition of the plan image from the graphic dimension by the category recognition model does not need to consider other complex parameters in the building design diagram, but only needs the model to understand the graphic features of the building elements to realize high-quality automatic recognition of the element types, which greatly improves the labeling efficiency and accuracy of the building elements in the building design diagram and ensures the mockup efficiency and precision.

[0081] The "determining the plan image of the architectural design drawing" mentioned in the foregoing S202 can be determined in different ways, including: (1) determining a geospatial graph from the architectural design drawing, and then obtaining the plan image from the geospatial graph; and (2) directly determining the plan image from the architectural design drawing.

[0082] The architectural design drawing can have many versions, and the difference between different versions can be large. In this case, in order to facilitate the type identification model to successfully process the plan image to obtain a type graph, the geospatial graph can be determined from the architectural design drawing, and then the plan image can be obtained from the geospatial graph (i.e., the first way mentioned above). The purpose is to obtain the plan image by using the geospatial graph that is more suitable for the type identification model, to improve the adaptability between the plan image and the type identification model to a certain extent, and the geospatial data of the building elements carried in the geospatial graph can determine the spatial position and size of the building elements, which is beneficial to the subsequent process of labeling the element types of the building elements in the architectural design drawing.

[0083] The first way of determining the plan image will be described in detail below. The method includes: first determining a geospatial graph from the architectural design drawing, and then determining a first adjustment parameter according to the image size of the geospatial graph and the input size requirement of the type identification model. Based on the first adjustment parameter, the geospatial graph is converted into a plan image that meets the input size requirement.

[0084] Compared with the architectural design drawing, the geospatial graph carries less information. In the process of determining the geospatial graph from the architectural design drawing, key information such as geometric shapes, sizes, and attributes is extracted from the architectural design drawing in advance. For example, the key information can include geometric data of building elements such as walls, doors, and windows, and attribute information such as layers and tiles. The geospatial graph is determined based on the extracted key information. In the embodiments of the present application, the architectural design drawing can be understood as a CAD drawing, and the geospatial graph can be understood as a GeoJSON graph. Compared with the architectural design drawing, the geospatial graph does not have the problem of many versions and large differences between versions, and can better adapt to the type identification model and has strong adaptability. In the process of determining the geospatial graph (GeoJSON graph) from the architectural design drawing (CAD drawing), a geospatial data abstraction library (GDAL) can be used to convert the architectural design drawing (CAD drawing) into the geospatial graph (GeoJSON graph). The conversion process can be: converting the CAD drawing into GeoJSON data, and obtaining the geospatial graph from the GeoJSON data.

[0085] The process of converting the geospatial graph through the GeoJSON data is relatively well controlled. According to the GeoJSON data, the distance of the building element in the geospatial graph from the image boundary of the geospatial graph can be determined, so as to facilitate subsequent superposition of the to-be-superimposed graph and the geospatial graph to obtain the to-be-labeled design graph. When determining the overlapping degree of the building element and the type pixel region in the to-be-labeled design graph, the GeoJSON data can be used for determination. For example, it is assumed that the distance between the building element "door" and the upper boundary of the image is 10 pixels in the geospatial graph, and the to-be-labeled design graph is obtained based on the geospatial graph. When the distance between the type pixel region corresponding to the building element "door" in the to-be-labeled design graph and the upper boundary of the image of the to-be-labeled design graph is also 10 pixels, it can be considered that the building element and the type pixel region are overlapped. Moreover, when the GeoJSON data generates the geospatial graph, the generated geospatial graph can be controlled, such as filtering of some interference data, generation of the geospatial graph in layers, customization of information such as the background of the geospatial graph, and the like.

[0086] The geospatial graph is used to reflect the geospatial data of the building element in the architectural design graph. The geospatial data of the building element refers to the information such as the position, shape and size of the building element. In the geospatial graph, the involved range of the graphical features of the building element in space can be described in the form of coordinates.

[0087] In fact, the image size of the plane image can be different from the image size of the architectural design graph. The reason is that the image size of the architectural design graph can be quite different, while the plane image as the input data of the type recognition model needs to meet the input size requirement when inputting the model. The input size requirement for the same type recognition model is generally fixed, so the image size needs to be adjusted during the process of obtaining the plane image from the architectural design graph. In the embodiments of the present application, the plane image can be a PNG image.

[0088] The planar image corresponding to the architectural design drawing is input into a type recognition model as input data. Based on the planar image, the type recognition model generates a type map corresponding to the architectural design drawing. To improve the model's computational efficiency or meet the model's structural requirements, the size of the input planar image must be adjusted to meet the type recognition model's input size requirements. The input size requirement refers to a size restriction imposed on the input image data (i.e., the planar image) to adapt to the model structure and enable the type recognition model to correctly process and analyze the image data. If the size of the input planar image does not meet the input size requirement, the type recognition model may be unable to correctly process and analyze the data in the planar image. When determining a geospatial map from an architectural design drawing, image size adjustment is generally not performed; that is, the image size of the generated geospatial map is generally consistent with that of the architectural design drawing. However, the type recognition model itself has input size requirements for the size of the input image data, so the geospatial map must be resized to meet these input size requirements. During this resizing process, a first adjustment parameter must be determined, which indicates the scaling factor for the resized geospatial map. When the first adjustment parameter is determined, the geospatial image is resized according to the first adjustment parameter to be converted into a planar image that meets the input size requirement of the type recognition model.

[0089] The specific first adjustment parameter needs to be determined based on the difference between the image size of the geographic spatial map and the input size requirement of the type recognition model. Different first adjustment parameters are determined for different input size requirements and different image sizes.

[0090] Figure 3 This is a schematic diagram of a planar image generation provided in an embodiment of the present application, see Figure 3 As shown, a geospatial map is determined based on an architectural design drawing, and the geospatial data of architectural elements in the architectural design drawing can be reflected in the geospatial map. The geospatial map has a corresponding image size, and the type recognition model has a corresponding input size requirement. A first adjustment parameter can be determined based on the image size and the input size requirement, and the geospatial map is converted into a planar image based on the first adjustment parameter.

[0091] In a possible implementation, the awt library (Abstract Window Toolkit, a standard library for creating graphical user interface applications) of Java can be used to convert the geospatial graph into a planar image that meets the input size requirement. The process includes determining the bounding box size of the geospatial graph and the image size corresponding to the input size requirement. According to the size of the bounding box of the geospatial graph and the image size corresponding to the input size requirement, the scaling ratio (i.e., the first adjustment parameter) of the geospatial graph is calculated. The geospatial graph is scaled based on the first adjustment parameter to adapt to the input size requirement of the type recognition model. The scaled geospatial graph is subjected to film offsetting and other operations to obtain a planar image.

[0092] Through the above-mentioned method for determining a planar image, the geospatial graph is first determined through the architectural design graph, and then the geospatial graph is subjected to size adjustment based on the first adjustment parameter to obtain a planar image. The introduction of the geospatial graph in the process can ensure that the determined planar image has a high degree of adaptation to the type recognition model to some extent. In addition, since the geospatial data of the architectural elements are embodied in the geospatial graph, the image range and size of the architectural elements can be determined, which is beneficial to the subsequent process of labeling the element types of the architectural elements in the architectural design graph.

[0093] The aforementioned S204 mentioned “labeling the element types of the architectural elements in the architectural design graph based on the type graph to obtain a labeled design graph”. In order to improve the labeling efficiency, the geospatial graph and the type graph having the same image size can be superimposed based on the overlapping degree of the architectural elements and the type pixel region to realize fast labeling. Therefore, in a possible implementation, the method for obtaining a labeled design graph is:

[0094] A1: performing inverse conversion of the type graph based on the first adjustment parameter to obtain a to-be-superimposed graph.

[0095] When the first method is used to generate a planar image, the type graph corresponding to the planar image has the same size as the planar image. Through the inverse process (i.e., inverse conversion) of the geospatial graph conversion process of the type graph based on the aforementioned first adjustment parameter, a to-be-superimposed graph having the same image size as the geospatial graph can be obtained.

[0096] In the generation process of the type diagram, the type recognition model determines the corresponding type pixel region based on the building elements in the planar image, so there is a position correspondence relationship between the building elements in the planar image and the type pixel region in the type diagram. The planar image is obtained by size conversion of the geospatial graph based on the first adjustment parameter, so there is a clear scaling relationship between the planar image and the geospatial graph. The geospatial graph can be obtained again by inversely converting the planar image according to the first adjustment parameter.

[0097] In the process of obtaining the type diagram from the planar image by the type recognition model, the image size is generally not changed, that is, the image size of the type diagram is consistent with that of the planar image, and there is a scaling relationship corresponding to the first adjustment parameter between the planar image and the geospatial graph. Then it can be determined that there is also a scaling relationship corresponding to the first adjustment parameter between the image size of the geospatial graph and the type diagram. The type diagram can be inversely converted according to the first adjustment parameter to obtain the to-be-stacked graph with consistent image size with the geospatial graph, and there is a position correspondence relationship between the type pixel region in the to-be-stacked graph and the building elements in the geospatial graph.

[0098] A2: Stacking the geospatial graph and the to-be-stacked graph to obtain a to-be-labeled design graph.

[0099] In the to-be-stacked graph, the type pixel region corresponding to the element type of different building elements scaled according to the first adjustment parameter is included. The process of stacking the geospatial graph and the to-be-stacked graph with consistent image size can be understood as determining the building elements in the geospatial graph and the type pixel region in the to-be-stacked graph, which are placed in the same image for display. The result of the display is the to-be-labeled design graph.

[0100] A3: According to the overlapping degree of the building elements and the type pixel region in the to-be-labeled design graph, labeling the element type of the type pixel region in the building elements in the to-be-labeled design graph, and taking the labeled to-be-labeled design graph as the labeled design graph.

[0101] The design drawing to be annotated includes both architectural elements in the geospatial map and type pixel areas in the map to be superimposed. Since the type pixel areas identify the element type of the corresponding architectural elements by pixel color, by determining the degree of overlap between the architectural elements and the type pixel areas, the element type of the architectural elements in the geospatial map that overlap with the type pixel areas can be quickly determined. The determined element type is then annotated accordingly in the design drawing to be annotated, and the annotated design drawing to be annotated is used as the annotated design drawing. The degree of overlap can be understood as the degree of overlap between the image area of ​​the architectural elements in the design drawing to be annotated and the type pixel areas in the map to be superimposed. The process of annotating the element type of the architectural elements based on the degree of overlap between the two areas is relatively simple and can be completed quickly.

[0102] By marking the architectural elements whose element types have been determined, a marked design drawing can be obtained. At the same time, the data corresponding to the architectural elements whose element types have been determined in the design drawing to be marked can be saved. For example, the program content involved in the saving process is as follows, which can be:

[0103] {

[0104] "id":0, / / uuid

[0105] "sweptArea":[

[0106] [-10.0,-10.0],[10.0,-10.0],[10.0,10.0],[-10.0,10.0],[-10.0,-10.0]

[0107] ], / / Stretch cross section, [-10.0,-10.0] represents a point x, y,

[0108] "position":{

[0109] "x":0,

[0110] "y":0,

[0111] "z":0

[0112] }, / / Stretch body position

[0113] "direction":{

[0114] "x":0,

[0115] "y":0,

[0116] "z":0

[0117] }, / / Stretching direction

[0118] "depth":0 / / stretch length

[0119] }

[0120] Figure 4 A schematic diagram of obtaining a marked design drawing provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the type map generated by the type recognition model based on the planar image is consistent in size with the planar image, both meeting the input size requirements of the type recognition model. By using the first adjustment parameter to inversely transform the type map to the geospatial map, a to-be-overlaid map with the same image size as the geospatial map is obtained. The geospatial map and the to-be-overlaid map are then overlaid to obtain the to-be-annotated design map.

[0121] The method for obtaining an annotated design drawing provided above scales the type map based on a first adjustment parameter to obtain an image to be overlaid. This ensures that the image sizes of the image to be overlaid and the geospatial map are consistent. Based on the consistency of the image sizes, the image to be overlaid and the geospatial map are overlaid to obtain the design drawing to be annotated. Based on the degree of overlap between the type pixel areas in the to-be-overlaid image and the architectural elements in the geospatial map, architectural elements in the geospatial map that overlap with the type pixel areas can be quickly annotated with the element types corresponding to the type pixel areas. The annotation process is simple and easy, enabling rapid annotation of the element types of architectural elements in the geospatial map based on the degree of overlap.

[0122] As mentioned above, based on the degree of overlap between the architectural elements and the type pixel areas in the design drawing to be annotated, the architectural elements in the design drawing to be annotated are annotated with the element types of the corresponding type pixel areas. In the design drawing to be annotated, there may be a situation where a certain architectural element partially overlaps with multiple type pixel areas. At this time, in order to improve the accuracy of the element type annotation of the architectural element, it is possible to consider setting consistency requirements for the degree of overlap. That is, the degree of overlap is preferentially screened for multiple type pixel areas to improve the accuracy of element type annotation. In a possible implementation method, for the target architectural element in the design drawing to be annotated, the element type of the target architectural element is annotated in the following manner: first, determine the degree of overlap between the target architectural element and the target type pixel area in the design drawing to be annotated. Then, in response to the overlap degree meeting the consistency requirement, the element type of the target architectural element is annotated as the element type corresponding to the target type pixel area.

[0123] The target building element can be understood as a building element in the design drawing to be labeled. When labeling the element type of the target building element, first, the type pixel region having an overlapping part with the target building element is determined. The target building element can have overlapping parts with multiple type pixel regions. In this case, all type pixel regions having overlapping parts with the target building element can be regarded as candidate type pixel regions. Then, the candidate type pixel regions are regarded as target type pixel regions, and the overlapping degrees of the target building element and the target type pixel regions are determined respectively.

[0124] The consistency requirement is used to measure the overlapping degree between the target building element and the target type pixel region, so as to determine whether the target building element and the target type pixel region have consistency. For example, when the area of the overlapping part between the target building element and the target type pixel region accounts for 90% of the area of the target building element, it can be determined that the overlapping degree between the target building element and the target type pixel region meets the consistency requirement, and the element type of the target building element is labeled as the element type corresponding to the target type pixel region.

[0125] Suppose that the target building element A in the design drawing to be labeled has overlapping parts with three type pixel regions, type pixel region A, type pixel region B and type pixel region C. The consistency requirement is set as follows: when the overlapping degree between the type pixel region and the target building element reaches 80%, it is determined that the type pixel region and the target building element have consistency. The above three type pixel regions are regarded as candidate type pixel regions, and the overlapping degrees between the target building element A and the target type pixel regions are determined respectively, obtaining the overlapping degrees between type pixel region A and target building element A as 5%, between type pixel region B and target building element A as 81%, and between type pixel region C and target building element A as 14%. It can be determined that the overlapping degree between type pixel region B and target building element A meets the consistency requirement, and the element type of target building element A is labeled as the element type corresponding to type pixel region B.

[0126] Through the above-mentioned method for labeling the element type of a target building element, by setting the consistency requirement of the overlapping degree, when multiple type pixel regions have overlapping parts with the target building element in the design drawing to be labeled, the overlapping degrees between the type pixel regions and the target building element are determined and selected based on the consistency condition as a measurement standard, which can ensure the accuracy of labeling the element type of the target building element to a certain extent.

[0127] The aforementioned determining the planar image of the architectural design drawing has two manners, and the second manner is: determining the planar image directly through the architectural design drawing. For the generation manner of the planar image, the calculation amount of the planar image determining process can be reduced to a certain extent, compared with the first manner of determining the geographic space image through the architectural design drawing and then obtaining the planar image from the geographic space image, the process is relatively simple, and can be applied to application scenarios in which it is expected to reduce the calculation amount of the planar image generated in the early stage. In a possible implementation manner, the method of "determining the planar image of the architectural design drawing" mentioned in S202 can be: first, determining the second adjustment parameter according to the image size of the architectural design drawing and the input size requirement of the type recognition model. Then, based on the second adjustment parameter, the architectural design drawing is converted into a planar image that meets the input size requirement.

[0128] The second adjustment parameter refers to a scaling ratio for scaling the architectural design drawing. In the aforementioned method of determining the planar image, the planar image is directly determined based on the architectural design drawing and the input size requirement of the type recognition model. In the process, the image size of the architectural design drawing is scaled by the second adjustment parameter to meet the input size requirement of the type recognition model. Because different architectural design drawings can have different image sizes, and the input size requirements of different type recognition models also differ, the second adjustment parameter determined according to the different image sizes of the architectural design drawings and the different input size requirements of the type recognition models is also different.

[0129] In the process of determining the planar image, first, the scaling ratio (i.e., the second adjustment parameter) of the architectural design drawing is determined according to the image size of the architectural design drawing and the input size requirement of the type recognition model. Then, a point in the architectural design drawing is selected as a reference point. Then, the architectural design drawing is scaled according to the second adjustment parameter based on the reference point. Finally, the scaled architectural design drawing is converted into a planar image.

[0130] Figure 5 Another schematic diagram of generating a planar image provided by an embodiment of the present application is shown in FIG. 6. Figure 5 As shown in FIG. 6, in the process of generating the planar image, the second adjustment parameter is directly determined according to the image size of the architectural design drawing and the input size requirement of the type recognition model. Then, the architectural design drawing is converted based on the second adjustment parameter to obtain a planar image. In the planar image, the graphic features of the architectural elements in the architectural design drawing can be reflected, and the content of the non-architectural elements can be removed in the planar image.

[0131] When the architectural design drawing is a CAD drawing and the plan image is a PNG image, after the scaling of the architectural design drawing is completed, the conversion of the CAD drawing to the PNG image can be realized through screenshot, changing the export picture format, conversion tools and the like. In the conversion process, in addition to the change of the image format, the architectural design drawing can also be subjected to black and white processing of the image color and obtaining of the picture bounding box and the like.

[0132] Through the method for determining a plan image of an architectural design drawing provided above, the plan image can be directly obtained from the architectural design drawing. In the process, the second adjustment parameter is determined according to the input size requirement of the model recognition model according to the image size and type of the architectural design drawing, and the architectural design drawing is converted into a plan image that meets the input size requirement according to the second adjustment parameter. The process of obtaining the plan image is relatively simple, which can save the computing resources required for generating the plan image and improve the generation efficiency of the plan image.

[0133] It is mentioned in the foregoing S204 that “based on the type map, the building elements in the architectural design drawing are labeled with element types to obtain a labeled design drawing”. When the plan image is generated in the second way, in order to realize the subsequent obtaining of the design drawing to be labeled, in addition to determining the plan image from the architectural design drawing, the geographic space map is also determined from the architectural design drawing, and the geographic space data of the building elements in the architectural design drawing is embodied in the geographic space map. In the second way, the design drawing to be labeled is also obtained by superimposing the geographic space map and the design drawing to be superimposed. However, since the image size relationship between the type map and the geographic space map cannot be directly determined in the second way, the image size conversion mode of the type map needs to be determined before the design drawing to be superimposed is obtained based on the type map. In a possible implementation manner, the method for obtaining the labeled design drawing is:

[0134] B1: determining first position information of a bounding box of a type pixel region in the type map and second position information of a bounding box of a building element in the geographic space map.

[0135] The bounding box can be understood as a frame, and the position and size of the image can be described through the bounding box. The positions and sizes of the bounding boxes of different type pixel regions in the type map are different, and the type pixel region has a corresponding relationship with the building element in the architectural design drawing based on the spatial relationship between the type map and the architectural design drawing. Since the geographic space map is determined from the architectural design drawing, and the geographic space map is used to embody the geographic space data of the building elements in the architectural design drawing, the type pixel region also has a corresponding relationship with the building element in the geographic space map.

[0136] The bounding box of the type pixel region in the type graph has its corresponding position information (i.e., first position information), and the building element corresponding to the type pixel region in the geographic space graph also has its corresponding position information (i.e., second position information). Both the first position information and the second position information can be represented in the form of coordinates.

[0137] B2: converting the type graph according to the first position information and the second position information to obtain a to-be-stacked graph, the to-be-stacked graph being consistent with the image size of the geographic space graph.

[0138] When the first position information of the bounding box of the type pixel region in the type graph and the second position information of the bounding box of the building element corresponding to the type pixel region in the geographic space graph are obtained, any one of the first position information and the second position information can be taken as a reference of the other position information. For example, the second position information can be taken as a reference of the first position information. In order to ensure that the image size of the to-be-stacked graph is consistent with the image size of the geographic space graph, the type graph can be converted to achieve this. During the conversion, it is necessary to ensure that the first position information of the bounding box of the type pixel region in the to-be-stacked graph obtained by conversion is consistent with the second position information. When the first position information of the bounding box of the type pixel region in the to-be-stacked graph is consistent with the second position information of the bounding box of the building element corresponding to the type pixel region in the geographic space graph, it can be determined that the size of the to-be-stacked graph is consistent with the size of the geographic space graph.

[0139] For example, it is assumed that the first position information of the bounding box of the type pixel region A in the type graph is (1, 1), and the second position information of the bounding box of the building element B corresponding to the type pixel region in the geographic space graph is (10, 10). When a coordinate position is used to represent the position information of the bounding box, the coordinate position can be the coordinate of the geometric center point of the bounding box. According to the first position information and the second position information, the image size of the type graph can be converted. Since the type pixel region A in the type graph has a spatial correspondence relationship with the building element B in the geographic space graph, and the coordinate value in the first position information is smaller than the coordinate value in the second position information, it can be determined that the image size of the type graph is smaller than the image size of the geographic space graph. In order to ensure the smooth progress of the subsequent stacking process, it is necessary to ensure that the to-be-stacked graph obtained by converting the type graph has the same image size as the geographic space graph. Therefore, the type graph needs to be enlarged. During the enlargement, the coordinate of the first position information can be taken as a reference. When the first position information of the bounding box of the type pixel region A in the to-be-stacked graph changes from (1, 1) to (10, 10), it can be determined that the image size of the to-be-stacked graph is consistent with the image size of the geographic space graph.

[0140] B3: superimpose the geospatial graph and the to-be-superimposed graph to obtain a to-be-labeled design graph.

[0141] After the foregoing conversion processing of the type graph, the to-be-superimposed graph having a consistent image size with the geospatial graph is obtained. Superimposing the geospatial graph and the to-be-superimposed graph can obtain the to-be-labeled design graph, which includes the building elements and the type pixel regions, and there is an overlapping part between the building elements and the type pixel regions.

[0142] B4: according to the overlapping degree of the building elements and the type pixel regions in the to-be-labeled design graph, labeling the element type of the corresponding type pixel region to the building element in the to-be-labeled design graph, and taking the labeled to-be-labeled design graph as the labeled design graph.

[0143] By determining the overlapping degree of the building elements and the type pixel regions in the to-be-labeled design graph, the more the overlapping part between the building elements and the type pixel regions, the higher the corresponding overlapping degree between the building elements and the type pixel regions; the less the overlapping part between the building elements and the type pixel regions, the lower the corresponding overlapping degree between the building elements and the type pixel regions.

[0144] According to the overlapping degree of the building elements and the type pixel regions in the to-be-labeled design graph, that is, according to the more or less of the overlapping part between the building elements and the type pixel regions in the to-be-labeled design graph, the element type of the corresponding type pixel region can be labeled to the building element in the to-be-labeled design graph, and the labeled design graph is obtained after labeling. The labeled design graph includes the building elements and the labeling situation of the building elements.

[0145] Figure 6 A schematic diagram of a to-be-labeled design graph provided by an embodiment of the present application is shown in FIG. 1. Figure 6As shown, since the second method for determining the planar image is adopted, the size ratio relationship between the type image corresponding to the planar image and the geospatial graph is unknown because the planar image is directly converted from the architectural design graph. Therefore, the first position information of the bounding box of the type pixel region in the type graph and the second position information of the bounding box of the architectural element in the geospatial graph are needed to determine the size ratio relationship between the type graph and the image size of the geospatial graph, and then the type graph is converted so that the converted to-be-stacked graph is consistent with the image size of the geospatial graph. The geospatial graph and the to-be-stacked graph are stacked to obtain the to-be-labeled design graph. As shown in the to-be-labeled design graph, it can be seen that the architectural elements in the geospatial graph and the type pixel regions in the to-be-stacked graph are included in the to-be-labeled design graph, and there is more or less overlapping part between different architectural elements and type pixel regions. Subsequently, according to the overlapping degree of the architectural elements and the type pixel regions in the to-be-labeled design graph, the element type of the type pixel region corresponding to the architectural element in the to-be-labeled design graph can be labeled to obtain the labeled design graph.

[0146] According to the above-mentioned method for obtaining a labeled graph, since the planar image is directly converted from the architectural design graph by the second method, the size ratio relationship between the geospatial graph and the type graph is unknown. The type graph is converted to obtain the to-be-stacked graph by using the first position information of the bounding box of the type pixel region in the type graph and the second position information of the bounding box of the architectural element in the geospatial graph, so that the image size of the to-be-stacked graph is consistent with that of the geospatial graph. This facilitates subsequent rapid labeling of the element type of the architectural element in the to-be-labeled design graph by stacking. The conversion of the type graph is indirectly determined by using the position information as a reference to obtain the to-be-stacked graph.

[0147] Figure 7 A schematic diagram of three-dimensional modeling based on the second planar image generation mode provided by the embodiment of the present application is shown in Figure 7 The method comprises the following steps:

[0148] C1: generating image data from a CAD graph.

[0149] The CAD graph is the architectural design graph of the target building, and the image data is generated from the CAD graph (i.e., the planar image of the architectural design graph). In the process of generating the image data, the CAD graph needs to be converted into a PNG image. The PNG image is the image data. In the conversion process, the CAD graph is scaled at a certain reference point to generate an m*n PNG image. The specific scaling ratio needs to be determined according to the input size requirement of the type recognition model and the image size of the CAD graph itself. At the same time, the CAD graph is converted into GeoJSON data, and the geospatial graph is obtained from the GeoJSON data.

[0150] C2: Generating training samples.

[0151] The data annotation is needed in the process of generating training samples, which refers to annotating the element types of the building elements in the sample building design drawings. The specific annotation method can be through machine learning or manually.

[0152] C3: Training an initial recognition model.

[0153] After obtaining the training sample set corresponding to the sample building design drawing after completion of annotation, the initial recognition model is trained based on the training sample set to obtain a type recognition model.

[0154] C4: Element type inference of building elements.

[0155] According to the picture data (i.e. plan image), the type recognition model is used for data reasoning of the element type of the building elements, to obtain a type graph corresponding to the plan image. Based on the type graph and the geospatial graph, the element type inference is performed, which can be: determining first position information of a bounding box of a type pixel region in the type graph, and second position information of a bounding box of a building element in the geospatial graph, converting the type graph according to the first position information and the second position information to obtain a to-be-stacked graph. Stacking the geospatial graph and the to-be-stacked graph obtains a to-be-annotated design drawing. According to the overlap degree of the building elements and the type pixel region in the to-be-annotated design drawing, the element type of the building elements is determined and annotated to obtain an annotated design drawing, and the corresponding generated annotation data is a json file.

[0156] C5: Three-dimensional modeling.

[0157] The annotated BIM elements (i.e. building elements) in the annotated design drawing are imported into a three-dimensional modeling software or a rendering engine, and a reverse modeling process is performed to generate a three-dimensional model. In this process, the wall can be drawn according to the wall cross section, stretching direction and length in the json file, and rendering parameters such as light, shadow and reflection can be configured to enhance the realism and visual experience of the model. Figure 8 A schematic diagram of three-dimensional modeling provided by an embodiment of the present application is shown in the figure, from which it can be seen that the image of the wall obtained by three-dimensional modeling.

[0158] The type image is an image that identifies building elements through type pixel regions by color and region size, and the planar image is an image that embodies the graphic features of building elements in the architectural design drawing. The type recognition model needs to have the ability to identify and convert the graphic features in the planar image in the graphic dimension to present the corresponding type pixel region, and the type recognition model can improve the efficiency of identifying and determining the type pixel region in the graphic dimension of the planar image. The training process of the type recognition model is introduced as follows:

[0159] D1: Obtain a training sample set.

[0160] The training sample in the training sample set includes a sample architectural design drawing, and a sample label of the training sample is used to identify the element type of the building element that has been labeled in the sample architectural design drawing. The sample architectural design drawing is used to identify the building structure of the building in a two-dimensional plane through building elements. The sample architectural design drawing includes building elements and spatial position information of the building elements, and the building structure in a two-dimensional plane can be presented by placing building elements in different spatial positions. The sample label can identify the element type of the building element that has been labeled in the sample architectural design drawing. For example, the element type can include walls, doors, windows, etc. The type pixel region in the sample label distinguishes different element types by pixel color, and identifies the image range corresponding to the building element by the size of the region.

[0161] D2: Convert the training sample into a sample planar image.

[0162] The sample planar image is used to embody the graphic features of the building elements in the sample architectural design drawing. Different building elements correspond to different graphic features, so the element type of the building elements in the architectural design drawing can be labeled by the difference in the image features of the building elements. Since the sample planar image can be used to embody the graphic features of the building elements, the sample planar image needs to be obtained in order to identify the graphic dimension of the sample planar image by the initial recognition model.

[0163] The method of converting the training sample (i.e. the sample architectural design drawing) into the sample planar image can include: (1) determining a geographic space map from the sample architectural design drawing, and then obtaining the sample planar image from the geographic space map; (2) directly determining the sample planar image from the sample architectural design drawing. The method of converting the sample architectural design drawing into the planar image is generally consistent with the method of obtaining the planar image based on the architectural design drawing as described above, and will not be repeated here.

[0164] D3: obtaining, according to the sample plan image, a predicted type graph corresponding to the sample architectural design drawing by an initial recognition model.

[0165] When the sample plan image is obtained, the initial recognition model is used to identify the graphic dimension of the sample plan image to obtain a predicted type graph corresponding to the sample architectural design drawing. The predicted type graph includes a predicted type pixel region. The predicted type pixel region is based on the spatial relationship between the predicted type graph and the sample architectural design drawing, and has a corresponding relationship with the architectural elements in the sample architectural design drawing. The predicted type pixel region identifies the element type of the corresponding architectural element by pixel color. Different pixel colors correspond to different element types. The predicted type pixel region identifies the architectural elements in the sample architectural design drawing by the size and color of the region.

[0166] The sample plan image is obtained based on the sample architectural design drawing. The spatial position relationship between the architectural elements of the two is consistent. At the same time, the initial recognition model does not change the spatial position relationship between the architectural elements in the process of obtaining the predicted type graph corresponding to the sample architectural design drawing based on the sample plan image. Therefore, the type pixel region identified in the predicted type graph has a corresponding relationship with the architectural elements in the sample architectural design drawing based on the spatial relationship between the predicted type graph and the sample architectural design drawing.

[0167] D4: training the initial recognition model according to the difference between the predicted type graph and the corresponding sample label to obtain the type recognition model.

[0168] The initial recognition model identifies the sample plan image mainly embodying the graphic features of the architectural elements. The predicted type graph is obtained by identifying the architectural elements from the graphic dimension according to the different graphic characteristics of different architectural elements in the sample architectural design drawing. In the predicted type graph, the predicted type pixel region identifies different element types of different architectural elements based on the difference in pixel color, and identifies the image region of the architectural element based on the size of the pixel region.

[0169] The sample label includes the element types of the architectural elements in the sample architectural design drawing. The sample label includes the type pixel region, which distinguishes different element types according to pixel color and identifies the corresponding image range of the architectural element by the size of the region. In the predicted type graph, the element types of different architectural elements are determined based on the predicted type pixel region. By comparing the sample label with the predicted type graph, the difference between the two can be determined. According to the difference, the initial recognition model can be trained to obtain the type recognition model.

[0170] The specific differences determined during the comparison process may include differences between element types determined for the sample label and the predicted type map, and may also include differences between image ranges of architectural elements determined for the sample label and the predicted type map.

[0171] Figure 9 A schematic diagram of a type recognition model obtained by training provided in an embodiment of the present application, see Figure 9 As shown, the training sample set includes multiple sample architectural design drawings (the sample architectural design drawings shown in the figure Figure 1 , sample architectural design Figure 2 ... sample building design drawing n) and the sample labels corresponding to each sample building design drawing (sample label 1, sample label 2 ... sample label n), there is a corresponding relationship between the sample building design drawings and the sample labels. Figure 1 As an example, the sample building design Figure 1 The sample plane image 1 is converted and the graphic features of the architectural elements in the sample architectural design drawing are reflected in the sample plane image. The sample plane image 1 is input into the initial recognition model to obtain the prediction type corresponding to the sample plane image 1. Figure 1 According to the prediction type Figure 1 The initial recognition model is trained based on the difference between the initial recognition model and the sample label 1. After the initial recognition model completes the learning for all sample plane images, the type recognition model can be obtained.

[0172] It should be noted that during the training of the initial training model, the training sample set can be divided into training samples and validation samples. The training samples are used to train the initial recognition model, and the validation samples are used to evaluate the performance of the trained initial recognition model. The initial recognition model can adopt machine learning algorithms, such as convolutional neural networks in deep learning, transformer models (a neural network model based on the self-attention mechanism), etc. In addition, model parameters and hyperparameters can be set for the initial recognition model to optimize model performance. Based on the performance of the initial recognition model on the validation samples, the model parameters and hyperparameters can be adjusted to improve recognition accuracy, ultimately obtaining a type recognition model.

[0173] The above-mentioned method for training a type recognition model can improve the efficiency of type map generation by generating a type recognition model using the type recognition model obtained by training the initial recognition model. Furthermore, during the process of training the initial recognition model to obtain the type recognition model, the difference between the sample labels and the predicted type map can enable the initial recognition model to better learn how to use the predicted type pixel areas to label the element types of architectural elements, thereby improving the accuracy of element type labeling for architectural elements.

[0174] The aforementioned "obtaining a set of training samples" in D1, in order to enable the initial recognition model to access more training samples, thereby improving the effectiveness of the type recognition model obtained by training, it is considered to generate new training samples after processing the original training samples, so as to enrich the data amount of the training samples of the initial recognition model. Therefore, in a possible implementation manner, the method for obtaining a set of training samples can be: first obtaining initial training samples, then performing data enhancement processing on the initial training samples to obtain new training samples, and generating a set of training samples according to the initial training samples and the new training samples.

[0175] The initial training samples include sample building design drawings and sample building design drawing corresponding sample labels, and the aforementioned data enhancement processing includes at least one of rotation, flip, scaling, shifting, and noise addition. The different data enhancement processing methods will be introduced in detail as follows:

[0176] (1) Rotation: by rotating the sample building design drawing in a clockwise or counterclockwise direction by a certain angle, the initial recognition model can learn how to recognize training data at different angles, thereby improving the generalization ability of the initial recognition model.

[0177] (2) Flip: by randomly flipping the sample building design drawing by a certain angle along the horizontal or vertical direction, the initial recognition model can recognize mirrored building elements and improve the symmetry recognition ability of the initial recognition model.

[0178] (3) Zoom In / Out: by zooming in or out of the sample building design drawing by a certain ratio, the initial recognition model can recognize building elements of different sizes, further enhancing the adaptability of the initial recognition model to the size of the building elements.

[0179] (4) Shift: by moving the sample building design drawing by a certain distance along the horizontal or vertical direction, the initial recognition model can recognize building elements in different positions, thereby improving the robustness of the initial recognition model to changes in the position of the building elements.

[0180] (5) Noise: by adding random noise to the sample building design drawing, the initial recognition model can learn to ignore noise, so that the initial recognition model performs more stably when processing sample building design drawings in noisy environments.

[0181] In the data augmentation processing of the initial training sample, one or more of the above processing methods can be used, and when the data augmentation processing is completed, the new training sample is obtained. The combination of the new training sample and the initial training sample is used as a training sample set for training the initial recognition model. By generating new training samples to increase the number of training samples in the training sample set, the initial recognition model can learn more graphical features and transformations, and learning from diversified training samples can reduce the risk of overfitting of the initial recognition model.

[0182] Through the above-mentioned method for obtaining a training sample set, data augmentation processing is performed on the obtained initial training sample. The initial training sample can be processed by one or more data augmentation processing methods, and different processing methods can obtain different new training samples. There is also a difference between the new training sample and the initial training sample. Therefore, the obtained training sample set can include initial training samples and new training samples that are different from each other, which increases the number of training samples in the training sample set, so that the initial recognition model can be fully learned and trained, and to a certain extent, the type recognition model obtained by training has good recognition effect.

[0183] As mentioned above, the initial training sample is subjected to data augmentation processing to obtain a new training sample, and the data augmentation processing includes at least one of rotation, flipping, scaling, translation, and noise addition. When the translation operation is used to perform data augmentation processing on the initial training sample, there are two ways of translation operation, which are local translation and global translation. Global translation refers to the translation of the position of the sample building design drawing as a whole, and local translation refers to the translation of the position of part of the building elements in the sample building design drawing. In the embodiment of the present application, local translation is selected to obtain a new training sample in order to expand the difference between the training samples. Therefore, in a possible implementation manner, the method for obtaining a new training sample is to move the position of the building element in the initial training sample in accordance with the building design requirements to obtain a new training sample.

[0184] In the design process of the building design drawing, there is a building design specification (i.e., building design requirements). The purpose of the building design requirements is to ensure that the target building designed has reasonable and feasible building structure in the two-dimensional plane. For example, the building design requirements can be that no building element "door" can be added at the building element "window", and a building element "door" can be added at the building element "wall".

[0185] When using translation to perform data augmentation on the initial training samples, a local translation is used to translate the architectural elements in the sample architectural design drawing, so that the positions of the architectural elements in the sample architectural design drawing are changed. The translation operation must comply with architectural design requirements. For example, if the architectural design requirements state that the architectural element "door" can only be set at the architectural element "wall", then when translating the architectural element "door" in the sample architectural design drawing, the architectural element "door" at architectural element "wall" A can only be moved to architectural element "wall" B through translation.

[0186] Figure 10 A schematic diagram of a translation of a building element provided in an embodiment of the present application is shown in FIG. Figure 10 As shown in the figure, assume that the architectural design requirements indicate that the architectural element "door" can only be set at the architectural element "wall." When performing a translation operation on initial training sample 1, architectural element A ("door") is translated to obtain new training sample 1. As shown in the figure, it can be seen that architectural element A is translated from architectural element B ("wall") in initial training sample 1 to architectural element C ("wall") in the new training sample, which meets the architectural design requirements.

[0187] By using the method provided above to obtain new training samples, when the initial training samples are subjected to data enhancement processing by translation, the architectural elements therein are moved in a local translation manner in accordance with the architectural design requirements. Compared with the overall translation method, this method can increase the difference between the newly obtained training samples and the initial training samples, thereby increasing the richness of the training samples in the training sample set used for training, which is beneficial to ensuring the training effect of the initial recognition model training, so that the obtained type recognition model has a better recognition effect.

[0188] As mentioned above, the training sample set includes training samples and corresponding sample labels. The sample labels are used to identify the element types of the labeled architectural elements in the sample architectural design drawings. The following describes how to determine the sample labels. In one possible implementation, the sample labels are determined by removing unlabeled elements from the sample architectural design drawings of the target training sample and determining the sample labels of the target training sample based on the sample architectural design drawings without the unlabeled elements.

[0189] Wherein, the unlabeled element is the element in the sample architectural design drawing except the labeled element type, for example, assuming that the sample architectural design drawing includes the following architectural elements: door, wall, window, table and chair. Wherein, the door, wall and window are the labeled element type, and the table and chair are the unlabeled element. In the process of determining the sample label, the unlabeled element needs to be removed, and then the sample label is determined according to the sample architectural design drawing after removing the unlabeled element.

[0190] In the process of training the initial recognition model, the training sample and the sample label corresponding to the training sample are needed. The sample label is used as a reference to measure the quality of the predicted type drawing generated by the initial recognition model, and then the initial recognition model is trained according to the difference between the sample label and the predicted type drawing to obtain the type recognition model. The sample label has a corresponding relationship with the sample architectural design drawing. The sample architectural design drawing includes various architectural elements, and the architectural elements include labeled element types and unlabeled elements. At this time, the sample label generated based on the sample architectural design drawing will also include the labeled element type and the unlabeled element. In fact, in the training process, the unlabeled element cannot provide positive help effect for the training process, on the contrary, because the unlabeled element is irrelevant in the training process, it may also affect the training process, so that the training process of the initial recognition model is disturbed by the unlabeled element, and the effectiveness of the training is reduced.

[0191] Therefore, in the process of determining the sample label, the unlabeled element in the sample architectural design drawing needs to be removed. In this way, it can be ensured that the sample design drawing does not have irrelevant architectural elements that are not labeled with element types. Then, the sample label is determined according to the sample architectural design drawing after removing the unlabeled element, so that the sample label only includes the label of the labeled element type. The interference of the unlabeled element can be avoided, and when the difference between the sample label and the predicted type drawing is determined, the accuracy of the difference description is ensured, and the training effect of the initial training model is improved.

[0192] Figure 11 A schematic diagram for obtaining a sample label provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the sample architectural design drawing includes the labeled element type "door", the labeled element type "wall" and the labeled element type "window", and the unlabeled element "table" and "chair". Figure 11 The sample architectural design drawing needs to be operated to remove the unlabeled element, and the processed sample architectural design drawing (i.e., the sample architectural design drawing after removing the unlabeled element) is obtained. Figure 1 The sample architectural design drawing needs to be operated to remove the unlabeled element, and the processed sample architectural design drawing (i.e., the sample architectural design drawing after removing the unlabeled element) is obtained. Figure 1 The sample architectural design drawing needs to be operated to remove the unlabeled element, and the processed sample architectural design drawing (i.e., the sample architectural design drawing after removing the unlabeled element) is obtained. Figure 1 The sample architectural design drawing needs to be operated to remove the unlabeled element, and the processed sample architectural design drawing (i.e., the sample architectural design drawing after removing the unlabeled element) is obtained. Figure 1), it can be seen from the figure that only the building element "door", the building element "wall" and the building element "window" are included in the sample building design Figure 1 According to the sample building design from which the un-labeled elements are removed Figure 1 corresponding sample label 1 is obtained. In the sample label 1, different colors can be used to show different element types. As shown in the figure, the building element "window" is shown in dark gray, the building element "door" is shown in light gray, and the building element "wall" is shown in black.

[0193] By the method for determining a sample label provided above, in the process of determining the sample label, the un-labeled elements in the sample building design are removed, so that only the building elements of the labeled element types are included in the sample building design used to generate the sample label. Thus, the corresponding sample label only contains the labels corresponding to the building elements of the labeled element types, avoiding the interference of data redundancy of the un-labeled elements on the subsequent determination of the difference between the sample label and the predicted type graph, to a certain extent, ensuring the accuracy of the difference description, and further improving the training effect on the initial training model.

[0194] The aforementioned "determining the sample label of the target training sample according to the sample building design from which the un-labeled elements are removed" can highlight the sample label in a manner of emphasizing the sample label of the target training sample, to further improve the accuracy of the difference between the predicted type graph and the corresponding sample label. The difference between the two can be expressed by a loss function. By highlighting the sample label, the expression accuracy of the loss function can be improved. Therefore, in a possible implementation manner, the method for determining a sample label is to add a background color to the sample building design from which the un-labeled elements are removed, to obtain the sample label of the target training sample.

[0195] For sample architectural design drawings that have been stripped of unlabeled elements, a background color is added to highlight the labeled elements within the image. The color difference between the background color and the labeled elements must meet a distinguishability condition. The distinguishability condition indicates the difference between the background color and the labeled elements, ensuring that the two can be distinguished. When determining sample labels, it is desirable to highlight the labeled elements, requiring the maximum possible difference between the background color and the labeled elements. For example, if the labeled element type is white, the background color can be set to black to emphasize the valid content within the sample label (i.e., the labeled elements). The background color layer is placed below the layer containing the labeled elements to avoid affecting the labeled elements' layer.

[0196] In the embodiment of the present application, the element types of the architectural elements marked in the sample architectural design drawing can be highlighted in the form of a mask, and architectural elements of different element types can be marked with masks of different colors. The color difference between the added background color and the mask color corresponding to the architectural element needs to meet the discrimination condition, so that the architectural element can be highlighted against the background color, so that the effective content in the sample label (i.e., the element type of the architectural element marked) is strengthened. In this way, when the difference between the predicted type map and the corresponding sample label is expressed by the loss function, the expression accuracy of the loss function is improved to a certain extent.

[0197] Figure 12 A schematic diagram of a method for determining sample labels provided in an embodiment of the present application, see Figure 12 As shown, for the sample building design with the unlabeled elements removed Figure 1 Add background color to get sample building design with added background color Figure 1 Sample architectural design for adding background color Figure 1 The element type of the architectural element is marked and represented by a mask, see Figure 12 As shown, in the obtained sample label 1, different element types are marked using masks of different colors, that is, the type pixel areas of the building elements are marked by masks of different colors.

[0198] It should be noted that when the element type of the building element marked by the mask is identified, the generated annotation data is a json file, and the structure of the file can include: a building design drawing (or a sample building design drawing), a set of element types of building elements (in text form, such as: wall, door, window, background, door, window, and background), an overlay of the building design drawing and the mask, and a mask image (that is, an image of the building element identified by the mask).

[0199] It should be noted that when the element type of the building element marked by the mask is identified, the generated annotation data is a json file. Figure 13 A schematic diagram of a data format provided by an embodiment of the present application is shown in the figure. The content shown on the right side of the figure corresponds to the content in the picture with "image_id": 1 and "call-center-offices.png" as the file_name. At the same time, the "category_id" on the right side of the figure is 255, and the element type corresponding to the content on the left side is "wall". As can be known from the above, the content described on the right side of the figure is the information of the building element with "category_id" 255 and the element type "wall" in the picture "call-center-offices.png", and the content described in "segmentation" is the position information of the pixel region corresponding to the type of the building element, wherein each two data represents a pixel coordinate, such as 107 and 367, which means the coordinate (107, 367). "area" is used to represent the size of the building element in the picture, "bbox" refers to the bounding box corresponding to the type of the pixel region, which is identified by the two coordinate points (100, 367) and (45, 6), and Segmentation represents the range of mask data. In subsequent steps of obtaining the design drawing to be labeled according to the overlay of the geographic space and the picture to be overlaid, the pixel coordinates can be converted into physical coordinates.

[0200] By the above-mentioned method for determining a sample label, a background color is added to the sample building design drawing from which the unlabeled elements are removed, and the color difference between the color of the background color and the color of the building element of the labeled element type satisfies the degree of distinction condition, so that the building element of the labeled element type can be highlighted, and the effective content (i.e., the element type of the building element marked) in the sample label is emphasized. This is beneficial to improve the accurate expression of the difference between the prediction type picture and the corresponding sample label during the training process of the initial recognition model, and improve the training efficiency of the initial training model.

[0201] Figure 14A schematic diagram for generating three-dimensional modeling provided by an embodiment of the present application, as shown, the process includes: Figure 14

[0202] S301: Labeling building elements.

[0203] The element type labeling of building elements is performed on the sample building design drawing. In the process, the BIM elements (i.e., building elements) in the CAD drawing (i.e., sample building design drawing) can be labeled using a label tool (such as Labelbox), including walls, beams, doors, columns, windows, etc.

[0204] S302: Generating a training sample set.

[0205] The training sample set is generated according to the sample building design drawing review of the completed element type labeling of building elements, which can include initial training samples, newly added training samples obtained through data augmentation processing, and corresponding sample labels.

[0206] S303: Training an initial recognition model.

[0207] The initial training model is trained based on the training sample set to obtain a type recognition model. The initial training model can use a convolutional neural network (CNN) or a recurrent neural network (RNN), etc., and through training, the type recognition model obtained can accurately identify the element type in the plan image.

[0208] S304: Three-dimensional modeling.

[0209] The type recognition model is used to identify the graphic dimension of the plan image of the building design drawing to obtain the type image corresponding to the building design drawing. In the type image, the corresponding building elements are accurately identified by color and area size from the type pixel area. The type image is used to quickly and conveniently label the element type of the building elements in the building design drawing to obtain a labeled design drawing, and a three-dimensional modeling of the target building is obtained based on the labeled design drawing.

[0210] Based on the foregoing Figures 1-14 corresponding embodiments, Figure 15 A device schematic diagram of a three-dimensional modeling generation device provided by an embodiment of the present application, the three-dimensional modeling generation device 1500 includes an acquisition module 1501, a determination module 1502, an identification module 1503, a labeling module 1504, and a generation module 1505;

[0211] ​The acquisition module 1501 is configured to acquire an architectural design drawing of a target building, the architectural design drawing being used to identify an architectural structure of the target building in a two-dimensional plane by means of an architectural element.

[0212] The determination module 1502 is configured to determine a planar image of the architectural design drawing, the planar image being used to embody a graphic feature of the architectural element in the architectural design drawing.

[0213] The identification module 1503 is configured to acquire, according to the planar image, a type drawing corresponding to the architectural design drawing by means of a type recognition model, the type drawing including a type pixel region, the type pixel region being based on a spatial relationship between the type drawing and the architectural design drawing and having a corresponding relationship with the architectural element in the architectural design drawing, the type pixel region identifying an element type of the corresponding architectural element by means of a pixel color, and different pixel colors corresponding to different element types.

[0214] The labeling module 1504 is configured to label the element type of the architectural element in the architectural design drawing based on the type drawing, to obtain a labeled design drawing.

[0215] The generation module 1505 is configured to perform a re-drawing process by means of the labeled design drawing, to generate a three-dimensional model corresponding to the target building.

[0216] In a possible implementation, the determination module 1502 is configured to:

[0217] determine a geographic space drawing according to the architectural design drawing, the geographic space drawing being used to embody geographic space data of the architectural element in the architectural design drawing;

[0218] determine a first adjustment parameter according to an image size of the geographic space drawing and an input size requirement of the type recognition model;

[0219] convert the geographic space drawing into the planar image that meets the input size requirement based on the first adjustment parameter.

[0220] In a possible implementation, the labeling module 1504 is configured to:

[0221] perform inverse conversion of the type drawing based on the first adjustment parameter, to obtain a to-be-stacked drawing, the to-be-stacked drawing being consistent with the image size of the geographic space drawing;

[0222] stack the geographic space drawing and the to-be-stacked drawing, to obtain a to-be-labeled design drawing;

[0223] According to an overlapping degree of the building element and the type pixel region in the to-be-labeled design drawing, an element type of the corresponding type pixel region is labeled for the building element in the to-be-labeled design drawing, and the to-be-labeled design drawing after labeling is taken as the labeled design drawing.

[0224] In a possible implementation, for a target building element in the to-be-labeled design drawing, the labeling module 1504 is configured to:

[0225] determine an overlapping degree of the target building element and a target type pixel region in the to-be-labeled design drawing;

[0226] in response to the overlapping degree meeting a consistency requirement, label an element type of the target building element as an element type corresponding to the target type pixel region.

[0227] In a possible implementation, the determining module 1502 is configured to:

[0228] determine a second adjustment parameter according to an image size of the building design drawing and an input size requirement of the type identification model;

[0229] convert the building design drawing into the planar image meeting the input size requirement based on the second adjustment parameter.

[0230] In a possible implementation, the apparatus further includes a space graph determining module, which is configured to:

[0231] determine a geographic space graph according to the building design drawing, the geographic space graph being used to embody geographic space data of a building element in the building design drawing;

[0232] The labeling module 1504 is configured to:

[0233] determine first position information of a bounding box of a type pixel region in the type graph and second position information of a bounding box of a building element in the geographic space graph;

[0234] convert the type graph according to the first position information and the second position information to obtain a to-be-stacked graph, the to-be-stacked graph being consistent with an image size of the geographic space graph;

[0235] stack the geographic space graph and the to-be-stacked graph to obtain a to-be-labeled design drawing;

[0236] According to an overlapping degree of the building element and the type pixel region in the to-be-labeled design drawing, an element type of the corresponding type pixel region is labeled for the building element in the to-be-labeled design drawing, and the to-be-labeled design drawing after labeling is taken as the labeled design drawing.

[0237] In a possible implementation, the apparatus further includes a training module configured to train a type identification model, the type identification model being trained in the following manner:

[0238] obtain a training sample set, each training sample in the training sample set including a sample architectural design drawing, and a sample label of the training sample being used to identify an element type of an annotated architectural element in the sample architectural design drawing;

[0239] convert the training sample into a sample planar image, the sample planar image being used to embody a graphical feature of an architectural element in the sample architectural design drawing;

[0240] obtain, according to the sample planar image, a predicted type image corresponding to the sample architectural design drawing by using an initial identification model, the predicted type image including a predicted type pixel region, the predicted type pixel region being based on a spatial relationship between the predicted type image and the sample architectural design drawing, having a corresponding relationship with the architectural element in the sample architectural design drawing, and identifying an element type of the corresponding architectural element by using a pixel color, different pixel colors corresponding to different element types;

[0241] train the initial identification model according to a difference between the predicted type image and the corresponding sample label, to obtain the type identification model.

[0242] In a possible implementation, the training module is configured to:

[0243] obtain an initial training sample;

[0244] perform data enhancement processing on the initial training sample to obtain an added training sample, the data enhancement processing including at least one of rotation, flipping, scaling, translation, and noise adding;

[0245] generate the training sample set according to the initial training sample and the added training sample.

[0246] In a possible implementation, when the data enhancement processing includes translation, the training module is configured to:

[0247] move a position of an architectural element in the initial training sample according to a requirement of architectural design to obtain the added training sample.

[0248] In a possible implementation, the training module includes a label determination module, and for a target training sample in the training sample set, the label determination module is configured to:

[0249] remove un-labeled elements in the sample architectural design drawing of the target training sample, the un-labeled elements being elements in the sample architectural design drawing other than the labeled element type of architectural elements;

[0250] determine the sample label of the target training sample according to the sample architectural design drawing from which the un-labeled elements are removed.

[0251] In a possible implementation, the label determining module is configured to:

[0252] add a background color to the sample architectural design drawing from which the un-labeled elements are removed to obtain the sample label of the target training sample, the layer of the background color being below the layer of the labeled element type of architectural elements, and the color difference between the background color and the color of the labeled element type of architectural elements satisfying a degree of distinction condition.

[0253] According to the three-dimensional modeling generation apparatus provided above, the architectural design scheme of a target building can be embodied by an architectural design drawing, and the architectural design drawing identifies the architectural structure of the target building in a two-dimensional plane through architectural elements. Since different architectural elements have different graphic features in the architectural design drawing, the architectural elements can be accurately identified in the graphic dimension according to the plan view image mainly embodying the graphic features of the architectural elements. In order to quickly and accurately label the element types of the architectural elements, the plan view image of the architectural design drawing needs to be determined, and then the plan view image is identified in the graphic dimension by a category recognition model to obtain a type view corresponding to the architectural design drawing. The type view includes a type pixel region, the type pixel region has a corresponding relationship with the architectural elements in the architectural design drawing based on the spatial relationship between the type view and the architectural design drawing, and different element types are identified by different pixel colors. Since the type pixel region accurately identifies the corresponding architectural elements by color and area size, the element types of the architectural elements in the architectural design drawing can be quickly and conveniently labeled by using the type view to obtain a labeled design drawing, and the three-dimensional modeling of the target building is obtained based on the labeled design drawing. The element types of the plan view image are identified in the graphic dimension by the type recognition model, without considering other complex parameters in the architectural design drawing, and the element types can be automatically and accurately identified by the model understanding the graphic features of the architectural elements, which greatly improves the labeling efficiency and accuracy of the architectural elements in the architectural design drawing and ensures the efficiency and precision of the modeling.

[0254] Embodiments of the present application also provide a computer device including a terminal device or a server, and the foregoing three-dimensional modeling generation apparatus can be configured in the computer device. The computer device will be described below with reference to the accompanying drawings.

[0255] If the computer device is a terminal device, please refer to Figure 16As shown, the embodiments of the present application provide a terminal device, taking a mobile phone as an example:

[0256] Figure 16 As shown is a block diagram of part of the structure of the mobile phone provided by the embodiments of the present application. Referring to Figure 16 , the mobile phone includes: a radio frequency (RF) circuit 1410, a memory 1420, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a wireless fidelity (WiFi) module 1470, a processor 1480, and a power supply 1490, and the like. Those skilled in the art can understand that Figure 16 The structure of the mobile phone shown in the above embodiment does not constitute a limitation on the mobile phone, and can include more or fewer components than those shown in the figure, or combine some components, or different arrangement of components.

[0257] The various components of the mobile phone will be specifically introduced below Figure 16 :

[0258] The RF circuit 1410 can be used for receiving and sending signals in the process of information or call, in particular, receiving the downlink information of the base station and processing by the processor 1480; in addition, sending the uplink data to the base station.

[0259] The memory 1420 can be used for storing software programs and modules, and the processor 1480 executes various functions and data processing of the mobile phone by running the software programs and modules stored in the memory 1420. The memory 1420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory 1420 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0260] The input unit 1430 can be used for receiving input digital or character information, and generating key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1430 can include a touch panel 1431 and other input devices 1432.

[0261] The display unit 1440 can be used for displaying information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1440 can include a display panel 1441.

[0262] The mobile phone may also include at least one sensor 1450, such as a light sensor, a motion sensor, and other sensors.

[0263] The audio circuit 1460 , the speaker 1461 , and the microphone 1462 can provide an audio interface between the user and the mobile phone.

[0264] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 1470, providing users with wireless broadband Internet access.

[0265] The processor 1480 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 1420 and calling data stored in the memory 1420.

[0266] The mobile phone also includes a power supply 1490 (such as a battery) for supplying power to various components.

[0267] In this embodiment, the processor 1480 included in the terminal device is also used to execute the steps in the methods of each embodiment of the present application.

[0268] If the computer device is a server, this embodiment of the application also provides a server, see Figure 17 As shown, Figure 17 The structural diagram of the server 1500 provided in the embodiment of the present application, the server 1500 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1522 (for example, one or more processors) and a memory 1532, and one or more storage media 1530 (for example, one or more mass storage devices) for storing application programs 1542 or data 1544. Among them, the memory 1532 and the storage medium 1530 can be temporary storage or permanent storage. The program stored in the storage medium 1530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1522 can be configured to communicate with the storage medium 1530 to execute a series of instruction operations in the storage medium 1530 on the server 1500.

[0269] The server 1500 may also include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input and output interfaces 1558, and / or one or more operating systems 1541, such as Windows Server 2003. TMMac OS X TM Unix TM Linux TM FreeBSD TM and so on.

[0270] The steps performed by the server in the above embodiments can be based on the server structure shown in Figure 17

[0271] In addition, the embodiments of the present application further provide a storage medium for storing a computer program, and the computer program is used for executing the method provided by the above embodiments.

[0272] The embodiments of the present application further provide a computer program product including a computer program, which, when running on a computer device, enables the computer device to execute the method provided by the above embodiments.

[0273] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by a program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium can be at least one of the following mediums: read-only memory (English: Read-only Memory, abbreviation: ROM), RAM, magnetic disk or optical disk and various mediums that can store computer programs.

[0274] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as processing circuit or memory) or combination thereof. Similarly, one processor (or multiple processors or memory) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that contains the function of the module or unit.

[0275] ​It should be noted that each of the embodiments of the present specification is described in a progressive manner, and the same or similar parts between each embodiment can be mutually referred to, and each embodiment focuses on the difference from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, they are described more simply, and the relevant parts can be referred to the part of the description of the method embodiments. The above-described device and system embodiments are only illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiments according to the actual needs. Those skilled in the art can understand and implement it without creative labor.

[0276] The above is only one specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Moreover, on the basis of the implementation manners provided by the above aspects, further combinations can be made to provide more implementation manners. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A three-dimensional modeling generation method characterized by, The method comprises: obtaining an architectural design drawing of a target building, the architectural design drawing being used to identify an architectural structure of the target building in a two-dimensional plane through architectural elements; determining a planar image of the architectural design drawing, the planar image being used to embody graphical features of architectural elements in the architectural design drawing; obtaining a type drawing corresponding to the architectural design drawing through a type recognition model according to the planar image, the type drawing comprising type pixel regions, the type pixel regions having a corresponding relationship with the architectural elements in the architectural design drawing based on a spatial relationship between the type drawing and the architectural design drawing, the type pixel regions identifying element types of the corresponding architectural elements through pixel colors, and different pixel colors corresponding to different element types; annotating the architectural elements in the architectural design drawing based on the type drawing to obtain an annotated design drawing; generating a three-dimensional model corresponding to the target building through a lofting process based on the annotated design drawing.

2. The method of claim 1, wherein, The determining of the planar image of the architectural design drawing comprises: determining a geospatial graph based on the architectural design drawing, the geospatial graph being used to embody geospatial data of the architectural elements in the architectural design drawing; determining a first adjustment parameter according to an image size of the geospatial graph and an input size requirement of the type recognition model; converting the geospatial graph into the planar image meeting the input size requirement based on the first adjustment parameter.

3. The method of claim 2, wherein, The annotating of the architectural elements in the architectural design drawing based on the type drawing to obtain an annotated design drawing comprises: performing inverse conversion of the type drawing based on the first adjustment parameter to obtain a to-be-stacked graph, the to-be-stacked graph having an image size consistent with that of the geospatial graph; stacking the geospatial graph and the to-be-stacked graph to obtain a to-be-annotated design drawing; annotating element types of corresponding type pixel regions in the to-be-annotated design drawing according to an overlapping degree of the architectural elements and the type pixel regions in the to-be-annotated design drawing, and taking the annotated to-be-annotated design drawing as the annotated design drawing.

4. The method of claim 3, wherein, For a target architectural element in the to-be-annotated design drawing, the element type of the target architectural element is annotated in the following manner: determining an overlapping degree of the target architectural element and a target type pixel region in the to-be-annotated design drawing; in response to the overlapping degree meeting a consistency requirement, annotating the element type of the target architectural element as an element type corresponding to the target type pixel region.

5. The method of claim 1, wherein, The determining of the planar image of the architectural design drawing comprises: determining a second adjustment parameter according to an image size of the architectural design drawing and an input size requirement of the type recognition model; converting the architectural design drawing into the planar image meeting the input size requirement based on the second adjustment parameter.

6. The method of claim 5, wherein, The method further comprises: determining a geospatial graph based on the architectural design drawing, the geospatial graph being used to embody geospatial data of the architectural elements in the architectural design drawing; The element type of the building element in the architectural design drawing is labeled based on the type graph to obtain a labeled design drawing, including: determining first position information of a bounding box of a type pixel region in the type graph and second position information of a bounding box of a building element in the geographic space graph; converting the type graph according to the first position information and the second position information to obtain a to-be-stacked graph, the to-be-stacked graph being consistent with an image size of the geographic space graph; stacking the geographic space graph and the to-be-stacked graph to obtain a to-be-labeled design drawing; labeling the element type of the corresponding type pixel region in the to-be-labeled design drawing according to an overlapping degree of the building element and the type pixel region in the to-be-labeled design drawing, and taking the to-be-labeled design drawing after labeling as the labeled design drawing.

7. The method of claim 1, wherein, The type recognition model is obtained by training in the following manner: obtaining a training sample set, a training sample in the training sample set including a sample architectural design drawing, and a sample label of the training sample being used to identify an element type of a building element in the sample architectural design drawing that has been labeled; converting the training sample into a sample planar image, the sample planar image being used to embody a graphic feature of the building element in the sample architectural design drawing; obtaining a predicted type graph corresponding to the sample architectural design drawing by an initial recognition model according to the sample planar image, the predicted type graph including a predicted type pixel region, the predicted type pixel region being based on a spatial relationship between the predicted type graph and the sample architectural design drawing, having a corresponding relationship with the building element in the sample architectural design drawing, and identifying the element type of the corresponding building element by pixel color, different pixel colors corresponding to different element types; training the initial recognition model according to a difference between the predicted type graph and the corresponding sample label to obtain the type recognition model.

8. The method of claim 7, wherein, The obtaining of the training sample set includes: obtaining an initial training sample; performing data enhancement processing on the initial training sample to obtain an added training sample, the data enhancement processing including at least one of rotation, flipping, scaling, translation, and noise addition; generating the training sample set according to the initial training sample and the added training sample.

9. The method of claim 8, wherein, When the data enhancement processing includes translation, the performing of the data enhancement processing on the initial training sample to obtain the added training sample includes: moving a position of a building element in the initial training sample in a manner conforming to architectural design requirements to obtain the added training sample.

10. The method of claim 7, wherein, For a target training sample in the training sample set, a sample label of the target training sample is determined in the following manner: removing a non-labeled element in a sample architectural design drawing of the target training sample, the non-labeled element being an element other than a building element of a labeled element type in the sample architectural design drawing; determining the sample label of the target training sample according to the sample architectural design drawing from which the non-labeled element is removed.

11. The method of claim 10, wherein, The determining of the sample label of the target training sample according to the sample architectural design drawing from which the non-labeled element is removed includes: Adding a background color to the sample architectural design drawing from which the unlabeled element is removed to obtain a sample label of the target training sample, a layer of the background color is below a layer in which the labeled element type building element is located, and a color difference between the background color and a color of the labeled element type building element meets a discrimination condition.

12. A three-dimensional modeling generation device, characterized by comprising: The device comprises an acquisition module, a determination module, an identification module, a labeling module, and a generation module. The acquisition module is configured to acquire an architectural design drawing of a target building, the architectural design drawing being used to identify an architectural structure of the target building in a two-dimensional plane through building elements. The determination module is configured to determine a planar image of the architectural design drawing, the planar image being used to embody graphical features of building elements in the architectural design drawing. The identification module is configured to obtain a type graph corresponding to the architectural design drawing through a type recognition model according to the planar image, the type graph comprising a type pixel region, the type pixel region being based on a spatial relationship between the type graph and the architectural design drawing and having a corresponding relationship with the building elements in the architectural design drawing, the type pixel region identifying an element type of the corresponding building element through a pixel color, and different pixel colors corresponding to different element types. The labeling module is configured to label the element types of the building elements in the architectural design drawing based on the type graph to obtain a labeled design drawing. The generation module is configured to perform a flip processing through the labeled design drawing to generate a three-dimensional modeling corresponding to the target building.

13. A computer device, comprising: The computer device comprises a processor and a memory: The memory is configured to store a computer program; The processor is configured to execute the method according to any one of claims 1-11 according to the computer program.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store a computer program, and the computer program is configured to implement the method according to any one of claims 1-11 when executed by a computer device.

15. A computer program product comprising a computer program which, when executed on a computer device, causes the computer device to perform the method according to any one of claims 1-11.

Images