Construction method and system for 3D model of smart city

Abstract

The invention relates to the field of 3D modeling, and particularly discloses a smart city 3D model construction method and system, and the method comprises the steps: carrying out the feature extraction of three-dimensional point cloud data of a to-be-modeled city region through employing a deep learning-based data processing technology, capturing the three-dimensional structure semantic features of the city region, and carrying out the feature extraction of the three-dimensional point cloud data of the to-be-modeled city region; meanwhile, in combination with natural language description about urban area characteristics input by a user, characteristic modulation optimization processing is carried out on the three-dimensional structure of the urban area, the detail expression of the three-dimensional model is further optimized, and therefore intelligent construction of the urban area three-dimensional model is achieved. Through the mode, the geometric information of the urban area and the urban characteristic language description information can be effectively integrated, so that the generated urban area three-dimensional model can reflect the urban internal structure and functional layout more accurately, and the application value of the model in urban planning and management is improved.

Description

Technical Field

[0001] The present application relates to the field of 3D modeling, and more specifically, to a method and system for constructing a smart city 3D model. Background Art

[0002] With the rapid development of information technology, the construction of smart cities has become an important trend in global urban development. Smart cities aim to improve urban management efficiency, improve the quality of life of residents, and promote sustainable economic development by integrating advanced information and communication technologies (ICT). In the process of building smart cities, three-dimensional city models, as an important part of the basic geographic information system (GIS), play an irreplaceable role in urban planning, disaster prevention, traffic management, etc.

[0003] Existing methods for constructing 3D city models mainly include reconstruction of 3D point cloud data obtained based on remote sensing technologies such as aerial photogrammetry and LiDAR scanning. These methods can accurately restore the topography of cities and the appearance of buildings, but there are still some limitations. For example, traditional 3D point cloud data reconstruction technology mainly focuses on the reconstruction of geometric shapes, and has limited ability to express detailed information such as the internal structure and functional layout of the city. It is usually difficult to take into account urban characteristics such as the function of buildings and the type of roads, resulting in the generated 3D model being not accurate enough in expressing detailed information.

[0004] Therefore, an optimized method for constructing a smart city 3D model is expected. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a method and system for constructing a 3D model of a smart city, which uses a data processing technology based on deep learning to extract features from the three-dimensional point cloud data of the urban area to be modeled, captures the semantic features of the three-dimensional structure of the urban area, and combines the natural language description of the characteristics of the urban area input by the user to perform feature modulation optimization processing on the three-dimensional structure of the urban area, and further optimizes the detailed expression of the three-dimensional model, thereby realizing the intelligent construction of the three-dimensional model of the urban area. In this way, the geometric information of the urban area and the language description information of the city's characteristics can be effectively integrated, so that the generated three-dimensional model of the urban area can more accurately reflect the internal structure and functional layout of the city, and improve the application value of the model in urban planning and management.

[0006] According to one aspect of the present application, a method for constructing a smart city 3D model is provided, which includes: Acquire a three-dimensional point cloud of the urban area to be modeled collected by a laser detector, and a natural language description of the characteristics of the urban area to be modeled input by a user; Based on the three-dimensional point cloud, extracting the features of the three-dimensional model of the urban area to obtain a local semantic coding feature map of the three-dimensional model of the urban area; Performing semantic coding on the characteristic natural language description of the urban area to be modeled to obtain a semantic coding feature vector of the urban area characteristics; Based on the semantic coding feature vector of the urban area characteristics, semantically jointly optimizing the local semantic coding feature map of the urban area three-dimensional model to obtain a multimodal jointly optimized semantic coding feature map of the urban area model; Based on the multimodal joint optimization semantic coding feature map of the urban area model, an optimized three-dimensional model of the urban area is generated.

[0007] In the above-mentioned method for constructing a 3D model of a smart city, based on the semantic coding feature vector of the urban area characteristics, the local semantic coding feature map of the urban area three-dimensional model is semantically jointly optimized, including: extracting the intrinsic semantic commonality features between the semantic coding feature vector of the urban area characteristics and the local semantic coding feature map of the urban area three-dimensional model to obtain the intrinsic semantic commonality feature vector of the urban area characteristics-three-dimensional model; based on the intrinsic semantic commonality feature vector of the urban area characteristics-three-dimensional model, performing dual feature modulation on the local semantic coding feature map of the urban area three-dimensional model to obtain the multimodal jointly optimized semantic coding feature map of the urban area model.

[0008] In the above-mentioned method for constructing a smart city 3D model, based on the three-dimensional point cloud, the three-dimensional model features of the urban area are extracted to obtain a local semantic coding feature map of the three-dimensional model of the urban area, including: constructing a three-dimensional model of the urban area based on the three-dimensional point cloud; inputting the three-dimensional model of the urban area into a city model semantic feature extractor based on a three-dimensional convolutional neural network model to obtain a local semantic coding feature map of the three-dimensional model of the urban area.

[0009] In the above-mentioned method for constructing a 3D model of a smart city, the characteristic natural language description of the urban area to be modeled is semantically encoded to obtain a semantically encoded feature vector of the urban area characteristics, including: inputting the characteristic natural language description of the urban area to be modeled into a semantic encoder based on the Bert model for semantic encoding to obtain the semantically encoded feature vector of the urban area characteristics.

[0010] In the above-mentioned method for constructing a 3D model of a smart city, the intrinsic semantic common features between the semantic coding feature vector of the urban area characteristics and the local semantic coding feature map of the urban area three-dimensional model are extracted to obtain the intrinsic semantic common feature vector of the urban area characteristics-three-dimensional model, including: performing feature distillation on the local semantic coding feature map of the urban area three-dimensional model to obtain the semantic distillation feature vector of the urban area three-dimensional model; inputting the semantic coding feature vector of the urban area characteristics and the semantic distillation feature vector of the urban area three-dimensional model into a multimodal implicit association feature capture network to obtain the intrinsic semantic common feature vector of the urban area characteristics-three-dimensional model.

[0011] In the above-mentioned method for constructing a smart city 3D model, feature distillation is performed on the local semantic coding feature map of the urban area three-dimensional model to obtain a semantic distillation feature vector of the urban area three-dimensional model, including: point convolution encoding is performed on the local semantic coding feature map of the urban area three-dimensional model to obtain a channel-modulated local semantic coding feature map of the urban area three-dimensional model; global mean pooling processing is performed on the local semantic coding feature map of the channel-modulated urban area three-dimensional model along the channel dimension and then input into a Sigmoid function for activation processing to obtain the semantic distillation feature vector of the urban area three-dimensional model.

[0012] In the above-mentioned method for constructing a 3D model of a smart city, based on the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector, dual feature modulation is performed on the local semantic coding feature map of the urban area three-dimensional model to obtain the multimodal joint optimization semantic coding feature map of the urban area model, including: inputting the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector into a feature preliminary screening module based on a gating unit to obtain a first-level urban area three-dimensional model semantic feature preliminary screening weight vector; based on the first-level urban area three-dimensional model semantic feature preliminary screening weight vector, performing preliminary feature screening on the local semantic coding feature map of the urban area three-dimensional model to obtain a preliminary screening urban area three-dimensional model local semantic coding feature map; inputting the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector into a feature secondary screening module based on a multi-level gating unit to obtain a second-level urban area three-dimensional model semantic feature screening weight vector; based on the second-level urban area three-dimensional model semantic feature screening weight vector, performing secondary feature screening on the local semantic coding feature map of the preliminary screening urban area three-dimensional model to obtain the multimodal joint optimization semantic coding feature map of the urban area model.

[0013] In the above-mentioned method for constructing a smart city 3D model, the urban area characteristics-three-dimensional model intrinsic semantic commonality feature vector is input into a feature preliminary screening module based on a gating unit to obtain a first-level urban area three-dimensional model semantic feature preliminary screening weight vector, including: using a first weight parameter matrix to multiply the urban area characteristics-three-dimensional model intrinsic semantic commonality feature vector, and then dot-adding it with a first bias item to obtain a first-level urban area characteristics-three-dimensional model intrinsic commonality modulation vector; inputting the first-level urban area characteristics-three-dimensional model intrinsic commonality modulation vector into a Sigmoid function for nonlinear activation processing to obtain a first-level urban area characteristics-three-dimensional model intrinsic commonality modulation activation vector; inputting the first-level urban area characteristics-three-dimensional model intrinsic commonality modulation activation vector into a mask unit to obtain the first-level urban area three-dimensional model semantic feature preliminary screening weight vector.

[0014] In the above-mentioned method for constructing a smart city 3D model, the urban area characteristics-three-dimensional model intrinsic semantic commonality feature vector is input into a feature secondary screening module based on a multi-level gating unit to obtain a secondary urban area three-dimensional model semantic feature screening weight vector, including: using a second weight parameter matrix to multiply the first-level urban area characteristics-three-dimensional model intrinsic commonality modulation vector, and then dot-adding it with a second bias item to obtain a secondary urban area characteristics-three-dimensional model intrinsic commonality modulation vector; inputting the secondary urban area characteristics-three-dimensional model intrinsic commonality modulation vector into a Sigmoid function for nonlinear activation processing to obtain a secondary urban area characteristics-three-dimensional model intrinsic commonality modulation activation vector; inputting the secondary urban area characteristics-three-dimensional model intrinsic commonality modulation activation vector into the mask unit to obtain the secondary urban area three-dimensional model semantic feature screening weight vector.

[0015] In the above-mentioned method for constructing a smart city 3D model, an optimized urban area three-dimensional model is generated based on the multimodal joint optimization semantic coding feature map of the urban area model, including: inputting the multimodal joint optimization semantic coding feature map of the urban area model into an urban model optimization engine based on a diffusion model to obtain the optimized urban area three-dimensional model.

[0016] According to another aspect of the present application, a system for constructing a smart city 3D model is provided, comprising: A data acquisition module, used to acquire a three-dimensional point cloud of the urban area to be modeled acquired by a laser detector, and a natural language description of the characteristics of the urban area to be modeled input by a user; An urban area three-dimensional feature extraction module, used to extract the features of the urban area three-dimensional model based on the three-dimensional point cloud to obtain a local semantic coding feature map of the urban area three-dimensional model; An urban area characteristic semantic coding module, used for semantically coding the characteristic natural language description of the urban area to be modeled to obtain a characteristic semantic coding feature vector of the urban area characteristic; A semantic joint optimization module, used for performing semantic joint optimization on the local semantic coding feature map of the urban area three-dimensional model based on the urban area characteristic semantic coding feature vector to obtain a multimodal joint optimized semantic coding feature map of the urban area model; The optimization model generation module is used to generate an optimized urban area three-dimensional model based on the multi-modal joint optimization semantic coding feature map of the urban area model.

[0017] Compared with the prior art, the method and system for constructing a 3D model of a smart city provided by this application uses a data processing technology based on deep learning to extract features from the 3D point cloud data of the urban area to be modeled, captures the semantic features of the 3D structure of the urban area, and combines the natural language description of the characteristics of the urban area input by the user to perform feature modulation optimization processing on the 3D structure of the urban area, further optimizes the detailed performance of the 3D model, and thus realizes the intelligent construction of the 3D model of the urban area. In this way, the geometric information of the urban area and the language description information of the urban characteristics can be effectively integrated, so that the generated 3D model of the urban area can more accurately reflect the internal structure and functional layout of the city, and improve the application value of the model in urban planning and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 This is a flowchart of a method for constructing a smart city 3D model according to an embodiment of the present application.

[0020] Figure 2 Schematic diagram of data flow of a method for constructing a 3D model of a smart city according to an embodiment of the present application.

[0021] Figure 3 This is a flowchart of sub-step S2 of the method for constructing a smart city 3D model according to an embodiment of the present application.

[0022] Figure 4 This is a flowchart of sub-step S4 of the method for constructing a smart city 3D model according to an embodiment of the present application.

[0023] Figure 5This is a flowchart of sub-step S42 of the method for constructing a smart city 3D model according to an embodiment of the present application.

[0024] Figure 6 It is a block diagram of a system for constructing a smart city 3D model according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0026] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0027] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0028] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0029] In response to the above technical problems, this application proposes an optimized method for constructing a 3D model of a smart city, which uses data processing technology based on deep learning to extract features from the three-dimensional point cloud data of the urban area to be modeled, captures the semantic features of the three-dimensional structure of the urban area, and combines the natural language description of the characteristics of the urban area input by the user to perform feature modulation optimization processing on the three-dimensional structure of the urban area, further optimizes the detailed expression of the three-dimensional model, and thus realizes the intelligent construction of the three-dimensional model of the urban area. In this way, the geometric information of the urban area and the language description information of the urban characteristics can be effectively integrated, so that the generated three-dimensional model of the urban area can more accurately reflect the internal structure and functional layout of the city, and improve the application value of the model in urban planning and management.

[0030] Figure 1This is a flowchart of a method for constructing a smart city 3D model according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of the method for constructing a 3D model of a smart city according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the method for constructing the smart city 3D model includes the following steps: S1, obtaining a three-dimensional point cloud of the urban area to be modeled collected by a laser detector, and a natural language description of the characteristics of the urban area to be modeled input by a user; S2, based on the three-dimensional point cloud, extracting the characteristics of the urban area three-dimensional model to obtain a local semantic coding feature map of the urban area three-dimensional model; S3, semantically encoding the characteristic natural language description of the urban area to be modeled to obtain a semantic coding feature vector of urban area characteristics; S4, based on the semantic coding feature vector of urban area characteristics, semantically jointly optimizing the local semantic coding feature map of the urban area three-dimensional model to obtain a multimodal jointly optimized semantic coding feature map of the urban area model; S5, generating an optimized urban area three-dimensional model based on the multimodal jointly optimized semantic coding feature map of the urban area model.

[0031] In the above-mentioned method for constructing a 3D model of a smart city, the step S1 is to obtain a three-dimensional point cloud of the urban area to be modeled collected by a laser detector, and a natural language description of the characteristics of the urban area to be modeled input by the user. It should be understood that a laser detector (such as LiDAR) can provide high-density and high-precision three-dimensional point cloud data, accurately reflecting spatial information such as the topography and building outlines of the urban area, and is the basis for constructing a true and accurate three-dimensional urban model. The natural language description of the characteristics of the urban area to be modeled can provide important semantic information about the urban area, such as functional area division (commercial area, residential area), road type, distribution of special facilities, etc. Therefore, in the technical solution of the present application, by combining the three-dimensional point cloud data and natural language description of the urban area to be modeled, it is possible to achieve multi-dimensional and multi-angle optimization of the three-dimensional model of the urban area, while ensuring the geometric accuracy of the model, improving its ability to express detailed information such as the internal structure and functional layout of the city, thereby providing a more accurate and practical tool for urban planning and management.

[0032] In the above-mentioned method for constructing a 3D model of a smart city, the step S2 extracts the features of the 3D model of the urban area based on the 3D point cloud to obtain a local semantic coding feature map of the 3D model of the urban area. Figure 3 FIG. 1 is a flowchart of sub-step S2 of the method for constructing a 3D model of a smart city according to an embodiment of the present application. Figure 3As shown, the step S2 includes the steps of: S21, constructing a three-dimensional model of the urban area based on the three-dimensional point cloud; S22, inputting the three-dimensional model of the urban area into a city model semantic feature extractor based on a three-dimensional convolutional neural network model to obtain a local semantic coding feature map of the three-dimensional model of the urban area.

[0033] Specifically, the step S21 constructs a three-dimensional model of the urban area based on the three-dimensional point cloud. It should be understood that the three-dimensional point cloud data is the basic data source for constructing the three-dimensional model of the city. In order to reconstruct the real form of the city, the present application uses three-dimensional reconstruction technology to construct a model of the three-dimensional point cloud to generate a preliminary three-dimensional model of the urban area, thereby laying the foundation for subsequent three-dimensional model optimization processing. In a specific implementation of the present application, the three-dimensional point cloud is processed using the three-dimensional real scene modeling software ContextCapture to construct the three-dimensional model of the urban area.

[0034] Specifically, in step S22, the urban area three-dimensional model is input into the urban model semantic feature extractor based on the three-dimensional convolutional neural network model to obtain the local semantic coding feature map of the urban area three-dimensional model. It should be understood that in order to achieve the optimization processing of the urban area three-dimensional model, it is necessary to further extract the three-dimensional structural features of the urban area. Based on this, the present application adopts a three-dimensional convolutional neural network (CNN) model under the deep learning framework to construct a city model semantic feature extractor. Specifically, the three-dimensional convolutional neural network is trained based on a large amount of annotated three-dimensional model data. By using a three-dimensional convolution kernel to perform a local sliding convolution operation in the three-dimensional space of the urban area three-dimensional model, the local three-dimensional structural semantic features of the urban area can be effectively captured, such as the shape and size of the building, the layout of the road, and the distribution of the green belt, thereby generating a local semantic coding feature map of the urban area three-dimensional model. It is worth mentioning that the three-dimensional convolutional neural network (CNN) model is a deep learning model. This network structure is particularly suitable for processing data with volume information. The three-dimensional convolutional neural network can capture spatial features in three dimensions and is particularly suitable for processing the three-dimensional model of the urban area.

[0035] In the above-mentioned method for constructing a 3D model of a smart city, the step S3 is to semantically encode the characteristic natural language description of the urban area to be modeled to obtain a semantic encoding feature vector of the urban area characteristics. In a specific example of the present application, the step S3 includes: inputting the characteristic natural language description of the urban area to be modeled into a semantic encoder based on the Bert model for semantic encoding to obtain the characteristic semantic encoding feature vector of the urban area. It should be understood that in urban planning, the characteristic natural language description of the urban area to be modeled input by the user can provide key information about regional functions, land use, traffic demand, etc., thereby enhancing the comprehensive understanding of the urban area. Therefore, in order to further improve the expression ability of the three-dimensional model of the urban area, the present application also introduces natural language processing technology, by semantically encoding the characteristic natural language description of the urban area to be modeled, extracting the contextual semantic features therein, and generating a characteristic semantic encoding feature vector of the urban area, so as to facilitate the joint optimization of features with the local semantic encoding feature map of the three-dimensional model of the urban area. In a specific example of the present application, the characteristic natural language description of the urban area to be modeled is input into a semantic encoder based on the Bert model for semantic encoding to obtain the characteristic semantic encoding feature vector of the urban area. It is worth mentioning that the Bert model is a pre-trained language model based on the Transformer architecture design, which captures contextual information in the text through bidirectional training. Unlike traditional unidirectional language models, the Bert model can consider both the left and right contexts of words at the same time, which makes it more effective in understanding sentence structure and semantic information.

[0036] In the above-mentioned method for constructing a 3D model of a smart city, the step S4, based on the semantic coding feature vector of the urban area characteristics, performs semantic joint optimization on the local semantic coding feature map of the three-dimensional model of the urban area to obtain a multimodal joint optimization semantic coding feature map of the urban area model. Specifically, the present application further utilizes the semantic coding feature vector of the urban area characteristics to perform feature modulation optimization processing on the local semantic coding feature map of the three-dimensional model of the urban area, so as to combine the semantic information in the natural language description of the characteristics of the urban area to be modeled with the geometric information of the three-dimensional model, so as to accurately express the internal structure and functional layout of the urban area in the geometric structure features of the three-dimensional model, and realize the optimization of the details of the three-dimensional model of the urban area. It is worth mentioning that the present application proposes a cross-modal feature joint encoding method based on intrinsic semantic commonality features, which guides the feature modulation process of the local semantic coding feature map of the three-dimensional model of the urban area by learning the intrinsic semantic commonality features between the semantic coding feature vector of the urban area characteristics and the local semantic coding feature map of the three-dimensional model of the urban area, so as to effectively identify and strengthen the association between the geometric structure features of the three-dimensional model and its functional layout, and realize deeper semantic fusion.

[0037] Figure 4 FIG. 4 is a flowchart of sub-step S4 of the method for constructing a 3D model of a smart city according to an embodiment of the present application. Figure 4 As shown, the step S4 includes the steps of: S41, extracting the intrinsic semantic common features between the semantic coding feature vector of the urban area characteristics and the local semantic coding feature map of the urban area three-dimensional model to obtain the intrinsic semantic common feature vector of the urban area characteristics-three-dimensional model; S42, based on the intrinsic semantic common feature vector of the urban area characteristics-three-dimensional model, performing dual feature modulation on the local semantic coding feature map of the urban area three-dimensional model to obtain the multimodal joint optimization semantic coding feature map of the urban area model.

[0038] Specifically, the step S41 extracts the intrinsic semantic common features between the semantic coding feature vector of the urban area characteristics and the local semantic coding feature map of the urban area three-dimensional model to obtain the intrinsic semantic common feature vector of the urban area characteristics-three-dimensional model. In a specific example of the present application, the step S41 includes: performing feature distillation on the local semantic coding feature map of the urban area three-dimensional model to obtain the semantic distillation feature vector of the urban area three-dimensional model; inputting the semantic coding feature vector of the urban area characteristics and the semantic distillation feature vector of the urban area three-dimensional model into the multimodal implicit association feature capture network to obtain the intrinsic semantic common feature vector of the urban area characteristics-three-dimensional model. That is, firstly, feature distillation is performed on the local semantic coding feature map of the urban area three-dimensional model to remove redundant information therein, obtain a more compact and representative semantic distillation feature vector of the urban area three-dimensional model, and unify the feature dimensions to promote effective feature interaction in subsequent stages. Next, the semantic encoding feature vector of the urban area characteristics and the semantic distillation feature vector of the urban area three-dimensional model are input into the implicit association feature capture network to identify and capture the potential common patterns or structures between the two, thereby constructing the intrinsic semantic commonality feature vector of the urban area characteristics-three-dimensional model.

[0039] In a specific example of the present application, feature distillation is performed on the local semantic coding feature map of the urban area three-dimensional model to obtain a semantic distillation feature vector of the urban area three-dimensional model, including: performing point convolution encoding on the local semantic coding feature map of the urban area three-dimensional model to obtain a channel-modulated local semantic coding feature map of the urban area three-dimensional model; performing global mean pooling processing on the local semantic coding feature map of the channel-modulated urban area three-dimensional model along the channel dimension and then inputting the global mean pooling processing into a Sigmoid function for activation processing to obtain the semantic distillation feature vector of the urban area three-dimensional model.

[0040] In a specific example of the present application, the semantic encoding feature vector of the urban area characteristics and the semantic distillation feature vector of the urban area three-dimensional model are input into a multimodal implicit association feature capture network to obtain the intrinsic semantic commonality feature vector of the urban area characteristics-three-dimensional model, including: cascading and fusing the semantic encoding feature vector of the urban area characteristics and the semantic distillation feature vector of the urban area three-dimensional model and inputting them into a neural network layer based on a tanh function to obtain the intrinsic semantic commonality feature vector of the urban area characteristics-three-dimensional model.

[0041] Specifically, the step S42 performs dual feature modulation on the local semantic coding feature map of the urban area three-dimensional model based on the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector to obtain the multi-modal joint optimization semantic coding feature map of the urban area model. Figure 5 FIG. 4 is a flowchart of sub-step S42 of the method for constructing a smart city 3D model according to an embodiment of the present application. Figure 5 As shown, the step S42 includes the steps of: S421 inputting the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector into a feature primary screening module based on a gating unit to obtain a primary urban area three-dimensional model semantic feature primary screening weight vector; S422, based on the primary urban area three-dimensional model semantic feature primary screening weight vector, performing a primary feature screening on the local semantic coding feature map of the urban area three-dimensional model to obtain a primary screening urban area three-dimensional model local semantic coding feature map; S423, inputting the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector into a feature secondary screening module based on a multi-level gating unit to obtain a secondary urban area three-dimensional model semantic feature screening weight vector; S424, based on the secondary urban area three-dimensional model semantic feature screening weight vector, performing a secondary feature screening on the primary screening urban area three-dimensional model local semantic coding feature map to obtain the multimodal joint optimization semantic coding feature map of the urban area model.

[0042] In a specific example of the present application, the step S421 includes: multiplying the urban area characteristics-three-dimensional model intrinsic semantic commonality feature vector using the first weight parameter matrix, and then dot-adding it with the first bias item to obtain a first-level urban area characteristics-three-dimensional model intrinsic commonality modulation vector; inputting the first-level urban area characteristics-three-dimensional model intrinsic commonality modulation vector into a Sigmoid function for nonlinear activation processing to obtain a first-level urban area characteristics-three-dimensional model intrinsic commonality modulation activation vector; inputting the first-level urban area characteristics-three-dimensional model intrinsic commonality modulation activation vector into a mask unit to obtain the first-level urban area three-dimensional model semantic feature preliminary screening weight vector. Among them, the first-level urban area characteristics-three-dimensional model intrinsic common modulation activation vector is input into the mask unit to obtain the first-level urban area three-dimensional model semantic feature preliminary screening weight vector, including: setting a first mask threshold and a second mask threshold, wherein the second mask threshold is twice the first mask threshold; if the eigenvalue in the first-level urban area characteristics-three-dimensional model intrinsic common modulation activation vector is greater than the second mask threshold, it remains unchanged; if the eigenvalue in the first-level urban area characteristics-three-dimensional model intrinsic common modulation activation vector is greater than the first mask threshold, and is less than or equal to the second mask threshold, it is doubled and reduced; if the eigenvalue in the first-level urban area characteristics-three-dimensional model intrinsic common modulation activation vector is less than or equal to the first mask threshold, it is set to 0, thereby obtaining the first-level urban area three-dimensional model semantic feature preliminary screening weight vector.

[0043] In a specific example of the present application, the step S422 includes: taking each feature value of the initial screening weight vector of the semantic feature of the first-level urban area three-dimensional model as a weight, weighting each feature matrix of the local semantic coding feature map of the urban area three-dimensional model along the channel dimension by position, and then inputting the tanh function for activation processing to obtain the initial screening urban area three-dimensional model local semantic coding feature map. That is, based on the gated mask mechanism, the gated mask processing is performed on the intrinsic semantic commonality feature vector of the urban area characteristic-three-dimensional model to achieve feature importance evaluation and selection, and generate a corresponding weight vector, and the local semantic coding feature map of the urban area three-dimensional model is initially screened, thereby suppressing irrelevant or minor parts therein, and strengthening the feature parts closely related to the urban area characteristics, so as to obtain the initial screening urban area three-dimensional model local semantic coding feature map focusing on the urban area characteristics-three-dimensional model associated features.

[0044] In a specific example of the present application, the step S423 includes: multiplying the first-level urban area characteristics-three-dimensional model intrinsic common modulation vector using the second weight parameter matrix, and then dot-adding it with the second bias item to obtain the second-level urban area characteristics-three-dimensional model intrinsic common modulation vector; inputting the second-level urban area characteristics-three-dimensional model intrinsic common modulation vector into the Sigmoid function for nonlinear activation processing to obtain the second-level urban area characteristics-three-dimensional model intrinsic common modulation activation vector; inputting the second-level urban area characteristics-three-dimensional model intrinsic common modulation activation vector into the mask unit to obtain the second-level urban area three-dimensional model semantic feature screening weight vector.

[0045] In a specific example of the present application, the step S424 includes: taking each feature value of the semantic feature screening weight vector of the secondary urban area three-dimensional model as a weight, weighting each feature matrix of the local semantic coding feature map of the primary screening urban area three-dimensional model along the channel dimension by position, and then inputting the Sigmoid function for activation processing to obtain the multi-modal joint optimization semantic coding feature map of the urban area model. That is, in order to further improve the performance of the model, the multi-level gated screening of the intrinsic semantic commonality feature vector of the urban area characteristic-three-dimensional model is performed again, and the process of information transmission is more finely managed through the multi-level gated architecture, and the local semantic coding feature map of the primary screening urban area three-dimensional model previously formed is subjected to secondary refinement processing. After two targeted feature selections and reconstructions, the multi-modal joint optimization semantic coding feature map of the urban area model containing significant correlation information between the semantic information of the urban area characteristics and the three-dimensional structural geometric information is finally obtained, thereby achieving deep optimization of the three-dimensional structural features of the urban area, and providing a more accurate feature expression for the reconstruction of the three-dimensional model.

[0046] Accordingly, the step S4 includes: processing the urban area characteristic semantic coding feature vector and the local semantic coding feature map of the urban area three-dimensional model by the following cross-modal fusion formula to obtain the multi-modal joint optimization semantic coding feature map of the urban area model, wherein the cross-modal fusion formula is: in, represents the semantic encoding feature vector of the urban area characteristics, represents a local semantic coding feature map of the three-dimensional model of the urban area, represents point convolution, represents the average pooling operation, represents the Sigmoid activation function, represents the semantic distillation feature vector of the three-dimensional model of the urban area, Indicates cascade, , and Represent different weight parameter matrices, , and Represent different bias terms, represents the hyperbolic tangent function, represents the characteristic vector of the urban area-the intrinsic semantic commonality of the three-dimensional model, Represents the first-level urban area characteristics-the intrinsic common modulation activation vector of the three-dimensional model, represents the mask function, represents the weight vector for filtering semantic features of the first-level urban area 3D model, Represents the secondary urban area characteristics - the intrinsic common modulation activation vector of the three-dimensional model, represents the weight vector for filtering semantic features of the 3D model of the secondary urban area, represents the first mask threshold, represents the matrix multiplication operation, represents dot product, It represents the local semantic coding feature map of the preliminary screening urban area 3D model. Represents the multimodal joint optimization semantic encoding feature map of the urban area model.

[0047] In the above-mentioned method for constructing a 3D model of a smart city, the step S5 generates an optimized three-dimensional model of the urban area based on the multimodal joint optimization semantic coding feature map of the urban area model. In a specific example of the present application, the step S5 includes: inputting the multimodal joint optimization semantic coding feature map of the urban area model into the urban model optimization engine based on the diffusion model to obtain the optimized three-dimensional model of the urban area. Specifically, the urban model optimization engine uses diffusion generation technology to capture the complex three-dimensional structural features contained in the multimodal joint optimization semantic coding feature map of the urban area model by gradually adding noise and learning the inverse process of noise diffusion, and gradually removes the noise therein, and gradually approaches the real three-dimensional form of the city through refined feature adjustment to generate a higher quality three-dimensional model of the urban area, thereby providing a more accurate visual reference for urban planning and management.

[0048] In a preferred example of the present application, the multimodal joint optimization semantic coding feature map of the urban area model is input into the urban model optimization engine based on the diffusion model to obtain the optimized three-dimensional model of the urban area, including: expanding the multimodal joint optimization semantic coding feature map of the urban area model into a multimodal joint optimization semantic coding feature vector of the urban area model; determining the multimodal joint optimization semantic coding association response matrix of the urban area model and the multimodal joint optimization semantic coding distance response matrix of the urban area model based on the association value and distance value of the eigenvalue of the multimodal joint optimization semantic coding feature vector of the urban area model; matrix multiplying the multimodal joint optimization semantic coding feature vector of the urban area model with the multimodal joint optimization semantic coding association response matrix of the urban area model to obtain the multimodal joint optimization semantic coding association response vector of the urban area model; and multiplying the multimodal joint optimization semantic coding distance response matrix of the urban area model with the multimodal joint optimization semantic coding feature vector of the urban area model. The transposed vector of the code feature vector is matrix multiplied to obtain the urban area model multimodal joint optimization semantic coding distance response vector; the urban area model multimodal joint optimization semantic coding association response matrix and the urban area model multimodal joint optimization semantic coding distance response matrix are point multiplied, and then the matrix is ​​multiplied with the transposed vector of the urban area model multimodal joint optimization semantic coding feature vector to obtain the urban area model multimodal joint optimization semantic coding logic bias vector; the urban area model multimodal joint optimization semantic coding association response vector, the urban area model multimodal joint optimization semantic coding distance response vector and the urban area model multimodal joint optimization semantic coding logic bias vector are point multiplied to obtain the optimized urban area model multimodal joint optimization semantic coding feature vector; the optimized urban area model multimodal joint optimization semantic coding feature vector is input into the urban model optimization engine based on the diffusion model to obtain the optimized urban area three-dimensional model.

[0049] Accordingly, the optimization process of the multimodal joint optimization semantic encoding feature vector of the urban area model is expressed by the following optimization formula: in, Multimodal joint optimization of semantic encoding association response matrix for urban area model, Multimodal joint optimization of semantic encoding distance response matrix for urban area model, In the multimodal joint optimization semantic encoding association response matrix of the urban area model The eigenvalues ​​of the positions, In the multimodal joint optimization semantic encoding distance response matrix of the urban area model The eigenvalues ​​of the positions, and The first and eigenvalues, The point-add vector between the urban area model multimodal joint optimization semantic coding association response vector and the urban area model multimodal joint optimization semantic coding distance response vector, Optimize the semantic encoding logic bias vector for multimodal urban area model. is the set of real numbers, Optimize the length of semantic encoding feature vectors for multimodal joint optimization of urban area models, represents dot product, represents the matrix multiplication operation, Indicates point addition, Multimodal joint optimization of semantic encoding feature vectors for urban area models, Multimodal joint optimization of semantic encoding feature vectors for optimized urban area models.

[0050] That is, the present application takes into account that when the urban area characteristic semantic coding feature vector and the urban area three-dimensional model local semantic coding feature map respectively represent the semantic features of the urban area characteristic description text and the model depth semantic features of the urban area three-dimensional model, when performing cross-modal preferential encoding based on multimodal implicit association, the feature modality difference and feature coding dimension between the urban area characteristic semantic coding feature vector and the urban area three-dimensional model local semantic coding feature map will lead to sparse distribution of cross-modal joint encoding, and then lead to large-scale simple repetition of cross-modal feature joint of coding features, affecting cross-modal semantic logical dependency and reducing the model quality details of the optimized urban area three-dimensional model obtained by inputting the urban model optimization engine based on the diffusion model.

[0051] Therefore, in a preferred example, the self-correlation matrix and self-distance matrix of the multimodal joint optimization semantic coding feature vector of the urban area model after the multimodal joint optimization semantic coding feature map of the urban area model are used as the statistical reference-free distribution response framework of the multimodal joint optimization semantic coding feature map of the urban area model, and the simple repetition of feature distribution is avoided based on the retrieval-enhanced reverse response construction of the multimodal joint optimization semantic coding feature vector of the urban area model, and the surface combinatorial mapping of the multimodal joint optimization semantic coding feature map of the urban area model is avoided by ensuring the intrinsic mapping logic of the multimodal joint optimization semantic coding feature vector of the urban area model based on the retrieval-response context, so as to achieve the logical dependency mapping of the multimodal joint optimization semantic coding feature map of the urban area model to the target generation domain while maintaining the intuitive response correlation, thereby improving the model quality of the optimized urban area three-dimensional model obtained by inputting the multimodal joint optimization semantic coding feature map of the urban area model into the urban model optimization engine based on the diffusion model.

[0052] In summary, the construction method of the smart city 3D model based on the embodiment of the present application is explained, which uses deep learning-based data processing technology to extract features from the three-dimensional point cloud data of the urban area to be modeled, captures the three-dimensional structural semantic features of the urban area, and combines the natural language description of the characteristics of the urban area input by the user to perform feature modulation optimization processing on the three-dimensional structure of the urban area, further optimizes the detailed expression of the three-dimensional model, and thus realizes the intelligent construction of the three-dimensional model of the urban area. In this way, the geometric information of the urban area and the language description information of the urban characteristics can be effectively integrated, so that the generated three-dimensional model of the urban area can more accurately reflect the internal structure and functional layout of the city, and improve the application value of the model in urban planning and management.

[0053] Furthermore, a system for constructing a smart city 3D model is also provided.

[0054] Figure 6 FIG. 1 is a block diagram of a system for constructing a 3D model of a smart city according to an embodiment of the present application. Figure 6As shown, according to the construction system 100 of the smart city 3D model of the embodiment of the present application, it includes: a data acquisition module 110, which is used to obtain a three-dimensional point cloud of the urban area to be modeled collected by a laser detector, and a natural language description of the characteristics of the urban area to be modeled input by a user; an urban area three-dimensional feature extraction module 120, which is used to extract the characteristics of the urban area three-dimensional model based on the three-dimensional point cloud to obtain a local semantic coding feature map of the urban area three-dimensional model; an urban area characteristic semantic coding module 130, which is used to semantically encode the characteristic natural language description of the urban area to be modeled to obtain a semantic coding feature vector of the urban area characteristics; a semantic joint optimization module 140, which is used to semantically joint optimize the local semantic coding feature map of the urban area three-dimensional model based on the urban area characteristic semantic coding feature vector to obtain a multimodal joint optimization semantic coding feature map of the urban area model; an optimization model generation module 150, which is used to generate an optimized urban area three-dimensional model based on the multimodal joint optimization semantic coding feature map of the urban area model.

[0055] Here, those skilled in the art can understand that the specific operations of each module in the above-mentioned smart city 3D model construction system have been referred to above. Figures 1 to 5 The description of the method for constructing the smart city 3D model has been introduced in detail, and therefore, its repeated description will be omitted.

[0056] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0057] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0058] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0059] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0060] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for constructing a 3D model of a smart city. It is characterized in that include: Acquire a three-dimensional point cloud of the urban area to be modeled collected by a laser detector, and a natural language description of the characteristics of the urban area to be modeled input by a user; Based on the three-dimensional point cloud, extracting the features of the three-dimensional model of the urban area to obtain a local semantic coding feature map of the three-dimensional model of the urban area; Performing semantic coding on the characteristic natural language description of the urban area to be modeled to obtain a semantic coding feature vector of the urban area characteristics; Based on the semantic coding feature vector of the urban area characteristics, semantically jointly optimizing the local semantic coding feature map of the urban area three-dimensional model to obtain a multimodal jointly optimized semantic coding feature map of the urban area model; Generate an optimized three-dimensional model of the urban area based on the multi-modal joint optimization semantic coding feature map of the urban area model; Among them, based on the semantic coding feature vector of the urban area characteristics, the local semantic coding feature map of the urban area three-dimensional model is semantically jointly optimized, including: extracting the intrinsic semantic common features between the semantic coding feature vector of the urban area characteristics and the local semantic coding feature map of the urban area three-dimensional model to obtain the intrinsic semantic common feature vector of the urban area characteristics-three-dimensional model; based on the intrinsic semantic common feature vector of the urban area characteristics-three-dimensional model, dual feature modulation is performed on the local semantic coding feature map of the urban area three-dimensional model to obtain the multimodal joint optimization semantic coding feature map of the urban area model.

2. The method for constructing a 3D model of a smart city according to claim 1, It is characterized in that Based on the three-dimensional point cloud, extracting the features of the three-dimensional model of the urban area to obtain a local semantic coding feature map of the three-dimensional model of the urban area, including: Based on the three-dimensional point cloud, construct a three-dimensional model of the urban area; The urban area three-dimensional model is input into an urban model semantic feature extractor based on a three-dimensional convolutional neural network model to obtain a local semantic coding feature map of the urban area three-dimensional model.

3. The method for constructing a 3D model of a smart city according to claim 2, It is characterized in that Semantically encoding the characteristic natural language description of the urban area to be modeled to obtain a semantically encoded feature vector of the urban area characteristics, including: The characteristic natural language description of the urban area to be modeled is input into a semantic encoder based on the Bert model for semantic encoding to obtain a characteristic semantic encoding feature vector of the urban area.

4. The method for constructing a 3D model of a smart city according to claim 3, It is characterized in that Extracting the intrinsic semantic commonality feature between the semantic coding feature vector of the urban area characteristic and the local semantic coding feature map of the urban area three-dimensional model to obtain the intrinsic semantic commonality feature vector of the urban area characteristic-three-dimensional model includes: Performing feature distillation on the local semantic encoding feature map of the urban area three-dimensional model to obtain a semantic distillation feature vector of the urban area three-dimensional model; The semantic encoding feature vector of the urban area characteristics and the semantic distillation feature vector of the urban area three-dimensional model are input into a multimodal implicit association feature capture network to obtain the intrinsic semantic commonality feature vector of the urban area characteristics-three-dimensional model.

5. The method for constructing a 3D model of a smart city according to claim 4, It is characterized in that Performing feature distillation on the local semantic coding feature map of the urban area three-dimensional model to obtain a semantic distillation feature vector of the urban area three-dimensional model, including: Performing point convolution coding on the local semantic coding feature map of the urban area three-dimensional model to obtain a channel-modulated local semantic coding feature map of the urban area three-dimensional model; The local semantic coding feature map of the channel-modulated urban area three-dimensional model is subjected to global mean pooling processing along the channel dimension and then input into a Sigmoid function for activation processing to obtain a semantic distillation feature vector of the urban area three-dimensional model.

6. The method for constructing a 3D model of a smart city according to claim 5, It is characterized in that Based on the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector, dual feature modulation is performed on the local semantic coding feature map of the urban area three-dimensional model to obtain the multi-modal joint optimization semantic coding feature map of the urban area model, including: Inputting the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector into a feature preliminary screening module based on a gate control unit to obtain a primary urban area three-dimensional model semantic feature preliminary screening weight vector; Based on the semantic feature preliminary screening weight vector of the first-level urban area three-dimensional model, preliminary feature screening is performed on the local semantic coding feature map of the urban area three-dimensional model to obtain a preliminary screening urban area three-dimensional model local semantic coding feature map; Inputting the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector into a feature secondary screening module based on a multi-level gating unit to obtain a secondary urban area three-dimensional model semantic feature screening weight vector; Based on the semantic feature screening weight vector of the secondary urban area three-dimensional model, the local semantic coding feature map of the initially screened urban area three-dimensional model is subjected to secondary feature screening to obtain the multimodal joint optimization semantic coding feature map of the urban area model.

7. The method for constructing a 3D model of a smart city according to claim 6, It is characterized in that The urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector is input into a feature preliminary screening module based on a gate control unit to obtain a primary urban area three-dimensional model semantic feature preliminary screening weight vector, including: After multiplying the urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector by the first weight parameter matrix, the first bias term is added to obtain a first-level urban area characteristic-three-dimensional model intrinsic commonality modulation vector; Inputting the first-level urban area characteristic-three-dimensional model intrinsic common modulation vector into a Sigmoid function for nonlinear activation processing to obtain a first-level urban area characteristic-three-dimensional model intrinsic common modulation activation vector; The first-level urban area characteristic-three-dimensional model intrinsic common modulation activation vector is input into the mask unit to obtain the initial screening weight vector of the semantic features of the first-level urban area three-dimensional model.

8. The method for constructing a smart city 3D model according to claim 7, It is characterized in that The urban area characteristic-three-dimensional model intrinsic semantic commonality feature vector is input into a feature secondary screening module based on a multi-level gating unit to obtain a secondary urban area three-dimensional model semantic feature screening weight vector, including: After multiplying the first-level urban area characteristic-three-dimensional model intrinsic common modulation vector by the second weight parameter matrix, the second-level urban area characteristic-three-dimensional model intrinsic common modulation vector is obtained by multiplying the second weight parameter matrix by the second bias term; Inputting the secondary urban area characteristics-three-dimensional model intrinsic common modulation vector into a Sigmoid function for nonlinear activation processing to obtain a secondary urban area characteristics-three-dimensional model intrinsic common modulation activation vector; The secondary urban area characteristic-three-dimensional model intrinsic common modulation activation vector is input into the mask unit to obtain the secondary urban area three-dimensional model semantic feature screening weight vector.

9. The method for constructing a smart city 3D model according to claim 8, It is characterized in that Based on the multi-modal joint optimization semantic coding feature map of the urban area model, an optimized urban area three-dimensional model is generated, including: The multimodal joint optimization semantic coding feature map of the urban area model is input into the urban model optimization engine based on the diffusion model to obtain the optimized urban area three-dimensional model.

10. A system for building a 3D model of a smart city. It is characterized in that include: A data acquisition module, used to acquire a three-dimensional point cloud of the urban area to be modeled acquired by a laser detector, and a natural language description of the characteristics of the urban area to be modeled input by a user; An urban area three-dimensional feature extraction module, used to extract the features of the urban area three-dimensional model based on the three-dimensional point cloud to obtain a local semantic coding feature map of the urban area three-dimensional model; An urban area characteristic semantic coding module, used for semantically coding the characteristic natural language description of the urban area to be modeled to obtain a characteristic semantic coding feature vector of the urban area characteristic; A semantic joint optimization module, used for performing semantic joint optimization on the local semantic coding feature map of the urban area three-dimensional model based on the urban area characteristic semantic coding feature vector to obtain a multimodal joint optimized semantic coding feature map of the urban area model; The optimization model generation module is used to generate an optimized urban area three-dimensional model based on the multi-modal joint optimization semantic coding feature map of the urban area model.

Images