Three-dimensional real scene construction method, device, equipment and computer-readable storage medium

Abstract

Embodiments of the present invention relate to the field of image processing and disclose a method for constructing a three-dimensional real scene. The method comprises: obtaining panoramic map data and satellite map data; the panoramic map data includes multiple panoramic images from different angles; the satellite map includes at least one overhead plan view of a building; determining at least one building facade overhead view based on the multiple panoramic images from different angles; matching the at least one building facade overhead view with the at least one building overhead plan view to obtain a matched target building facade overhead view; and vertically elevating the target building facade overhead view to obtain a 3D model of the building. Through the above-described method, embodiments of the present invention achieve the effect of quickly and accurately constructing a three-dimensional real scene.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and specifically to a three-dimensional real scene construction method, a three-dimensional real scene construction device, a three-dimensional real scene construction equipment, and a computer-readable storage medium. Background Art

[0002] Existing 3D map construction solutions typically rely on manual scanning, mapping, and data collection for real-world map elements such as roads and buildings. This data is then used to perform manual 3D modeling. 3D map construction requires significant manpower, material resources, and time for manual mapping and modeling. Furthermore, existing 3D map facades are often blank or require manual drawing, which consumes significant manpower, material resources, and time. Consequently, existing 3D real-world scene construction is inefficient and inaccurate. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention provide a three-dimensional real scene construction method, a three-dimensional real scene construction device, a three-dimensional real scene construction equipment and a computer-readable storage medium, which are used to solve the problems of low efficiency and insufficient accuracy of 3D real scene construction in the prior art.

[0004] According to one aspect of an embodiment of the present invention, a method for constructing a three-dimensional real scene is provided, the method comprising:

[0005] Obtaining panoramic map data and satellite map data; the panoramic map data includes multiple panoramic images from different angles; the satellite map includes at least one overhead plan view of a building;

[0006] Determining at least one top view of a building facade according to the plurality of panoramic images at different angles;

[0007] Matching the at least one building facade top view and the at least one building top plan view to obtain a matched target building facade top view;

[0008] The top view of the exterior facade of the target building is vertically elevated to obtain a 3D model of the building.

[0009] In an optional manner, after vertically raising the top view of the target building's facade to obtain a 3D building model, the method further includes:

[0010] Rotating the 3D building model to determine a target panoramic image of at least one shooting perspective corresponding to the 3D building model;

[0011] The building 3D model is calibrated and corrected according to the target panoramic image to obtain a calibrated and corrected building 3D model.

[0012] In an optional manner, the method further includes:

[0013] Extracting features from a target panoramic image of at least one shooting perspective corresponding to the 3D building model to obtain corresponding building elements;

[0014] The building elements are added to corresponding positions of the building 3D model.

[0015] In an optional manner, determining at least one top view of a building facade according to the plurality of panoramic images at different angles includes:

[0016] Extracting vertical contour lines and horizontal contour lines of building facades in the plurality of panoramic images at different angles;

[0017] At least one top view of the building facade is obtained according to the vertical contour lines and horizontal contour lines of the building facade in the plurality of panoramic images at different angles.

[0018] In an optional manner, before obtaining the panoramic map data and the satellite map data, the method further includes:

[0019] Extracting satellite map data using a satellite map; the satellite map data includes location information and shape information of a physical object; the shape information includes a top-down plan view of at least one building;

[0020] Panoramic image data is extracted using a panoramic map; the panoramic image data includes multiple panoramic images from different angles and location information of real objects in the multiple panoramic images from different angles; the real objects include at least one of roads, green belts, buildings and transportation facilities.

[0021] In an optional manner, the method further includes:

[0022] Performing one-to-one mapping on the satellite map data and the panoramic image data to obtain a mapping relationship between the satellite map and the real objects in the panoramic map;

[0023] fitting and obtaining 3D models of roads, green belts, and traffic facilities and corresponding positional relationships based on the panoramic image data and the mapping relationship between the panoramic image data;

[0024] A three-dimensional real-scene model of the urban road is obtained based on the 3D model of the building, the 3D model of the road, the green belt and the traffic facilities and the corresponding positional relationships.

[0025] In an optional manner, the method further includes:

[0026] Using traffic monitoring cameras to extract images of real-life traffic facilities in real time;

[0027] The 3D model of the building, and the 3D models of the roads, green belts and traffic facilities are updated according to the real-scene traffic facility image and the panoramic image data.

[0028] According to another aspect of an embodiment of the present invention, a three-dimensional real scene construction device is provided, comprising:

[0029] An acquisition module, configured to acquire panoramic map data and satellite map data; the panoramic map data includes a plurality of panoramic images from different angles; and the satellite map includes at least one overhead plan view of a building;

[0030] a determining module, configured to determine at least one top view of a building facade based on the plurality of panoramic images at different angles;

[0031] a matching module, configured to match the at least one building facade top view and the at least one building top plan view to obtain a matched target building facade top view;

[0032] The construction module is used to vertically raise the top view of the exterior facade of the target building to obtain a 3D model of the building.

[0033] According to another aspect of an embodiment of the present invention, there is provided a three-dimensional real scene construction device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0034] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the three-dimensional real scene construction method.

[0035] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores at least one executable instruction. When the executable instruction is executed on a three-dimensional real scene construction device, the three-dimensional real scene construction device executes the operations of the three-dimensional real scene construction method.

[0036] The embodiments of the present invention acquire panoramic map data and satellite map data, wherein the panoramic map data includes multiple panoramic images from different angles, and the satellite map includes at least one overhead plan view of a building. The system then determines at least one building facade overhead view based on the multiple panoramic images from different angles. The system then matches the at least one building facade overhead view with the at least one building overhead plan view to obtain a matched target building facade overhead view. Finally, the target building facade overhead view is vertically scaled to obtain a 3D model of the building. This allows for the rapid and accurate construction of a 3D real-world model. By periodically and automatically constructing a 3D real-world model of urban roads, it can be efficiently and conveniently used in games, maps, virtual reality apps, and other applications, significantly reducing manual modeling costs.

[0037] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to more clearly understand the technical means of the embodiments of the present invention, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the drawings. In the drawings:

[0039] Figure 1 A schematic diagram showing a process of a method for constructing a three-dimensional real scene provided by an embodiment of the present invention is shown;

[0040] Figure 2 A schematic diagram of a physical satellite image obtained by a satellite map in the three-dimensional real scene construction method provided by an embodiment of the present invention is shown;

[0041] Figure 3 A schematic diagram showing physical object information in satellite map data in a method for constructing a three-dimensional real scene provided by an embodiment of the present invention is shown;

[0042] Figure 4 A schematic diagram showing information of panoramic image data in a method for constructing a three-dimensional real scene provided by an embodiment of the present invention;

[0043] Figure 5 A schematic diagram showing building and shop information obtained by using traffic monitoring cameras in a 3D real scene construction method provided by an embodiment of the present invention is shown;

[0044] Figure 6 A schematic diagram of extracting a top view of a building facade in a three-dimensional real scene construction method provided by an embodiment of the present invention is shown;

[0045] Figure 7A schematic diagram illustrating image segmentation and completion in a method for constructing a three-dimensional real scene provided by an embodiment of the present invention is shown;

[0046] Figure 8 A schematic diagram showing the matching of a top view of a building facade and a top plan view of a building in the method for constructing a three-dimensional real scene provided by an embodiment of the present invention is shown;

[0047] Figure 9 A schematic diagram showing vertically lifting a top view of a building facade in a three-dimensional real scene construction method provided by an embodiment of the present invention is shown;

[0048] Figure 10 A schematic structural diagram of a three-dimensional real scene construction device provided by an embodiment of the present invention is shown;

[0049] Figure 11 The figure shows a schematic structural diagram of a three-dimensional real scene construction device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0051] Figure 1 The flowchart of the 3D real scene construction method provided by the embodiment of the present invention is shown. The method is executed by a 3D real scene construction device or an application. The 3D real scene construction device can be a computer device, a terminal device, a wearable device, a metaverse device, etc., and the embodiment of the present invention does not make specific restrictions. Figure 1 As shown, the method includes the following steps:

[0052] Step 110: Obtain panoramic map data and satellite map data.

[0053] The panoramic map data includes multiple panoramic images from different angles; and the satellite map includes at least one overhead plan view of a building.

[0054] In an embodiment of the present invention, satellite map data is extracted using satellite maps; the satellite map data includes location information and shape information of physical objects; the shape information includes at least one overhead plan view of a building. Specifically, image data of physical objects such as roads, green belts, and buildings are extracted using satellite maps, and satellite map data is obtained through image analysis. The satellite map data includes information such as the type, location, and shape of each physical object, uniquely identifies it, and records its location and shape. Figure 2 As shown in the figure, the satellite images of roads, green belts, buildings and other objects are obtained from satellite maps. By calculating and analyzing the satellite images of the objects, the object type, location information and shape information of each object are obtained, such as Figure 3 The object type, location information, and shape information of each object include at least one top-view plan of a building.

[0055] The satellite map data information is shown in Table 1:

[0056]

[0057]

[0058] Table 1

[0059] This embodiment of the present invention also utilizes a panoramic map to extract panoramic image data. The panoramic image data includes multiple panoramic images taken from different angles, and the location information of objects within these panoramic images. The objects include at least one of roads, green belts, buildings, and transportation facilities. Specifically, the panoramic map is used to identify objects such as roads, buildings, and green belts, generate unique identifiers for these objects, and extract basic feature information for each object, including color, shape, and location information. In this embodiment of the present invention, a preset stage segmentation method is used to extract building information. Assuming a road section is 1000 meters long from its starting point to its end point, a stage segmentation point is used every 100 meters. The maximum elevation angle on either side of the road is taken, resulting in 10 viewpoint images from each side of the road. These 10 images are analyzed using continuous image change, recognition, and tracking techniques to determine the unique identifier of the building. The location of the building's front viewpoint sampling point is then used as the building's relative position. The panoramic map is then used to analyze and calculate road width, building height, and other information. The system estimates road width, building height, and building length based on the road and building information collected from the panoramic map. Building height estimation: Based on the image recognition of an 18-story building, the estimated height is 18*3 = 54 meters. Building width estimation: The vertical viewing angle of the building is estimated from the starting position to the starting position. This method obtains building location and shape information from multiple panoramic images at different angles.

[0060] Furthermore, by performing image recognition on multiple panoramic images taken from different angles, information such as roads, medians, and green belts in panoramic map photos can be identified and their features extracted. For example, if the main road has six lanes in both directions, the road width can be estimated based on the number of lanes: 6*3 meters = 18 meters. If the median is a fence or a green belt, its width can be estimated, along with the width of the auxiliary lanes, whether the green belt consists of short shrubs or tall trees, the tree type, and tree height. This embodiment of the present invention also extracts information about traffic facilities, such as parking cameras, traffic light monitoring equipment, and street lights, using similar methods for building extraction. Their numbers and locations are recorded.

[0061] In the embodiment of the present invention, the panoramic map is used to extract the information of the panoramic image data as shown in Table 2 and Figure 4 As shown:

[0062]

[0063] Table 2

[0064] Among them, the embodiment of the present invention also uses traffic monitoring cameras to extract real-life traffic facility images in real time; based on the real-life traffic facility images and the panoramic image data, the 3D model of the building, as well as the 3D models of the roads, green belts and traffic facilities are updated. Due to the road monitoring camera, the shooting angle is not as large as the panoramic map, and it requires an interval of 100 meters to several hundred meters, but compared to the panoramic map, the road monitoring camera is updated in real time and can promptly detect changes in building and shop information. Among them, the extracted real-life traffic facility images can be analyzed by a similar image analysis and recognition method to the panoramic map to obtain building information and shop information. Among them, the information included in the building information and shop information is shown in Table 3 below:

[0065]

[0066] Table 3

[0067] According to the building information and shop information, the building information and shop information obtained by the traffic monitoring camera corresponding to the panoramic picture can be obtained, such as Figure 5 shown.

[0068] Step 120: Determine at least one top view of a building facade based on the multiple panoramic images at different angles.

[0069] Wherein, vertical contour lines and horizontal contour lines of the building facades in the plurality of panoramic images at different angles are extracted; and at least one top view of the building facade is obtained based on the vertical contour lines and horizontal contour lines of the building facades in the plurality of panoramic images at different angles. Specifically, the embodiment of the present invention automatically extracts the vertical contour lines and horizontal contour lines of the building facades in the plurality of panoramic images at different angles through a building contour model. Wherein, the building contour model is obtained by using a supervised machine learning algorithm and training the initial model of the building contour model through a large number of real-life pictures of urban buildings. Figure 6As shown, after obtaining the vertical and horizontal contour lines of the building facade, the data of the top building structure that is partially blocked due to different shooting angles is supplemented so that the vertex of the bending angle of each horizontal contour line can obtain the vertical contour line, thereby obtaining the angles and vertical lines of the horizontal lines, and obtaining a top view of the building facade. Among them, since the panoramic view is collected in a ground car, the probability of the building and billboard information being blocked by the green belt is relatively high. Therefore, when extracting the contour through the building contour model, the image is compensated in advance to obtain an image containing the complete building. Specifically, the panoramic images at different angles are obtained by moving to different road locations for shooting. Therefore, as Figure 7 As shown, compensation and restoration can be performed by image segmentation and completion methods, and each image is identified for obstructions, such as trees, and rectangular segmentation is performed. Divide into two blocks: a red frame (obstruction) and a green frame (building). Traverse other views to check whether there is any unobstructed part of the area containing the red frame. If so, merge the partial buildings in the two images into a complete building. If not, crop the part of the green frame that is equal to the length of the red frame, replace the red frame, and obtain a complete image of the building. In the embodiment of the present invention, OCR technology is also used to identify the text information of the billboards of shops on the facade of the building, and the image segmentation and multi-view completion methods are used to complete the billboard information, and the location information of the billboards is recorded for subsequent use. As shown in Table 4, the location information of the billboards of shops on the facade of the building is shown.

[0070]

[0071] Table 4

[0072] Step 130: Match the at least one building facade top view and the at least one building top plan view to obtain a matched target building facade top view.

[0073] Among them, such as Figure 8 As shown in the figure, the building top view on the satellite map is compared with the building top view corresponding to multiple panoramic images at different angles, the data with large errors are removed, and the other multiple groups of normal data are fitted to obtain the target building facade top view with the highest overlap.

[0074] Step 140: vertically elevate the top view of the target building's facade to obtain a 3D model of the building.

[0075] Specifically, if Figure 9 As shown, the top view of the building facade is vertically raised to obtain a 3D model of the building.

[0076] In an embodiment of the invention, after vertically raising the top view of the facade of the target building to obtain a 3D model of the building, the method further includes: rotating the 3D model of the building to determine a target panoramic image of at least one shooting angle corresponding to the 3D model of the building; calibrating and correcting the 3D model of the building according to the target panoramic image to obtain a calibrated and corrected 3D model of the building. Panoramic images of different angles of a certain building are selected, and the 3D model of the building is rotated in a three-dimensional space. The rotation angles include vertical front side, 45 degrees to the left, 30 degrees to the left, left side, 30 degrees to the right, 45 degrees to the right, right side, etc. Find the shooting angle of the panoramic image corresponding to the 3D model of the building, compare the picture taken at this angle of the panoramic image with the picture of the rotated 3D model of the building, and compare the degree of overlap. If the error is less than a certain range, it is considered that the 3D model of the building is correct at this perspective. The other panoramic images of the building and the rotated 3D model are then compared in turn. If the similarity between all panoramic images and the model reaches a consistent standard, the model is considered to match. Otherwise, other images are used to recalculate and adjust the parameters of the building's 3D model until the panoramic images from all perspectives are consistent with the building's 3D model, thus obtaining a calibrated and corrected building 3D model.

[0077] Among them, after obtaining the building 3D model or the calibrated and corrected building 3D model, the embodiment of the present invention also performs feature extraction on the target panoramic image of at least one shooting angle corresponding to the building 3D model to obtain corresponding building elements; and adds the building elements to the corresponding positions of the building 3D model. The building elements include elements such as windows and balconies in the 2 buildings, as well as elements such as the color and material of the building facade, and also elements such as the business billboard information on the building facade. Specifically, elements such as windows and balconies in the panoramic picture are identified and feature extracted, and these elements are added to the corresponding positions of the 3D model; the facade material (color, pattern) in the panoramic picture is extracted and pasted into the 3D model, etc.

[0078] The embodiment of the present invention further performs a one-to-one mapping between the satellite map data and the panoramic image data to obtain a mapping relationship between the satellite map and the objects in the panoramic map. Based on the mapping relationship between the panoramic image data and the panoramic image data, 3D models of roads, green belts, and transportation facilities and their corresponding positional relationships are fitted. A three-dimensional real-scene model of urban roads is obtained based on the 3D building models, as well as the 3D models of roads, green belts, and transportation facilities and their corresponding positional relationships. 3D models of roads, green belts, and transportation facilities are constructed by selecting appropriate models from a predefined model library. Specifically, the road model is adapted using a predefined road model based on the shape and width of the road in the panoramic map and calibrated based on the panoramic map. The green belt model is populated with appropriate predefined green strips for the median road and low roadside green belts using features identified in the panoramic map. This approach maintains similarity while reducing redundant 3D panorama data, thereby optimizing 3D model loading speed and browsing experience. Traffic facilities: Using panoramic maps to identify traffic facilities, predefined traffic facility models are used to populate the 3D model, resulting in a final 3D realistic model of the city road. This embodiment of the present invention uses predefined models and image recognition to select similar models for construction, reducing the amount of duplicate data, speeding up 3D model loading, and enhancing the user experience.

[0079] The embodiment of the present invention also uses traffic monitoring cameras to extract real-time images of traffic facilities; based on the real-time images of traffic facilities and the panoramic image data, the 3D model of the building, as well as the 3D models of the roads, green belts and traffic facilities are updated. By comparing the building information extracted from satellite maps at different times, changes in buildings can be discovered in a timely manner. The real-time images of traffic facilities can be used to uniquely number traffic facilities and record location information, reducing the cost of installing positioning equipment for traffic equipment. The embodiment of the present invention also detects changes by comparing real-time images of real-time traffic facilities with store billboard information in panoramic maps, and can promptly notify map manufacturers or local life service providers to reduce their operating costs.

[0080] The embodiments of the present invention acquire panoramic map data and satellite map data, wherein the panoramic map data includes multiple panoramic images from different angles, and the satellite map includes at least one overhead plan view of a building. The system then determines at least one building facade overhead view based on the multiple panoramic images from different angles. The system then matches the at least one building facade overhead view with the at least one building overhead plan view to obtain a matched target building facade overhead view. Finally, the target building facade overhead view is vertically scaled to obtain a 3D model of the building. This allows for the rapid and accurate construction of a 3D real-world model. By periodically and automatically constructing a 3D real-world model of urban roads, it can be efficiently and conveniently used in games, maps, virtual reality apps, and other applications, significantly reducing manual modeling costs.

[0081] Figure 10 FIG. 1 shows a schematic diagram of the structure of a three-dimensional real scene construction device provided by an embodiment of the present invention. Figure 10 As shown, the device 300 includes:

[0082] The acquisition module 310 is used to acquire panoramic map data and satellite map data; the panoramic map data includes multiple panoramic images from different angles; the satellite map includes at least one overhead plan view of a building;

[0083] a determination module 320, configured to determine at least one top view of a building facade based on the plurality of panoramic images taken at different angles;

[0084] A matching module 330 is configured to match the at least one building facade top view and the at least one building top plan view to obtain a matched target building facade top view;

[0085] The construction module 340 is used to vertically raise the top view of the exterior facade of the target building to obtain a 3D model of the building.

[0086] In an optional manner, after vertically raising the top view of the target building's facade to obtain a 3D building model, the method further includes:

[0087] Rotating the 3D building model to determine a target panoramic image of at least one shooting perspective corresponding to the 3D building model;

[0088] The building 3D model is calibrated and corrected according to the target panoramic image to obtain a calibrated and corrected building 3D model.

[0089] In an optional manner, the method further includes: performing feature extraction on a target panoramic image of at least one shooting perspective corresponding to the building 3D model to obtain corresponding building elements; and adding the building elements to corresponding positions of the building 3D model.

[0090] In an optional manner, determining at least one top view of a building facade according to the plurality of panoramic images at different angles includes:

[0091] Extracting vertical contour lines and horizontal contour lines of building facades in the plurality of panoramic images at different angles;

[0092] At least one top view of the building facade is obtained according to the vertical contour lines and horizontal contour lines of the building facade in the plurality of panoramic images at different angles.

[0093] In an optional manner, before obtaining the panoramic map data and the satellite map data, the method further includes:

[0094] Extracting satellite map data using a satellite map; the satellite map data includes location information and shape information of a physical object; the shape information includes a top-down plan view of at least one building;

[0095] Panoramic image data is extracted using a panoramic map; the panoramic image data includes multiple panoramic images from different angles and location information of real objects in the multiple panoramic images from different angles; the real objects include at least one of roads, green belts, buildings and transportation facilities.

[0096] In an optional manner, it also includes: mapping the satellite map data and the panoramic image data one by one to obtain a mapping relationship between the satellite map and the real objects in the panoramic map; fitting 3D models of roads, green belts and traffic facilities and corresponding positional relationships based on the panoramic image data and the mapping relationship between the panoramic image data; and obtaining a three-dimensional real-scene model of the urban road based on the 3D model of the building, and the 3D models of the roads, green belts and traffic facilities and their corresponding positional relationships.

[0097] In an optional method, it also includes: using traffic monitoring cameras to extract real-time images of traffic facilities; updating the 3D model of the building, and the 3D models of the roads, green belts and traffic facilities based on the real-life traffic facility images and the panoramic image data.

[0098] The embodiments of the present invention acquire panoramic map data and satellite map data, wherein the panoramic map data includes multiple panoramic images from different angles, and the satellite map includes at least one overhead plan view of a building. The system then determines at least one building facade overhead view based on the multiple panoramic images from different angles. The system then matches the at least one building facade overhead view with the at least one building overhead plan view to obtain a matched target building facade overhead view. Finally, the target building facade overhead view is vertically scaled to obtain a 3D model of the building. This allows for the rapid and accurate construction of a 3D real-world model. By periodically and automatically constructing a 3D real-world model of urban roads, it can be efficiently and conveniently used in games, maps, virtual reality apps, and other applications, significantly reducing manual modeling costs.

[0099] Figure 11 The diagram shows the structure of a 3D real scene construction device provided by an embodiment of the present invention. The specific embodiment of the present invention does not limit the specific implementation of the 3D real scene construction device.

[0100] like Figure 11 As shown, the 3D real scene construction device may include: a processor 402 , a communications interface 404 , a memory 406 , and a communication bus 408 .

[0101] Processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other devices, such as clients or other server network elements. Processor 402 is used to execute program 410, which can specifically perform the steps described in the above-mentioned embodiment of the method for constructing a 3D real scene.

[0102] Specifically, the program 410 may include program code including computer-executable instructions.

[0103] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in the 3D reality construction device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0104] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0105] The program 410 can be specifically called by the processor 402 to enable the 3D real scene construction device to perform the following operations:

[0106] Obtaining panoramic map data and satellite map data; the panoramic map data includes multiple panoramic images from different angles; the satellite map includes at least one overhead plan view of a building;

[0107] Determining at least one top view of a building facade according to the plurality of panoramic images at different angles;

[0108] Matching the at least one building facade top view and the at least one building top plan view to obtain a matched target building facade top view;

[0109] The top view of the exterior facade of the target building is vertically elevated to obtain a 3D model of the building.

[0110] In an optional manner, after vertically raising the top view of the target building's facade to obtain a 3D building model, the method further includes:

[0111] Rotating the 3D building model to determine a target panoramic image of at least one shooting perspective corresponding to the 3D building model;

[0112] The building 3D model is calibrated and corrected according to the target panoramic image to obtain a calibrated and corrected building 3D model.

[0113] In an optional manner, the method further includes:

[0114] Extracting features from a target panoramic image of at least one shooting perspective corresponding to the 3D building model to obtain corresponding building elements;

[0115] The building elements are added to corresponding positions of the building 3D model.

[0116] In an optional manner, determining at least one top view of a building facade according to the plurality of panoramic images at different angles includes:

[0117] Extracting vertical contour lines and horizontal contour lines of building facades in the plurality of panoramic images at different angles;

[0118] At least one top view of the building facade is obtained according to the vertical contour lines and horizontal contour lines of the building facade in the plurality of panoramic images at different angles.

[0119] In an optional manner, before obtaining the panoramic map data and the satellite map data, the method further includes:

[0120] Extracting satellite map data using a satellite map; the satellite map data includes location information and shape information of a physical object; the shape information includes a top-down plan view of at least one building;

[0121] Panoramic image data is extracted using a panoramic map; the panoramic image data includes multiple panoramic images from different angles and location information of real objects in the multiple panoramic images from different angles; the real objects include at least one of roads, green belts, buildings and transportation facilities.

[0122] In an optional manner, the method further includes:

[0123] Performing one-to-one mapping on the satellite map data and the panoramic image data to obtain a mapping relationship between the satellite map and the real objects in the panoramic map;

[0124] fitting and obtaining 3D models of roads, green belts and traffic facilities and corresponding positional relationships based on the panoramic image data and the mapping relationship between the panoramic image data;

[0125] A three-dimensional real-scene model of the urban road is obtained based on the 3D model of the building, the 3D model of the road, the green belt and the traffic facilities and the corresponding positional relationships.

[0126] In an optional manner, the method further includes:

[0127] Using traffic monitoring cameras to extract images of real-life traffic facilities in real time;

[0128] The 3D model of the building, and the 3D models of the roads, green belts and traffic facilities are updated according to the real-scene traffic facility image and the panoramic image data.

[0129] The embodiments of the present invention acquire panoramic map data and satellite map data, wherein the panoramic map data includes multiple panoramic images from different angles, and the satellite map includes at least one overhead plan view of a building. The system then determines at least one building facade overhead view based on the multiple panoramic images from different angles. The system then matches the at least one building facade overhead view with the at least one building overhead plan view to obtain a matched target building facade overhead view. Finally, the target building facade overhead view is vertically scaled to obtain a 3D model of the building. This allows for the rapid and accurate construction of a 3D real-world model. By periodically and automatically constructing a 3D real-world model of urban roads, it can be efficiently and conveniently used in games, maps, virtual reality apps, and other applications, significantly reducing manual modeling costs.

[0130] An embodiment of the present invention provides a computer-readable storage medium storing at least one executable instruction. When the executable instruction is executed on a three-dimensional real scene construction device, the three-dimensional real scene construction device executes the three-dimensional real scene construction method in any of the above method embodiments.

[0131] The executable instructions can be specifically used to enable the 3D real scene construction device to perform the following operations:

[0132] Obtaining panoramic map data and satellite map data; the panoramic map data includes multiple panoramic images from different angles; the satellite map includes at least one overhead plan view of a building;

[0133] Determining at least one top view of a building facade according to the plurality of panoramic images at different angles;

[0134] Matching the at least one building facade top view and the at least one building top plan view to obtain a matched target building facade top view;

[0135] The top view of the exterior facade of the target building is vertically elevated to obtain a 3D model of the building.

[0136] In an optional manner, after vertically raising the top view of the target building's facade to obtain a 3D building model, the method further includes:

[0137] Rotating the 3D building model to determine a target panoramic image of at least one shooting perspective corresponding to the 3D building model;

[0138] The building 3D model is calibrated and corrected according to the target panoramic image to obtain a calibrated and corrected building 3D model.

[0139] In an optional manner, the method further includes:

[0140] Extracting features from a target panoramic image of at least one shooting perspective corresponding to the 3D building model to obtain corresponding building elements;

[0141] The building elements are added to corresponding positions of the building 3D model.

[0142] In an optional manner, determining at least one top view of a building facade according to the plurality of panoramic images at different angles includes:

[0143] Extracting vertical contour lines and horizontal contour lines of building facades in the plurality of panoramic images at different angles;

[0144] At least one top view of the building facade is obtained according to the vertical contour lines and horizontal contour lines of the building facade in the plurality of panoramic images at different angles.

[0145] In an optional manner, before obtaining the panoramic map data and the satellite map data, the method further includes:

[0146] Extracting satellite map data using a satellite map; the satellite map data includes location information and shape information of a physical object; the shape information includes a top-down plan view of at least one building;

[0147] Panoramic image data is extracted using a panoramic map; the panoramic image data includes multiple panoramic images from different angles and location information of real objects in the multiple panoramic images from different angles; the real objects include at least one of roads, green belts, buildings and transportation facilities.

[0148] In an optional manner, the method further includes:

[0149] Performing one-to-one mapping on the satellite map data and the panoramic image data to obtain a mapping relationship between the satellite map and the real objects in the panoramic map;

[0150] fitting and obtaining 3D models of roads, green belts, and traffic facilities and corresponding positional relationships based on the panoramic image data and the mapping relationship between the panoramic image data;

[0151] A three-dimensional real-scene model of the urban road is obtained based on the 3D model of the building, the 3D model of the road, the green belt and the traffic facilities and the corresponding positional relationships.

[0152] In an optional manner, the method further includes:

[0153] Using traffic monitoring cameras to extract images of real-life traffic facilities in real time;

[0154] The 3D model of the building, and the 3D models of the roads, green belts and traffic facilities are updated according to the real-scene traffic facility image and the panoramic image data.

[0155] The embodiments of the present invention acquire panoramic map data and satellite map data, wherein the panoramic map data includes multiple panoramic images from different angles, and the satellite map includes at least one overhead plan view of a building. The system then determines at least one building facade overhead view based on the multiple panoramic images from different angles. The system then matches the at least one building facade overhead view with the at least one building overhead plan view to obtain a matched target building facade overhead view. Finally, the target building facade overhead view is vertically scaled to obtain a 3D model of the building. This allows for the rapid and accurate construction of a 3D real-world model. By periodically and automatically constructing a 3D real-world model of urban roads, it can be efficiently and conveniently used in games, maps, virtual reality apps, and other applications, significantly reducing manual modeling costs.

[0156] An embodiment of the present invention provides a three-dimensional real scene construction device, which is used to execute the above-mentioned three-dimensional real scene construction method.

[0157] An embodiment of the present invention provides a computer program, which can be called by a processor to enable a 3D real scene construction device to execute the 3D real scene construction method in any of the above method embodiments.

[0158] An embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are run on a computer, the computer executes the three-dimensional real scene construction method in any of the above method embodiments.

[0159] The algorithm or demonstration provided herein are not inherently relevant to any particular computer, virtual system or other equipment. Various general-purpose systems may also be used together with the teachings based on this. According to the above description, it is apparent that the structure required for constructing this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0160] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0161] Similarly, it should be understood that in order to streamline the present invention and facilitate understanding of one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0162] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed so far can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0163] It should be noted that the above embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by a person skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

Claims

1. A method for constructing a three-dimensional real scene, characterized in that: The method comprises: Obtaining panoramic map data and satellite map data; the panoramic map data includes multiple panoramic images from different angles; the satellite map includes at least one overhead plan view of a building; Determining at least one top view of a building facade based on the plurality of panoramic images at different angles, wherein vertical contour lines and horizontal contour lines of the building facades in the plurality of panoramic images at different angles are extracted, and obtaining the at least one top view of the building facade based on the vertical contour lines and horizontal contour lines of the building facades in the plurality of panoramic images at different angles; Matching the at least one building facade top view and the at least one building top plan view to obtain a matched target building facade top view; Vertically elevating the top view of the target building's facade to obtain a 3D model of the building; Performing one-to-one mapping on the satellite map data and the panoramic image data to obtain a mapping relationship between the satellite map and the real objects in the panoramic map; Fitting the 3D models of roads, green belts and traffic facilities and their corresponding positional relationships based on the panoramic image data and the mapping relationship between the panoramic image data; A three-dimensional real-scene model of the urban road is obtained based on the 3D model of the building, the 3D model of the road, the green belt and the traffic facilities and the corresponding positional relationships.

2. The method according to claim 1, characterized in that After vertically raising the top view of the target building's facade to obtain a 3D model of the building, the method further includes: Rotating the 3D building model to determine a target panoramic image of at least one shooting perspective corresponding to the 3D building model; The building 3D model is calibrated and corrected according to the target panoramic image to obtain a calibrated and corrected building 3D model.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Extracting features from a target panoramic image of at least one shooting perspective corresponding to the 3D building model to obtain corresponding building elements; The building elements are added to corresponding positions of the building 3D model.

4. The method according to claim 1 or 2, characterized in that Before obtaining the panoramic map data and the satellite map data, the method further includes: Extracting satellite map data using a satellite map; the satellite map data includes location information and shape information of a physical object; the shape information includes a top-down plan view of at least one building; Panoramic image data is extracted using a panoramic map; the panoramic image data includes multiple panoramic images from different angles and location information of real objects in the multiple panoramic images from different angles; the real objects include at least one of roads, green belts, buildings and transportation facilities.

5. The method according to claim 1 or 2, characterized in that The method further comprises: Using traffic monitoring cameras to extract images of real-life traffic facilities in real time; The 3D model of the building, and the 3D models of the roads, green belts and traffic facilities are updated according to the real-scene traffic facility image and the panoramic image data.

6. A three-dimensional real scene construction device, characterized in that: The device comprises: An acquisition module, configured to acquire panoramic map data and satellite map data; the panoramic map data includes a plurality of panoramic images from different angles; and the satellite map includes at least one overhead plan view of a building; a determination module, configured to determine at least one top view of a building facade based on the plurality of panoramic images at different angles, wherein vertical contour lines and horizontal contour lines of the building facades in the plurality of panoramic images at different angles are extracted, and the at least one top view of the building facade is obtained based on the vertical contour lines and horizontal contour lines of the building facades in the plurality of panoramic images at different angles; a matching module, configured to match the at least one building facade top view and the at least one building top plan view to obtain a matched target building facade top view; A construction module is used to vertically raise the top view of the exterior facade of the target building to obtain a 3D model of the building; Performing one-to-one mapping on the satellite map data and the panoramic image data to obtain a mapping relationship between the satellite map and the real objects in the panoramic map; Fitting the 3D models of roads, green belts and traffic facilities and their corresponding positional relationships based on the panoramic image data and the mapping relationship between the panoramic image data; A three-dimensional real-scene model of the urban road is obtained based on the 3D model of the building, the 3D model of the road, the green belt and the traffic facilities and the corresponding positional relationships.

7. A three-dimensional real scene construction device, characterized in that: include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the three-dimensional real scene construction method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The storage medium stores at least one executable instruction. When the executable instruction is executed on the 3D real scene construction device, the 3D real scene construction device executes the operation of the 3D real scene construction method according to any one of claims 1 to 5.

Images