Three-dimensional modeling method, three-dimensional modeling device, electronic device and readable storage medium

Abstract

The present application discloses a three-dimensional modeling method, a three-dimensional modeling device, an electronic device, and a readable storage medium, belonging to the field of virtual reality technology. The three-dimensional modeling method is used for three-dimensional modeling of a house, and the three-dimensional modeling method includes: obtaining multiple target panoramic images, each of which corresponds to multiple rooms in the house; annotating first annotation data of each target panoramic image; establishing multiple three-dimensional sub-models based on the first annotation data corresponding to the multiple target panoramic images, each of which corresponds to multiple rooms; and splicing the multiple three-dimensional sub-models to obtain a target three-dimensional model of the house.

Description

Technical Field

[0001] The present application belongs to the field of virtual reality technology, and specifically relates to a three-dimensional modeling method, a three-dimensional modeling device, an electronic device, and a readable storage medium. Background Art

[0002] Virtual reality technology simulates a virtual environment to create an immersive experience. In existing technologies, 3D modeling requires users to manually edit the model based on photos and 2D floor plans, resulting in low efficiency. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a three-dimensional modeling method, a three-dimensional modeling device, an electronic device and a readable storage medium, which reduce the user's operation steps and improve the efficiency of establishing a three-dimensional model of a house.

[0004] In a first aspect, an embodiment of the present application provides a three-dimensional modeling method, which is used for three-dimensional modeling of a house. The three-dimensional modeling method includes: obtaining multiple target panoramic images, and the multiple target panoramic images correspond to multiple rooms in the house; marking first annotation data of each target panoramic image; establishing multiple three-dimensional sub-models based on the first annotation data corresponding to the multiple target panoramas, and the multiple three-dimensional sub-models correspond to multiple rooms; and splicing the multiple three-dimensional sub-models to obtain a target three-dimensional model of the house.

[0005] In a second aspect, an embodiment of the present application provides a three-dimensional modeling device, which is used for three-dimensional modeling of a house. The three-dimensional modeling device includes: an acquisition module for acquiring multiple target panoramic images, and the multiple target panoramic images correspond to multiple rooms in the house; a labeling module for labeling first labeling data of each target panoramic image; a modeling module for establishing multiple three-dimensional sub-models based on the first labeling data corresponding to the multiple target panoramic images, and the multiple three-dimensional sub-models correspond to multiple rooms; a splicing module for splicing the multiple three-dimensional sub-models to obtain a target three-dimensional model of the house.

[0006] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the three-dimensional modeling method of the first aspect are implemented.

[0007] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the three-dimensional modeling method of the first aspect are implemented.

[0008] The embodiment of the present application obtains a target panoramic image corresponding to each room in a house, automatically annotates first annotation data based on image features in the target panoramic image, and creates a three-dimensional sub-model for each individual room based on the first annotation data. By stitching together the three-dimensional sub-models, the three-dimensional modeling of the entire house can be completed. The modeling system automatically annotates the first annotation data and automatically generates corresponding three-dimensional sub-models based on the first annotation data, reducing the number of user operations and improving the efficiency of building a three-dimensional model of a house. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 One of the flow charts of the three-dimensional modeling method provided in the embodiment of the present application is shown;

[0010] Figure 2 The second flowchart of the three-dimensional modeling method provided in the embodiment of the present application is shown;

[0011] Figure 3 The third flowchart of the three-dimensional modeling method provided in the embodiment of the present application is shown;

[0012] Figure 4 FIG4 shows a fourth flow chart of the three-dimensional modeling method provided in an embodiment of the present application;

[0013] Figure 5 FIG5 shows a fifth flow chart of the three-dimensional modeling method provided in an embodiment of the present application;

[0014] Figure 6 FIG6 shows a sixth flow chart of the three-dimensional modeling method provided in an embodiment of the present application;

[0015] Figure 7 FIG7 shows a flow chart of the three-dimensional modeling method provided in an embodiment of the present application;

[0016] Figure 8 FIG8 shows an eighth flow chart of the three-dimensional modeling method provided in an embodiment of the present application;

[0017] Figure 9 The following is a structural block diagram of a three-dimensional modeling device provided in an embodiment of the present application;

[0018] Figure 10 A structural block diagram of an electronic device provided in an embodiment of the present application is shown;

[0019] Figure 11 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0021] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0022] The following is combined with Figures 1 to 11 , the three-dimensional modeling method, three-dimensional modeling device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0023] In an embodiment of the present application, a three-dimensional modeling method is provided. Figure 1 FIG. 1 shows one of the flow charts of the three-dimensional modeling method provided in the embodiment of the present application, such as Figure 1 As shown, the three-dimensional modeling method is used for three-dimensional modeling of a house, and the three-dimensional modeling method includes:

[0024] Step 102: Acquire multiple target panoramic images, where the multiple target panoramic images correspond to multiple rooms in the house;

[0025] A house includes multiple rooms, and each of the multiple target panoramas corresponds to each room in the house. Specifically, multiple panoramas corresponding to each room in the house are obtained.

[0026] For example, a modeler uses a panoramic camera to capture multiple panoramas of each room in a house, and manually selects a target panorama. The target panorama is a panorama with higher image clarity obtained by manual screening, and duplicate panoramas are removed from the multiple panoramas to obtain the target panorama.

[0027] Step 104, annotating first annotation data of each target panoramic image;

[0028] The modeling system includes an image recognition module, which can identify target image features in each target panoramic image, and obtain first annotation data by annotating the target image features.

[0029] Step 106: creating a plurality of 3D sub-models based on the first annotation data corresponding to the plurality of target panoramic images, wherein the plurality of 3D sub-models correspond to the plurality of rooms;

[0030] Step 108 : splicing the multiple 3D sub-models to obtain a target 3D model of the house.

[0031] The three-dimensional modeling method proposed in the embodiment of the present application is applied in a modeling system. The annotator inputs the target panorama into the annotating system, and the annotating system can annotate each target panorama to obtain first annotated data. The first annotated data is annotated by the annotating system based on the image features in the target panorama. Since each panorama corresponds to a separate room in the house, a three-dimensional sub-model of each separate room can be established based on the first annotated data. After the three-dimensional sub-models of all rooms are established based on each target panorama, the modeling system can splice the multiple three-dimensional sub-models to generate the target three-dimensional model.

[0032] Specifically, the first annotated data is the annotated data obtained by the modeling system automatically annotating image features in the target panorama. The image recognition unit in the modeling system can be configured in the cloud, and the modeling system uploads the target panorama to the cloud server so that the cloud server recognizes and annotates the image features in the target panorama.

[0033] The first annotation data includes contour features of the room, including but not limited to the annotation data of the wall, the annotation data of the floor, and the annotation data of the ceiling. The modeling system can build a basic model of each room based on the contour features.

[0034] The embodiment of the present application obtains a target panoramic image corresponding to each room in a house, automatically annotates first annotation data based on image features in the target panoramic image, and creates a three-dimensional sub-model for each individual room based on the first annotation data. By stitching together the three-dimensional sub-models, the three-dimensional modeling of the entire house can be completed. The modeling system automatically annotates the first annotation data and automatically generates corresponding three-dimensional sub-models based on the first annotation data, reducing the number of user operations and improving the efficiency of building a three-dimensional model of a house.

[0035] In some embodiments of the present application, Figure 2 The second flow chart of the three-dimensional modeling method provided in the embodiment of the present application is shown as follows: Figure 2 As shown, the first annotation data for annotating each target panoramic image includes:

[0036] Step 202: identifying target image features in the target panoramic image, where the target image features include wall lines between adjacent walls;

[0037] Step 204 : annotate first annotation data in the target panoramic image according to the target image features.

[0038] In an embodiment of the present application, the modeling system is capable of identifying target image features in a target panoramic image. Specifically, the target image features are selected as wall lines between adjacent walls in a room, hereinafter referred to as wall lines. After the modeling system identifies the wall lines, it annotates the wall lines in the panoramic image using annotation lines to obtain first annotation data. The wall lines are selected as target image features because they can reflect the contour characteristics of the room. Specifically, the modeling system can establish each wall surface of the room based on the first annotation data obtained from the annotated wall lines, thereby facilitating the creation of a three-dimensional sub-model of each room.

[0039] Specifically, the modeling system is equipped with an image recognition model. After training, the model can identify wall line features in panoramic images. Wall lines include not only the lines connecting the side walls of a room, but also the lines connecting the side walls to the ceiling and the floor.

[0040] In the embodiment of the present application, the wall lines between adjacent walls are identified and annotated as target image features, so that the modeling system can establish a three-dimensional sub-model based on the first annotation data obtained by the annotation.

[0041] In some embodiments of the present application, Figure 3 The third flow chart of the three-dimensional modeling method provided in the embodiment of the present application is shown as follows: Figure 3 As shown, multiple three-dimensional sub-models are established based on the first annotation data corresponding to multiple target panoramic images, including:

[0042] Step 302: determining the contour line of the model according to the first annotation data;

[0043] Step 304: Create a three-dimensional sub-model using the contour lines.

[0044] In this embodiment of the present application, after the modeling system annotates the wall lines to obtain first annotation data, the model's outline can be drawn in the modeling interface using this first annotation data. The outline is the inner contour of the model, and the modeling system automatically generates wall features within the model based on two adjacent outlines. Because the first annotation data is obtained based on the annotation of all the wall lines in the room, the 3D sub-model created based on the outline includes not only the side walls of the room, but also the ceiling and floor.

[0045] Specifically, the operation interface displays a panoramic interface and a model interface. The panoramic interface displays a panoramic image with the first annotation data, and the model interface displays a model of the generated wall surface. The panoramic interface and the model interface are displayed in correspondence, so that the panoramic image and the model are displayed on the same screen and at the same viewing angle.

[0046] In the embodiment of the present application, the modeling system can generate contour lines in the model through the first annotation data in the panoramic image, and fill materials between the generated contour lines to achieve the effect of establishing a three-dimensional sub-model of each room based on the first annotation data.

[0047] In some embodiments, after the 3D sub-model is established, the target panorama corresponding to the 3D sub-model is mapped to the 3D sub-model, so that the modeler can intuitively distinguish the correspondence between multiple 3D sub-models and multiple target panoramas.

[0048] In some embodiments of the present application, Figure 4 FIG4 shows a flow chart of the three-dimensional modeling method provided in an embodiment of the present application. Figure 4 As shown, before stitching multiple 3D sub-models to obtain the target 3D model of the house, the following steps are also included:

[0049] Step 402: in response to the annotation input for the target panoramic image, annotate the target panoramic image with second annotation data, where the second annotation data corresponds to the door structure in the house;

[0050] Step 404: Add door features to the three-dimensional sub-model based on the second annotation data.

[0051] In this embodiment of the present application, the first annotation data corresponds to the wall lines of the room, and the modeling system can establish the main framework of the 3D sub-model based on the first annotation data. Before splicing multiple 3D sub-models, it is necessary to establish splicing points for each 3D sub-model. It is understandable that multiple rooms in a house are connected by doors, so it is necessary to add door models to the 3D sub-models.

[0052] Specifically, users can manually annotate the target panorama in panoramic mode. After the modeling system receives the user's annotation input for the target panorama, it annotates the target panorama with second annotation data based on the user's annotation input. The second annotation data is the user's annotation of the door structure in the target panorama. Based on the second annotation data, the modeling system automatically adds the door feature to the 3D sub-model.

[0053] It is understandable that the doors in the room are all set on the walls. Therefore, the modeling system can automatically capture the wall corresponding to the second annotation data based on the second annotation data, and configure the generated door features on the corresponding wall in the 3D sub-model.

[0054] In an embodiment of the present application, the user manually annotates the door body in the panoramic view mode, and the modeling system can establish the door body features in the three-dimensional sub-model based on the second annotation data input by the user.

[0055] In some embodiments of the present application, Figure 5 The fifth flow chart of the three-dimensional modeling method provided in the embodiment of the present application is shown as follows: Figure 5 As shown, multiple 3D sub-models are spliced ​​together to obtain the target 3D model of the house, including:

[0056] Step 502 , receiving a first selection input for a first three-dimensional sub-model and a second three-dimensional sub-model among a plurality of three-dimensional sub-models;

[0057] Step 504 , in response to the first selection input, the first 3D sub-model and the second 3D sub-model are spliced ​​according to the door body features in the first 3D sub-model and the second 3D sub-model to obtain a target 3D model.

[0058] In the implementation of this application, after the modeling system completes the establishment of the three-dimensional sub-model of each room, when receiving the user's first selection input for the first three-dimensional sub-model and the second three-dimensional sub-model among multiple three-dimensional sub-models, it is determined that the user needs to splice the first three-dimensional sub-model and the second three-dimensional sub-model. The modeling system automatically identifies the door body features in the first three-dimensional sub-model and the second three-dimensional sub-model, and sets the door body features in the first three-dimensional sub-model and the second three-dimensional sub-model to overlap, completing the splicing of the first three-dimensional sub-model and the second three-dimensional sub-model.

[0059] When there are multiple rooms in a house, the modeling system splices each individual 3D sub-model to complete the modeling of the target 3D model of the house.

[0060] Specifically, the modeler selects two 3D sub-models in model mode. The modeling system automatically stitches the two sub-models together based on the door body features within the two 3D sub-models. After stitching the two sub-models together, an updated 3D sub-model is generated. By repeating the above steps, the user can stitch all the sub-models together into a complete target 3D model.

[0061] In the embodiment of the present application, the modeler only needs to select the target three-dimensional sub-model (the first three-dimensional sub-model and the second three-dimensional sub-model) from multiple three-dimensional sub-models, and the modeling system can automatically splice the first three-dimensional sub-model and the second three-dimensional sub-model, further simplifying the user's operation steps.

[0062] In some embodiments, the user can also select the spliced ​​3D sub-model and explode the 3D sub-model.

[0063] In some embodiments of the present application, Figure 6 FIG6 shows a flow chart of the three-dimensional modeling method provided in the embodiment of the present application, as shown in FIG6 Figure 6As shown, according to the door body features in the first 3D sub-model and the second 3D sub-model, the first 3D sub-model and the second 3D sub-model are spliced ​​together, including:

[0064] Step 602 , receiving a second selection input for door features in the first 3D sub-model and the second 3D sub-model;

[0065] Step 604 , in response to the second selection input, determining target door features in the first 3D sub-model and the second 3D sub-model;

[0066] Step 606: splice the first 3D sub-model and the second 3D sub-model according to the target door body features.

[0067] In an embodiment of the present application, when the first three-dimensional sub-model and the second three-dimensional sub-model include multiple door body features, the modeler needs to step by step select the first three-dimensional sub-model and the second three-dimensional sub-model from the multiple three-dimensional sub-models, and also needs to select the target door body features in the first three-dimensional sub-model and the second three-dimensional sub-model, so that the modeling system can splice the first three-dimensional sub-model and the second three-dimensional sub-model according to the target door body features manually selected by the modeler.

[0068] When the annotation system receives a second selection input from the user for a door body in the first and second 3D sub-models, it splices the first and second 3D sub-models using the target door body feature corresponding to the second selection input. Specifically, in model mode, the annotator selects the first 3D sub-model and the target door body feature in the first 3D sub-model, then selects the second 3D sub-model and the target door body feature in the second 3D sub-model. After the modeling system recognizes the second selection input for the target door body feature, it splices the first and second 3D sub-models together using the target door body feature.

[0069] In an embodiment of the present application, the user manually selects the door body features for splicing the two three-dimensional sub-models, thereby achieving accurate splicing of the two three-dimensional sub-models and avoiding the occurrence of three-dimensional sub-model splicing errors caused by the modeling system failing to recognize the door body features.

[0070] In some embodiments of the present application, Figure 7 FIG. 7 shows a flow chart of the three-dimensional modeling method provided in an embodiment of the present application. Figure 7 As shown, according to the door body features in the first 3D sub-model and the second 3D sub-model, the first 3D sub-model and the second 3D sub-model are spliced ​​together, including:

[0071] Step 702 , identifying the size information of the door features in the first 3D sub-model and the second 3D sub-model;

[0072] Step 704 , identifying target door features in the first 3D sub-model and the second 3D sub-model based on the size information;

[0073] Step 706: splice the first 3D sub-model and the second 3D sub-model according to the target door body features.

[0074] In an embodiment of the present application, the modeling system can automatically identify the size information of the door body features in the first three-dimensional sub-model and the second three-dimensional sub-model, and can determine whether the door body features in the first three-dimensional sub-model and the second three-dimensional sub-model have the same door body features based on the size information. If it is detected that the first three-dimensional sub-model and the second three-dimensional sub-model have the same door body features, the same door body features are used as target door body features, and the first three-dimensional sub-model and the second three-dimensional sub-model are spliced ​​through the target door body features. If it is detected that the first three-dimensional sub-model and the second three-dimensional sub-model do not have the same door body features, it is determined that there is no connection relationship between the first three-dimensional sub-model and the second three-dimensional sub-model, and the other three-dimensional sub-models are automatically screened for adaptability.

[0075] Specifically, after the modeling system has created multiple 3D sub-models, the modeler can automatically select and merge them. The modeling system identifies the connection between the multiple 3D sub-models by identifying common door features within them. Once the modeling system determines that common door features exist between the first and second 3D sub-models, the system uses these common door features as target door features to merge the first and second 3D sub-models.

[0076] In an embodiment of the present application, the modeling system can identify the target door body features based on the size information of the door body features in multiple three-dimensional sub-models, and then splice the multiple three-dimensional sub-models through the identified target door body features, thereby achieving the effect of the modeling system automatically splicing multiple three-dimensional sub-models, further simplifying the user's operation.

[0077] In some embodiments of the present application, Figure 8 FIG8 shows a flow chart of the three-dimensional modeling method provided in an embodiment of the present application. Figure 8 As shown, before stitching multiple 3D sub-models to obtain the target 3D model of the house, the following steps are also included:

[0078] Step 802: obtaining a mapping relationship between multiple target panoramas and multiple 3D sub-models;

[0079] Step 804 : Display multiple target panoramas and multiple 3D sub-models on the same screen according to the mapping relationship.

[0080] In the embodiment of the present application, the modeling system can display the target panorama and the 3D sub-model on the same screen with the same viewing angle based on the mapping relationship between the target panorama and the 3D sub-model. When the modeler adjusts the display angle of the target panorama, the display angle of the 3D sub-model will be linked accordingly.

[0081] Specifically, the operation interface of the modeling system displays a panoramic interface and a model interface. The panoramic image in the panoramic interface and the 3D sub-model in the model interface are displayed correspondingly according to the mapping relationship, so that the panoramic image and the model are displayed on the same screen and at the same viewing angle.

[0082] After displaying the multiple target panoramic images and the multiple 3D sub-models on the same screen according to the mapping relationship, the method further includes: receiving a viewing angle switching input and adjusting the display viewing angles of the 3D sub-models and the target panoramic images at the same time.

[0083] In the embodiment of the present application, by displaying the three-dimensional sub-model and the target panoramic view on the same screen with the same viewing angle, the modeling personnel can operate the three-dimensional sub-model and the target panoramic view more conveniently.

[0084] In some embodiments of the present application, after displaying multiple target panoramas and multiple three-dimensional sub-models on the same screen according to the mapping relationship, it also includes: receiving adjustment input for the first annotation data and / or the second annotation data; in response to the adjustment input, adjusting the target panorama and the three-dimensional sub-model at the same time according to the mapping relationship.

[0085] In an embodiment of the present application, the first annotation data and the second annotation data correspond to the wall features and door features in the three-dimensional sub-model respectively. When the modeler performs adjustment input on the first annotation data and / or the second annotation data in the panoramic view, the modeling system adjusts the first annotation data and / or the second annotation data in the target panoramic view and the three-dimensional sub-model at the same time according to the adjustment input, so that the annotation data in the three-dimensional sub-model and the target panoramic view can be adjusted in conjunction.

[0086] Specifically, after the modeling system generates a 3D sub-model based on the current first and second annotated data, the modeler determines whether the modeling meets the requirements by observing the 3D sub-model and the target panoramic view. If the modeler needs to adjust the 3D sub-model, the modeler adjusts the wall features in the 3D sub-model by adjusting the first annotated data in the target panoramic view, and adjusts the door features in the 3D sub-model by adjusting the second annotated data in the target panoramic view.

[0087] In the embodiment of the present application, the target panorama and the 3D sub-model are displayed on the same screen in the display interface of the modeling system. If the modeler needs to adjust the 3D sub-model, there is no need to operate the 3D sub-model. Instead, the modeler only needs to adjust the annotation data in the target panorama to complete the editing of the 3D sub-model, thereby simplifying the modeler's operation steps.

[0088] In some embodiments of the present application, after splicing multiple three-dimensional sub-models to obtain a target three-dimensional model of a house, the method further includes: obtaining size information of the target three-dimensional model; and generating a two-dimensional floor plan of the house based on the size information.

[0089] In this embodiment of the present application, the modeling system can also obtain dimensional information of the completed target 3D model. This dimensional information includes, but is not limited to, the detailed dimensions of each room in the house and the proportional relationship between the dimensions of each room in the house. Based on the dimensional information of each room, the modeling system can generate a 2D floor plan corresponding to the target 3D model, thereby achieving the effect of the modeling system automatically generating a 2D floor plan based on the target 3D model.

[0090] The 3D modeling method provided in the embodiment of the present application can be executed by a 3D modeling device. In the embodiment of the present application, the 3D modeling device provided in the embodiment of the present application is described by taking the 3D modeling method executed by the 3D modeling device as an example.

[0091] In some embodiments of the present application, a three-dimensional modeling device is provided. Figure 9 The structural block diagram of the three-dimensional modeling device provided in the embodiment of the present application is shown as follows: Figure 9 As shown, the three-dimensional modeling device 900 includes:

[0092] An acquisition module 902 is configured to acquire a plurality of target panoramic images, where the plurality of target panoramic images correspond to a plurality of rooms in a house;

[0093] Annotation module 904, configured to annotate first annotation data of each target panoramic image;

[0094] A modeling module 906 is configured to establish a plurality of three-dimensional sub-models according to the first annotation data corresponding to the plurality of target panoramic images, wherein the plurality of three-dimensional sub-models correspond to the plurality of rooms;

[0095] The splicing module 908 is used to splice multiple 3D sub-models to obtain a target 3D model of the house.

[0096] The embodiment of the present application obtains a target panoramic image corresponding to each room in a house, automatically annotates first annotation data based on image features in the target panoramic image, and creates a three-dimensional sub-model for each individual room based on the first annotation data. By stitching together the three-dimensional sub-models, the three-dimensional modeling of the entire house can be completed. The modeling system automatically annotates the first annotation data and automatically generates corresponding three-dimensional sub-models based on the first annotation data, reducing the number of user operations and improving the efficiency of building a three-dimensional model of a house.

[0097] In some embodiments of the present application, the three-dimensional modeling device 900 further includes:

[0098] A recognition module, configured to recognize target image features in the target panoramic image, wherein the target image features include wall lines between adjacent walls;

[0099] The annotation module 904 is further configured to annotate first annotation data in the target panoramic image according to the target image features.

[0100] In the embodiment of the present application, the wall lines between adjacent walls are identified and annotated as target image features, so that the modeling system can establish a three-dimensional sub-model based on the first annotation data obtained by the annotation.

[0101] In some embodiments of the present application, the three-dimensional modeling device 900 further includes:

[0102] A determination module, configured to determine a contour line of the model based on the first annotation data;

[0103] The modeling module 906 is used to establish a three-dimensional sub-model through contour lines.

[0104] In the embodiment of the present application, the modeling system can generate contour lines in the model through the first annotation data in the panoramic image, and fill materials between the generated contour lines to achieve the effect of establishing a three-dimensional sub-model of each room based on the first annotation data.

[0105] In some embodiments of the present application, the annotation module 904 is further configured to, in response to an annotation input for the target panoramic image, annotate the target panoramic image with second annotation data, where the second annotation data corresponds to a door structure in the house;

[0106] The three-dimensional modeling device 900 further includes:

[0107] The adding module is also used to add door body features in the three-dimensional sub-model according to the second annotation data.

[0108] In an embodiment of the present application, the user manually annotates the door body in the panoramic view mode, and the modeling system can establish the door body features in the three-dimensional sub-model based on the second annotation data input by the user.

[0109] In some embodiments of the present application, the three-dimensional modeling device 900 further includes:

[0110] A receiving module, configured to receive a first selection input for a first three-dimensional sub-model and a second three-dimensional sub-model among the plurality of three-dimensional sub-models;

[0111] The splicing module 908 is used to respond to the first selection input and splice the first 3D sub-model and the second 3D sub-model according to the door body features in the first 3D sub-model and the second 3D sub-model to obtain a target 3D model.

[0112] In the embodiment of the present application, the modeler only needs to select the target three-dimensional sub-model (the first three-dimensional sub-model and the second three-dimensional sub-model) from multiple three-dimensional sub-models, and the modeling system can automatically splice the first three-dimensional sub-model and the second three-dimensional sub-model, further simplifying the user's operation steps.

[0113] In some embodiments of the present application, the three-dimensional modeling device 900 further includes:

[0114] A receiving module, configured to receive a second selection input for door body features in the first three-dimensional sub-model and the second three-dimensional sub-model;

[0115] The determination module is further configured to determine target door features in the first three-dimensional sub-model and the second three-dimensional sub-model in response to a second selection input;

[0116] The splicing module 908 is used to splice the first 3D sub-model and the second 3D sub-model according to the target door body features.

[0117] In an embodiment of the present application, the user manually selects the door body features for splicing the two three-dimensional sub-models, thereby achieving accurate splicing of the two three-dimensional sub-models and avoiding the occurrence of three-dimensional sub-model splicing errors caused by the modeling system failing to recognize the door body features.

[0118] In some embodiments of the present application, the three-dimensional modeling device 900 further includes:

[0119] A recognition module, configured to recognize size information of door features in the first 3D sub-model and the second 3D sub-model;

[0120] The recognition module is further used to recognize target door features in the first 3D sub-model and the second 3D sub-model according to the size information;

[0121] The splicing module 908 is used to splice the first 3D sub-model and the second 3D sub-model according to the target door body characteristics.

[0122] In an embodiment of the present application, the modeling system can identify the target door body features based on the size information of the door body features in multiple three-dimensional sub-models, and then splice the multiple three-dimensional sub-models through the identified target door body features, thereby achieving the effect of the modeling system automatically splicing multiple three-dimensional sub-models, further simplifying the user's operation.

[0123] In some embodiments of the present application, the acquisition module 902 is configured to acquire a mapping relationship between a plurality of target panoramas and a plurality of three-dimensional sub-models;

[0124] The display module is used to display multiple target panoramas and multiple three-dimensional sub-models on the same screen according to the mapping relationship.

[0125] In the embodiment of the present application, by displaying the three-dimensional sub-model and the target panoramic view on the same screen with the same viewing angle, the modeling personnel can operate the three-dimensional sub-model and the target panoramic view more conveniently.

[0126] In some embodiments of the present application, the acquisition module 902 is used to obtain size information of the target three-dimensional model;

[0127] The three-dimensional modeling device 900 further includes:

[0128] The generation module is used to generate a two-dimensional floor plan of a house based on the size information.

[0129] In this embodiment of the present application, the modeling system can also obtain dimensional information of the completed target 3D model. This dimensional information includes, but is not limited to, the detailed dimensions of each room in the house and the proportional relationship between the dimensions of each room in the house. Based on the dimensional information of each room, the modeling system can generate a 2D floor plan corresponding to the target 3D model, thereby achieving the effect of the modeling system automatically generating a 2D floor plan based on the target 3D model.

[0130] The three-dimensional modeling device in the embodiment of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not specifically limit it.

[0131] The three-dimensional modeling device in the embodiment of the present application can be a device having an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0132] The three-dimensional modeling device provided in the embodiment of the present application can implement each process implemented in the above method embodiment. To avoid repetition, it will not be described here.

[0133] Alternatively, as Figure 10 As shown, an embodiment of the present application also provides an electronic device 1000, which includes a processor 1002 and a memory 1004. The memory 1004 stores programs or instructions that can be run on the processor 1002. When the program or instructions are executed by the processor 1002, the various steps of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.

[0134] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0135] Figure 11 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0136] The electronic device 1100 includes but is not limited to components such as a radio frequency unit 1101 , a network module 1102 , an audio output unit 1103 , an input unit 1104 , a sensor 1105 , a display unit 1106 , a user input unit 1107 , an interface unit 1108 , a memory 1109 , and a processor 1110 .

[0137] Those skilled in the art will understand that the electronic device 1100 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 1110 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 11 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0138] The input unit 1104 is used to obtain multiple target panoramic images, where the multiple target panoramic images correspond to multiple rooms in the house;

[0139] Processor 1110, configured to annotate first annotation data of each target panoramic image;

[0140] Processor 1110, configured to establish a plurality of 3D sub-models according to the first annotation data corresponding to the plurality of target panoramic images, wherein the plurality of 3D sub-models correspond to the plurality of rooms;

[0141] The processor 1110 is configured to combine the multiple 3D sub-models to obtain a target 3D model of the house.

[0142] The embodiment of the present application obtains a target panoramic image corresponding to each room in a house, automatically annotates first annotation data based on image features in the target panoramic image, and creates a three-dimensional sub-model for each individual room based on the first annotation data. By stitching together the three-dimensional sub-models, the three-dimensional modeling of the entire house can be completed. The modeling system automatically annotates the first annotation data and automatically generates corresponding three-dimensional sub-models based on the first annotation data, reducing the number of user operations and improving the efficiency of building a three-dimensional model of a house.

[0143] In the embodiment of the present application, the processor 1110 is configured to identify target image features in the target panoramic image, where the target image features include wall lines between adjacent walls;

[0144] The processor 1110 is configured to annotate first annotation data in the target panoramic image according to target image features.

[0145] In the embodiment of the present application, the wall lines between adjacent walls are identified and annotated as target image features, so that the modeling system can establish a three-dimensional sub-model based on the first annotation data obtained by the annotation.

[0146] Furthermore, the processor 1110 is configured to determine a contour line of the model based on the first annotation data;

[0147] The processor 1110 is configured to create a three-dimensional sub-model using contour lines.

[0148] In the embodiment of the present application, the modeling system can generate contour lines in the model through the first annotation data in the panoramic image, and fill materials between the generated contour lines to achieve the effect of establishing a three-dimensional sub-model of each room based on the first annotation data.

[0149] Furthermore, the processor 1110 is configured to, in response to a labeling input for the target panoramic image, label the target panoramic image with second labeling data, where the second labeling data corresponds to a door structure in the house;

[0150] The processor 1110 is configured to add door features to the three-dimensional sub-model according to the second annotation data.

[0151] In an embodiment of the present application, the user manually annotates the door body in the panoramic view mode, and the modeling system can establish the door body features in the three-dimensional sub-model based on the second annotation data input by the user.

[0152] Furthermore, the input unit 1104 is configured to receive a first selection input for a first three-dimensional sub-model and a second three-dimensional sub-model among the plurality of three-dimensional sub-models;

[0153] The processor 1110 is configured to, in response to a first selection input, splice the first 3D sub-model and the second 3D sub-model according to door features in the first 3D sub-model and the second 3D sub-model to obtain a target 3D model.

[0154] In the embodiment of the present application, the modeler only needs to select the target three-dimensional sub-model (the first three-dimensional sub-model and the second three-dimensional sub-model) from multiple three-dimensional sub-models, and the modeling system can automatically splice the first three-dimensional sub-model and the second three-dimensional sub-model, further simplifying the user's operation steps.

[0155] Furthermore, the input unit 1104 is configured to receive a second selection input for door features in the first three-dimensional sub-model and the second three-dimensional sub-model;

[0156] Processor 1110 is configured to determine target door features in the first 3D sub-model and the second 3D sub-model in response to a second selection input;

[0157] The processor 1110 is configured to combine the first three-dimensional sub-model and the second three-dimensional sub-model according to the target door body feature.

[0158] In an embodiment of the present application, the user manually selects the door body features for splicing the two three-dimensional sub-models, thereby achieving accurate splicing of the two three-dimensional sub-models and avoiding the occurrence of three-dimensional sub-model splicing errors caused by the modeling system failing to recognize the door body features.

[0159] Furthermore, the processor 1110 is configured to identify size information of door features in the first three-dimensional sub-model and the second three-dimensional sub-model;

[0160] Processor 1110, configured to identify target door features in the first 3D sub-model and the second 3D sub-model based on the size information;

[0161] The processor 1110 is configured to combine the first three-dimensional sub-model and the second three-dimensional sub-model according to the target door body feature.

[0162] In an embodiment of the present application, the modeling system can identify the target door body features based on the size information of the door body features in multiple three-dimensional sub-models, and then splice the multiple three-dimensional sub-models through the identified target door body features, thereby achieving the effect of the modeling system automatically splicing multiple three-dimensional sub-models, further simplifying the user's operation.

[0163] Furthermore, the processor 1110 is configured to obtain a mapping relationship between a plurality of target panoramic images and a plurality of three-dimensional sub-models;

[0164] The display unit 1106 is configured to display multiple target panoramas and multiple 3D sub-models on the same screen according to the mapping relationship.

[0165] In the embodiment of the present application, by displaying the three-dimensional sub-model and the target panoramic view on the same screen with the same viewing angle, the modeling personnel can operate the three-dimensional sub-model and the target panoramic view more conveniently.

[0166] Furthermore, the processor 1110 is configured to obtain size information of the target three-dimensional model;

[0167] Processor 1110 is configured to generate a two-dimensional floor plan of the house based on the size information.

[0168] In this embodiment of the present application, the modeling system can also obtain dimensional information of the completed target 3D model. This dimensional information includes, but is not limited to, the detailed dimensions of each room in the house and the proportional relationship between the dimensions of each room in the house. Based on the dimensional information of each room, the modeling system can generate a 2D floor plan corresponding to the target 3D model, thereby achieving the effect of the modeling system automatically generating a 2D floor plan based on the target 3D model.

[0169] It should be understood that in an embodiment of the present application, the input unit 1104 may include a graphics processing unit (GPU) 11041 and a microphone 11042, and the graphics processor 11041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1107 includes a touch panel 11071 and at least one of other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include two parts: a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be repeated here.

[0170] The memory 1109 can be used to store software programs and various data. The memory 1109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1109 may include a volatile memory or a non-volatile memory, or the memory 1109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 1109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0171] Processor 1110 may include one or more processing units. Optionally, processor 1110 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 1110.

[0172] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned three-dimensional modeling method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0173] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0174] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0175] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0176] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned three-dimensional modeling method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0177] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0178] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0179] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A three-dimensional modeling method, characterized in that: The three-dimensional modeling method is used for three-dimensional modeling of a house, and the three-dimensional modeling method includes: Acquire a plurality of target panoramic images captured by a camera, wherein the plurality of target panoramic images correspond to a plurality of rooms in the house; annotating first annotation data of each target panoramic image; establishing a plurality of three-dimensional sub-models according to the first annotation data corresponding to the plurality of target panoramic images, wherein the plurality of three-dimensional sub-models correspond to the plurality of rooms; splicing the multiple three-dimensional sub-models to obtain a target three-dimensional model of the house; The first annotation data for annotating each target panoramic image includes: identifying target image features in the target panoramic image, wherein the target image features include wall lines between adjacent walls; marking the first annotation data in the target panoramic image according to the target image feature; Before splicing the multiple three-dimensional sub-models to obtain the target three-dimensional model of the house, the method further includes: In response to a labeling input for the target panoramic image, labeling second labeling data in the target panoramic image, where the second labeling data corresponds to a door structure in the house; adding a door feature to the three-dimensional sub-model according to the second annotation data; Before splicing the multiple three-dimensional sub-models to obtain the target three-dimensional model of the house, the method further includes: Acquire a mapping relationship between the plurality of target panoramas and the plurality of three-dimensional sub-models; displaying the plurality of target panoramas and the plurality of three-dimensional sub-models on the same screen according to the mapping relationship; After displaying the plurality of target panoramas and the plurality of three-dimensional sub-models on the same screen according to the mapping relationship, the method further includes: receiving an adjustment input for the first annotation data and / or the second annotation data corresponding to the plurality of target panoramic images; In response to the adjustment input, simultaneously adjusting the target panoramic image and the three-dimensional sub-model according to the mapping relationship; and After displaying the plurality of target panoramas and the plurality of three-dimensional sub-models on the same screen according to the mapping relationship, the method further includes: A perspective switching input for the plurality of target panoramic images is received, and a display perspective of the three-dimensional sub-model and the target panoramic image is adjusted simultaneously.

2. The three-dimensional modeling method according to claim 1, characterized in that: The step of establishing a plurality of three-dimensional sub-models according to the first annotation data corresponding to the plurality of target panoramic images includes: determining a contour line of the model according to the first annotation data; The three-dimensional sub-model is established using the contour lines.

3. The three-dimensional modeling method according to claim 1, characterized in that: The step of splicing the multiple three-dimensional sub-models to obtain a target three-dimensional model of the house includes: receiving a first selection input for a first three-dimensional sub-model and a second three-dimensional sub-model among the plurality of three-dimensional sub-models; In response to the first selection input, the first three-dimensional sub-model and the second three-dimensional sub-model are spliced ​​according to the door features in the first three-dimensional sub-model and the second three-dimensional sub-model to obtain the target three-dimensional model.

4. The three-dimensional modeling method according to claim 3, characterized in that: In response to the first selection input, splicing the first three-dimensional sub-model and the second three-dimensional sub-model according to the door body features in the first three-dimensional sub-model and the second three-dimensional sub-model includes: receiving a second selection input for the door features in the first three-dimensional sub-model and the second three-dimensional sub-model; In response to the second selection input, determining target door features in the first three-dimensional sub-model and the second three-dimensional sub-model; The first three-dimensional sub-model and the second three-dimensional sub-model are spliced ​​according to the target door body features.

5. The three-dimensional modeling method according to claim 3, characterized in that: In response to the first selection input, splicing the first three-dimensional sub-model and the second three-dimensional sub-model according to the door body features in the first three-dimensional sub-model and the second three-dimensional sub-model includes: Identifying dimensional information of the door features in the first three-dimensional sub-model and the second three-dimensional sub-model; identifying target door features in the first three-dimensional sub-model and the second three-dimensional sub-model according to the size information; The first three-dimensional sub-model and the second three-dimensional sub-model are spliced ​​according to the target door body features.

6. The three-dimensional modeling method according to any one of claims 1 to 2, characterized in that: After the plurality of three-dimensional sub-models are spliced ​​together to obtain the target three-dimensional model of the house, the method further includes: Acquiring dimension information of the target three-dimensional model; A two-dimensional floor plan of the house is generated according to the size information.

7. A three-dimensional modeling device, characterized in that: The three-dimensional modeling device is used for three-dimensional modeling of a house, and the three-dimensional modeling device includes: an acquisition module, configured to acquire a plurality of target panoramic images captured by a camera, wherein the plurality of target panoramic images correspond to a plurality of rooms in the house; a marking module, configured to mark first marking data of each target panoramic image; a modeling module, configured to establish a plurality of three-dimensional sub-models according to the first annotation data corresponding to the plurality of target panoramic images, wherein the plurality of three-dimensional sub-models correspond to the plurality of rooms; a splicing module, configured to splice the multiple three-dimensional sub-models to obtain a target three-dimensional model of the house; A recognition module, configured to recognize target image features in the target panoramic image, wherein the target image features include wall lines between adjacent walls; The annotation module is further configured to annotate the first annotation data in the target panorama according to the target image features. The annotation module is further configured to, in response to an annotation input for the target panoramic image, annotate second annotation data in the target panoramic image, wherein the second annotation data corresponds to a door structure in the house; an adding module, further configured to add door features to the three-dimensional sub-model according to the second annotation data; The acquisition module is configured to acquire a mapping relationship between the plurality of target panoramas and the plurality of three-dimensional sub-models; A display module, configured to display the multiple target panoramas and the multiple three-dimensional sub-models on the same screen according to the mapping relationship; The display module is specifically used to receive adjustment input for the first annotation data and / or the second annotation data corresponding to the multiple target panoramic images; in response to the adjustment input, simultaneously adjust the target panoramic image and the three-dimensional sub-model according to the mapping relationship; and receive perspective switching input for the multiple target panoramic images, and simultaneously adjust the display perspective of the three-dimensional sub-model and the target panoramic image.

8. An electronic device, characterized in that: include: a memory on which programs or instructions are stored; A processor, configured to implement the steps of the three-dimensional modeling method according to any one of claims 1 to 7 when executing the program or instructions.

9. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the three-dimensional modeling method according to any one of claims 1 to 7 are implemented.

Images