Housing space layout estimation method, device, equipment and storage medium based on multi-view panorama and multi-label graph cut

Abstract

The present application discloses a method, device, equipment and storage medium for estimating the spatial layout of a house based on multi-view panorama and multi-label graph cuts. The present application relates to the field of three-dimensional reconstruction technology. The method includes: performing two-dimensional layout prediction on the indoor multi-view panoramic view of the target house to obtain a two-dimensional layout view; performing preset ray projection based on the two-dimensional layout view to obtain an initial plan view; performing preset multi-label graph cut regularization on the initial plan view to obtain a regularized plan view; performing panoramic geometric transformation on the regularized plan view to obtain a target layout view corresponding to the target house. The present application first obtains the initial two-dimensional layout view corresponding to the indoor panoramic view of the house, then performs ray projection on it to obtain the initial plan view, then regularizes the initial plan view, and uses multi-label graph cuts to obtain an accurate and complete regularized plan view. The regularized plan view is then transformed to obtain a target layout view corresponding to the target house that is truly and reliably represented by a panoramic geometric relationship.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction technology, and in particular to a method, device, equipment and storage medium for estimating the spatial layout of a house based on multi-view panorama and multi-label graph cuts. Background Art

[0002] Room layout estimation is an indispensable key component of real-world 3D. Its goal is to estimate the abstract model of indoor scenes to support various applications such as virtual reality, asset management, indoor navigation, and games.

[0003] However, due to the common problems of textureless, transparent, and highlight areas in indoor scenes and limited training data, as well as the fact that a single panoramic image is easily occluded by walls or furniture, existing multi-vision reconstruction techniques for complex indoor scenes find it difficult to estimate the complete layout.

[0004] Therefore, how to provide a real and reliable house space layout estimation technology has become an urgent problem to be solved. Summary of the invention

[0005] The main purpose of this application is to provide a method, device, equipment and storage medium for estimating the spatial layout of a house based on multi-view panorama and multi-label graph cuts, aiming to solve the technical problem of how to provide a truly reliable technology for estimating the spatial layout of a house.

[0006] To achieve the above objectives, this application proposes a housing space layout estimation method based on multi-view panorama and multi-label graph cuts, the method comprising:

[0007] Perform two-dimensional layout prediction on the indoor multi-view panoramic image of the target house to obtain a two-dimensional layout view;

[0008] Perform preset ray projection based on the two-dimensional layout view to obtain an initial plan view;

[0009] Performing a preset multi-label graph cut regularization on the initial plane graph to obtain a regularized plane graph;

[0010] Performing panoramic geometric transformation on the regularized plan view to obtain a target layout view corresponding to the target house.

[0011] In one embodiment, the step of performing two-dimensional layout prediction on the indoor multi-view panoramic image of the target house to obtain a two-dimensional layout view includes:

[0012] Predict plane intersections of the indoor multi-view panoramic image of the target house to obtain the initial layout view;

[0013] Post-processing vectorization is performed on the initial layout view to obtain a two-dimensional layout view.

[0014] In one embodiment, the step of performing preset ray projection based on the two-dimensional layout view to obtain an initial plan view includes:

[0015] Projecting the two-dimensional layout view onto the ground to obtain a full set of candidate layout panoramas;

[0016] The entire candidate layout panorama set is aggregated along several ray directions through a preset ray projection function to obtain an initial plan view.

[0017] In one embodiment, the step of projecting the two-dimensional layout view onto the ground to obtain a full set of candidate layout panoramas includes:

[0018] Get the current camera external parameters;

[0019] Performing preliminary line projection on the two-dimensional layout view by using a projection function and the current camera extrinsic parameters to obtain an initial layout panorama;

[0020] The initial layout panorama is split to obtain a complete set of candidate layout panoramas.

[0021] In one embodiment, the step of performing a preset multi-label graph cut regularization on the initial plan graph to obtain a regularized plan graph includes:

[0022] Segmenting the entire candidate layout panorama to obtain initial candidate walls;

[0023] Performing a multi-label graph cut on the initial plan view based on the initial candidate walls to obtain a target plan view label;

[0024] The normal of the initial plane map is migrated based on the target plane map label to obtain a regularized plane map.

[0025] In one embodiment, the step of performing a multi-label graph cut on the initial plan view based on the initial candidate wall to obtain a target plan view label includes:

[0026] Constructing a label allocation energy function corresponding to the initial plane graph based on the initial candidate wall;

[0027] The label assignment function is used to assign labels to the initial plan view to obtain target plan view labels.

[0028] In one embodiment, the step of constructing a label allocation energy function corresponding to the initial plan view based on the initial candidate wall includes:

[0029] Taking the initial candidate wall as a candidate label, obtaining the candidate label cost corresponding to the initial plane map;

[0030] Obtaining adjacent point label energy items corresponding to the initial plane graph;

[0031] A label allocation energy function corresponding to the initial plane graph is determined according to the candidate label cost and the adjacent point label energy item.

[0032] In addition, to achieve the above purpose, the present application also proposes a housing space layout estimation device based on multi-view panorama and multi-label graph cut, and the housing space layout estimation device based on multi-view panorama and multi-label graph cut includes:

[0033] An initial layout estimation module is used to predict the 2D layout of the indoor multi-view panoramic image of the target house and obtain a 2D layout view;

[0034] A layout projection module, used for performing preset ray projection based on the two-dimensional layout view to obtain an initial plan view;

[0035] A layout optimization module, used for performing a preset multi-label graph cut regularization on the initial plan view to obtain a regularized plan view;

[0036] The layout conversion module is used to perform panoramic geometric conversion on the regularized plan view to obtain a target layout view corresponding to the target house.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a house space layout estimation device based on multi-view panorama and multi-label graph cuts, the device including: a memory, a processor, and a house space layout estimation program based on multi-view panorama and multi-label graph cuts stored in the memory and runnable on the processor, the house space layout estimation program based on multi-view panorama and multi-label graph cuts being configured to implement the steps of the house space layout estimation method based on multi-view panorama and multi-label graph cuts as described above.

[0038] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which stores a program for implementing a house space layout estimation method based on multi-view panorama and multi-label graph cuts, and the program for implementing a house space layout estimation method based on multi-view panorama and multi-label graph cuts is executed by a processor to implement the steps of the house space layout estimation method based on multi-view panorama and multi-label graph cuts as described above.

[0039] The present application provides a method, device, equipment and storage medium for estimating the spatial layout of a house based on multi-view panorama and multi-label graph cuts. The method includes performing two-dimensional layout prediction on the indoor multi-view panorama of the target house to obtain a two-dimensional layout view; performing preset ray projection based on the two-dimensional layout view to obtain an initial plan view; performing preset multi-label graph cut regularization on the initial plan view to obtain a regularized plan view; performing panoramic geometric transformation on the regularized plan view to obtain a target layout view corresponding to the target house.

[0040] This application first obtains the initial two-dimensional layout view corresponding to the indoor panoramic view of the house. Then, considering the problem that a single panorama is easily blocked by walls or furniture, this application performs ray projection based on the two-dimensional layout view to obtain the initial plan view. The initial plan view is then regularized, and a multi-label graph cut is used to obtain an accurate and complete regularized plan view. Finally, the regularized plan view is converted into a layout with a panoramic geometric relationship to obtain a real and reliable target layout view corresponding to the target house. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] Figure 1 This is a first flow chart of the first embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of the present application;

[0044] Figure 2 This is a second flow chart of the first embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of the present application;

[0045] Figure 3 This is a schematic diagram of the initial plane analysis process of the first embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of the present application;

[0046] Figure 4 This is a schematic diagram of the initial plan view effect of the second embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of the present application;

[0047] Figure 5 This is a flow chart of a second embodiment of a housing space layout estimation method based on multi-view panorama and multi-label graph cuts of the present application;

[0048] Figure 6 A schematic diagram of the regularization process of the second embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of the present application;

[0049] Figure 7 A brief flowchart of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts in this application;

[0050] Figure 8A schematic diagram showing the comparison of layout estimation effects of the housing space layout estimation method based on multi-view panorama and multi-label graph cut in this application;

[0051] Fig. 9 A schematic diagram of the module structure of a housing space layout estimation device based on multi-view panorama and multi-label graph cuts according to an embodiment of the present application;

[0052] Fig.10 Schematic diagram of the device structure of the hardware operating environment involved in the housing space layout estimation method based on multi-view panorama and multi-label graph cuts in the embodiment of the present application.

[0053] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0055] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] The main solution of this application is: to predict the two-dimensional layout of the indoor multi-view panoramic image of the target house to obtain a two-dimensional layout view; to perform preset ray projection based on the two-dimensional layout view to obtain an initial floor plan; to perform preset multi-label graph cut regularization on the initial floor plan to obtain a regularized floor plan; to perform panoramic geometric transformation on the regularized floor plan to obtain a target layout view corresponding to the target house.

[0057] At present, due to the common problems of textureless, transparent, highlight areas and limited training data in indoor scenes, the existing multi-vision reconstruction technology for indoor scenes cannot generate accurate and complete 3D points, which affects the quality of image post-processing, and most of the research work that has been published so far focuses on improving the estimation accuracy of a single panorama. However, a single panorama is easily blocked by walls or furniture. Therefore, it is difficult to estimate an accurate and complete layout in complex indoor scenes. Therefore, the accuracy of the existing multi-vision reconstruction technology for complex indoor scenes needs to be improved.

[0058] To solve the above problems, this application first obtains the initial two-dimensional layout view corresponding to the indoor panoramic view of the house. Then, considering the problem that a single panorama is easily blocked by walls or furniture, this application performs ray projection based on the two-dimensional layout view to obtain the initial plan view. The initial plan view is then regularized, and a multi-label graph cut is used to obtain an accurate and complete regularized plan view. Finally, the regularized plan view is converted into a layout with a panoramic geometric relationship to obtain a real and reliable target layout view corresponding to the target house.

[0059] It should be noted that the execution subject of this embodiment can be a house space layout estimation system based on multi-view panorama and multi-label graph cuts, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a house space layout estimation device based on multi-view panorama and multi-label graph cuts that can achieve the above functions, etc. This embodiment does not specifically limit this. The following takes the house space layout estimation device based on multi-view panorama and multi-label graph cuts (referred to as the estimation device) as an example of the execution subject to illustrate this embodiment and the following embodiments.

[0060] Based on this, the embodiment of the present application provides a housing space layout estimation method based on multi-view panorama and multi-label graph cuts, referring to Figure 1 , Figure 1 This is a first flow chart of the first embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of the present application.

[0061] In this embodiment, the housing space layout estimation method based on multi-view panorama and multi-label graph cuts includes steps S10 to S40:

[0062] Step S10, performing two-dimensional layout prediction on the indoor multi-view panoramic image of the target house to obtain a two-dimensional layout view;

[0063] It is easy to understand that the above-mentioned indoor multi-view panoramic image can be a set of known camera extrinsic parameters corresponding to the target house, and an indoor calibrated panoramic image is obtained by shooting at multiple angles. The purpose of this embodiment is to estimate the accurate and complete layout of each view so as to support virtual reality, asset management, indoor navigation, games and other applications corresponding to the target house.

[0064] It should be understood that the traditional edge- or semantic-based panorama-to-layout conversion method has the defect of easily failing in occluded areas. To avoid this problem, this embodiment can use the plane intersection in the indoor multi-view panorama of the target house as a reference to convert it into a two-dimensional layout view.

[0065] In a feasible implementation manner, in this embodiment, step S10 may include steps A1 to A2:

[0066] Step A1, predicting plane intersections of the indoor multi-view panoramic image of the target house to obtain an initial layout view;

[0067] Step A2: performing post-processing vectorization on the initial layout view to obtain a two-dimensional layout view.

[0068] It should be noted that the predicted plane intersection in this embodiment can be the intersection of the ground wall plane and the top wall plane in the indoor multi-view panoramic view. You can use layout elements express, It can represent the intersection of ground and wall planes. It can represent the intersection point of ground and wall planes.

[0069] Specifically, this embodiment can use any existing single-view panoramic image layout prediction model, such as View Crafter or Horizon Net, to generate a pre-trained two-dimensional layout of each image based on the intersection of the ground wall plane and the top wall plane with the existing indoor multi-angle image dataset, obtain an intersection layout prediction model, and estimate the 2D layout of the indoor multi-view panoramic image of the target house based on the pre-trained intersection layout prediction model to obtain an initial layout view. The processing process can be expressed as follows:

[0070] (1)

[0071] in, is the initial layout view, It is a multi-view panoramic picture of the interior. Layout prediction model for intersections.

[0072] It is important to understand that the layout estimated by the intersection layout prediction model may contain many inaccurate lines. In addition, in similar indoor scenes, the deep learning method can achieve better results with sufficient training data. However, it may fail in unseen scenes.

[0073] Therefore, after the initial layout map is predicted in the form of dense line points on the floor-wall and ceiling-wall planes by the intersection layout prediction model, in order to make full use of the pre-trained model and the multi-view panorama, the present embodiment can further use the post-processed vectorized line results of the initial layout view as the final two-dimensional layout view containing a superset of all precise layout elements. Since there may be multiple indoor scenes in an actual house that do not obey the Manhattan assumption, that is, adjacent walls are not perpendicular to each other. Therefore, this embodiment can select a post-processing method that is not restricted by Manhattan to process the initial layout view to adapt to non-Manhattan indoor scenes and expand the applicable scenarios of the solution proposed in this embodiment.

[0074] Step S20, performing preset ray projection based on the two-dimensional layout view to obtain an initial plan view;

[0075] It is easy to understand that in order to avoid the distortion and discreteness problems that occur in the image domain, this embodiment chooses to directly represent the layout geometry in Euclidean space. For this purpose, the planar layout diagram representation is a perfect choice. In a feasible implementation, refer to Figure 2 , Figure 2This is a second flow chart of the first embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of this application. In this embodiment, step S20 may include steps B1-B2:

[0076] Step B1, projecting the two-dimensional layout view onto the ground to obtain a full set of candidate layout panoramas;

[0077] It is understandable that, considering the problem that a single panorama is easily blocked by walls or furniture, this embodiment can project the two-dimensional layout view onto the ground to obtain a complete set of candidate layout panoramas that can represent the intersection of the ground and the wall, so as to perform accurate plane analysis based on the complete set of candidate layout panoramas.

[0078] In a feasible implementation manner, in this embodiment, step B1 may include steps B11 to B13:

[0079] Step B11, obtaining the current camera external parameters;

[0080] Step B12, performing preliminary line projection on the two-dimensional layout view by using a projection function and the current camera extrinsic parameters to obtain an initial layout panorama;

[0081] Step B13, splitting the initial layout panorama to obtain a complete set of candidate layout panoramas.

[0082] It is easy to understand that the above current camera external parameters can be the known camera external parameters corresponding to the indoor multi-view panoramic image of the target house, which can be expressed as In order to integrate the layout elements of different panoramas, the process of projecting the above two-dimensional layout view to the ground can be achieved by using the projection function , combined with the current camera external parameters All layout elements in the two-dimensional layout view are converted into a world coordinate system, and a preliminary line projection of the layout elements onto the ground is performed to obtain a plan view, thereby obtaining the above-mentioned initial layout panorama.

[0083] Furthermore, since there may be many inaccurate wall-floor intersections in the initial layout panorama, this embodiment can further split the plan view candidate points in the initial layout panorama into wall-floor intersection candidate points according to different straight lines to form a superset containing all accurate wall-floor intersection candidate points, and obtain the above candidate layout panorama set, which is expressed as , the specific process can be expressed as follows:

[0084] ; (2)

[0085] For ease of understanding, refer to Figure 3 The process of obtaining the full set of candidate layout panoramas in this embodiment is described by way of example. Figure 3This is a schematic diagram of the initial plane analysis process of the first embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts in this application. Figure 3 As shown, Figure 3 (a) can be the initial layout view output by the intersection layout prediction model. After post-processing, the two-dimensional layout view of the vectorized layout can be obtained, that is, Figure 3 (b). Then, after preliminary line projection of the 2D layout view, we can obtain Figure 3 (c) shows the initial layout panorama, which can be further split into straight lines to obtain Figure 3 (d) represents the full set of candidate layout panoramas.

[0086] Step B2: Aggregate the entire candidate layout panorama set along several ray directions using a preset ray projection function to obtain an initial plan view.

[0087] It is easy to understand that considering that a single panorama is easily blocked by walls or furniture, and the layout obtained from the pre-trained model contains many noisy estimates in unseen scenes. Further analysis of multiple noise planes Aggregate along several ray directions to obtain the initial plane map , the process can be expressed as follows:

[0088] (3)

[0089] It should be noted that the basic principle of the above preset ray casting function is to simulate the process of light starting from a certain point (such as the camera position), passing through the space and intersecting with objects in the indoor scene (such as walls, furniture, etc.). By recording the position and properties (such as distance, direction, etc.) of these intersection points, the geometric layout of the indoor scene can be constructed.

[0090] Step S30, performing a preset multi-label graph cut regularization on the initial plane graph to obtain a regularized plane graph;

[0091] Step S40, performing panoramic geometric transformation on the regularized plan view to obtain a target layout view corresponding to the target house.

[0092] It is understandable that due to the geometric complexity and changes in scene conditions, the ground truth data required in the training process is difficult to collect and manually label. And the estimation accuracy of the self-training layout estimation method that relies on the basic truth annotation is still limited. To solve this problem, this embodiment refines the position of each point in the initial plan view by performing a preset multi-label graph cut regularization on the initial plan view to obtain a regularized plan view with a reasonable geometry, which is more similar to the real layout.

[0093] Finally, this embodiment performs a panoramic geometric transformation on the regularized floor plan, converting it into a layout of ceiling heights estimated by a pre-trained intersection layout prediction model, thereby obtaining an accurate target layout view of the target house corresponding to each panorama.

[0094] In this embodiment, in order to avoid the problem of easy estimation failure in the occluded area, the layout of the indoor multi-view panoramic image of the target house is first predicted based on the plane intersection method. At the same time, in order to ensure the applicable scene of the scheme, a post-processing method that is not restricted by Manhattan is adopted, so as to obtain an accurate and widely applicable two-dimensional layout view corresponding to the indoor panoramic image of the house. Since there may be many inaccurate wall-floor intersections in the two-dimensional layout view, this embodiment further projects the two-dimensional layout view to the ground based on the projection function and the current camera extrinsic parameter to obtain a full set of candidate layout panoramas. At the same time, considering the problem that a single panorama is easily blocked by walls or furniture, and the noise estimation that may exist in the full set of candidate layout panoramas in unseen scenes, this embodiment can further aggregate the full set of candidate layout panoramas along several ray directions through a preset ray projection function to obtain an initial plan view. Afterwards, considering the defect of limited accuracy of existing manually labeled ground truth estimation, this embodiment attempts to regularize the initial plan view, and uses multi-label graph cuts to obtain a regularized plan view with reasonable geometric shapes and closer to the real layout of the house. Finally, the regularized plan view is converted into a layout with panoramic geometric relationships to obtain a real and reliable target layout view corresponding to the target house. Therefore, the house space layout estimation method proposed in this embodiment can effectively improve the authenticity and reliability of the house space layout.

[0095] The present embodiment provides a method for estimating the spatial layout of a house based on multi-view panorama and multi-label graph cuts, the method comprising: predicting plane intersections of the indoor multi-view panorama of the target house to obtain an initial layout view; performing post-processing vectorization on the initial layout view to obtain a two-dimensional layout view; obtaining the current camera extrinsic parameter; performing preliminary line projection on the two-dimensional layout view through the projection function and the current camera extrinsic parameter to obtain an initial layout panorama; splitting the initial layout panorama to obtain a full set of candidate layout panoramas. The full set of candidate layout panoramas is aggregated along several ray directions through a preset ray projection function to obtain an initial plan view. The initial plan view is regularized by a preset multi-label graph cut to obtain a regularized plan view; the regularized plan view is subjected to a panoramic geometric transformation to obtain a target layout view corresponding to the target house. In order to avoid the problem of easy estimation failure in the occluded area, the present embodiment first predicts the layout of the indoor multi-view panorama of the target house based on the plane intersection method, and at the same time, in order to ensure the applicable scenario of the scheme, a post-processing method that is not restricted by Manhattan is adopted, so as to obtain an accurate and widely applicable two-dimensional layout view corresponding to the indoor panorama of the house. Since there may be many inaccurate wall-floor intersections in the two-dimensional layout view, this embodiment further projects the two-dimensional layout view to the ground based on the projection function and the current camera extrinsic parameters to obtain a full set of candidate layout panoramas. Taking into account the problem that a single panorama is easily blocked by walls or furniture, and the noise estimation that may exist in the full set of candidate layout panoramas in unseen scenes, this embodiment can further aggregate the full set of candidate layout panoramas along several ray directions through a preset ray projection function to obtain an initial floor plan. Afterwards, considering the defects of the limited accuracy of existing manually labeled ground truth estimation, this embodiment attempts to regularize the initial floor plan and use multi-label graph cuts to obtain a regularized floor plan with a reasonable geometric shape that is closer to the actual layout of the house. Finally, the regularized floor plan is converted into a layout with a panoramic geometric relationship to obtain a real and reliable target layout view corresponding to the target house. Therefore, the house space layout estimation method proposed in this embodiment can effectively improve the authenticity and reliability of the house space layout.

[0096] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction and will not be repeated later.

[0097] It is important to understand that referring to Figure 4 , Figure 4 This is a schematic diagram of the initial plan view effect of the second embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of this application, wherein Figure 4 (a) is a set of multiple noise plane images corresponding to the candidate layout panorama. Figure 4 (b) is the initial plane map (red line), and the green line is the ground truth. Figure 4As shown, the initial floor plan is compared with the ground truth (green line). It still cannot represent the indoor scene well. This embodiment needs further processing to obtain a plan view with more reasonable geometry.

[0098] Therefore, based on the first embodiment, please refer to Figure 5 , Figure 5 This is a flow chart of a second embodiment of a housing space layout estimation method based on multi-view panorama and multi-label graph cuts of the present application. In this embodiment, step S30 includes steps C1 to C3:

[0099] Step C1, segmenting the entire candidate layout panorama to obtain initial candidate walls;

[0100] Step C2, performing a multi-label graph cut on the initial plan view based on the initial candidate wall to obtain a target plan view label;

[0101] It should be noted that the traditional layout estimation method based on multi-view panoramic images relies on manually annotated ground truth data and has limited accuracy. These problems increase the complexity of the reconstruction algorithm and reduce the reliability and authenticity of the generated 3D model. Therefore, compared with the existing methods, this embodiment further improves the estimated unsatisfactory initial floor plan. , the accuracy and effect of house layout estimation can be improved by presetting multi-label segmentation regularization on the initial floor plan.

[0102] It is easy to understand that this embodiment can divide the initial plan into dense two-dimensional points, and try to refine the position of each point by multi-label graph cut through the entire set of candidate layout panoramas. Specifically, we regard the two-dimensional point refinement task as a labeling problem, and regard the entire set of candidate layout panoramas as labels.

[0103] As mentioned above, the candidate layout panorama set estimated by the pre-trained model contains many inaccurate wall-floor intersection lines. From another perspective, the candidate layout panorama set also contains some accurate wall-floor intersection lines. Therefore, the candidate layout panorama set can be a superset of all accurate wall-floor intersection lines with multi-view panoramas. Figure 3 The multiple straight lines of different colors shown in (d) segment the entire candidate layout panorama set, and each straight line of different colors can represent a wall label, thereby obtaining a number of candidate walls corresponding to the entire candidate layout panorama set, namely the above-mentioned initial candidate walls.

[0104] In a feasible implementation manner, in this embodiment, step C2 may include steps C21-C22:

[0105] Step C21, constructing a label allocation energy function corresponding to the initial plan view based on the initial candidate wall;

[0106] Step C22, assigning labels to the initial plan view through the label assignment function to obtain target plan view labels.

[0107] It should be understood that, in this embodiment, wall labels can be assigned to a number of two-dimensional points after the initial plan map is segmented based on the initial candidate wall, that is, each two-dimensional point will have a corresponding candidate wall label, thereby forming a target plan map label corresponding to the initial plan map. Therefore, in order to accurately assign a corresponding wall label to each two-dimensional point after the initial plan map is segmented, in this embodiment, a label assignment energy function can be formulated based on the initial candidate wall to perform label assignment.

[0108] In a feasible implementation manner, in this embodiment, step S20 may include steps C211 to C213:

[0109] Step C211, taking the initial candidate wall as a candidate label, and obtaining the candidate labeling cost corresponding to the initial plan view;

[0110] Step C212, obtaining the adjacent point label energy items corresponding to the initial plane graph;

[0111] Step C213: determining a label allocation energy function corresponding to the initial plane graph according to the candidate label cost and the adjacent point label energy item.

[0112] It is easy to understand that, from the above analysis, the initial candidate walls are the corresponding wall labels after the candidate layout is segmented. In this embodiment, the initial candidate walls can be used as candidate labels to determine the labeling cost corresponding to each two-dimensional point in the initial plane when it is labeled by each candidate label, that is, the candidate labeling cost. The adjacent point label energy item can be the energy item corresponding to adjacent two-dimensional points in the initial plane when the wall labels are not equal to each other.

[0113] Therefore, in order to accurately assign a corresponding wall label to each two-dimensional point after the initial plan view is segmented, this embodiment may formulate a label assignment energy function E as shown below to perform label assignment.

[0114] (4)

[0115] In the formula, Candidate labels Marking 2D points The candidate marking cost at . are adjacent two-dimensional points and Neighboring point label energy terms when labels are not equal to each other.

[0116] Specifically, this embodiment can be a two-dimensional point To label line The distance between them is used to calculate the candidate tag cost, which can be shown as follows:

[0117] (5)

[0118] For the smooth energy , which can be set as a constant , to penalize adjacent 2D points labeled with different candidate labels. It is observed that the wall-floor intersections of artificial indoor scenes are long and straight with relatively few turning points.

[0119] Step C3: performing normal migration on the initial plan view based on the target plan view label to obtain a regularized plan view.

[0120] For ease of understanding, refer to Figure 6 The plane regularization process of this embodiment is explained. Figure 6 This is a schematic diagram of the regularization process of the second embodiment of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of this application, wherein: Figure 6 (a) is the initial plane diagram, Figure 6 (b) is the full set of candidate layout panoramas, Figure 6 (c) is the initial floor plan with labels assigned based on the full set of candidate layout panoramas, Figure 6 (d) is the regularized planar graph.

[0121] like Figure 6 As shown, this embodiment can use the full set of candidate layouts ( Figure 6 (b)) is an unsatisfactory initial plan ( Figure 6 (a)) for each two-dimensional point Assign labels and get Figure 6 (c). Then, by Move to the specified wall tag along the normal of the corresponding line , that is, moving each 2D point to the bottom of the line perpendicular to the assigned wall label, we can finally obtain a plan with reasonable geometry (such as Figure 6 (d)), which is more similar to the real layout (i.e. Figure 6 Finally, by Figure 6 The regularized floor plan shown in (d) is converted into a layout with ceiling heights estimated by the pre-trained intersection layout prediction model, which can obtain the accurate target layout view of the target house corresponding to each panorama.

[0122] In addition, by using the regularization method proposed in this embodiment, the panoramic layouts of different positions estimated by the pre-trained model can be fused into the initial layout, naturally solving the occlusion problem of a single panoramic image. This method does not require ground truth labels, and since there is no assumption that adjacent walls should be perpendicular to each other, this method can also handle non-Manhattan indoor scenes, and has high accuracy and precision for indoor layout reconstruction.

[0123] In summary, in order to further improve the estimated unsatisfactory initial plan, this embodiment proposes a plane regularization method for multi-label graph cuts to make full use of the full set of candidate layout panoramas estimated by the pre-trained model. This embodiment first divides the initial initial plan into dense two-dimensional points, and attempts to refine the position of each point through multi-label graph cuts. Specifically, this embodiment regards the full set of candidate layout panoramas as labels, and refines the two-dimensional points by solving the multi-label graph cut problem. At the same time, a label is assigned to each two-dimensional point by constructing a label assignment energy function, and a candidate wall is assigned to each point of the initial layout by moving each two-dimensional point along the normal of the corresponding line to the specified label, that is, a regularized plan with a reasonable geometry is obtained, which is more similar to the real layout, thereby effectively improving the accuracy of the target layout view containing each panorama. At the same time, the method proposed in this embodiment does not require a basic truth label, and does not assume that adjacent walls should be perpendicular to each other. This method can also handle non-Manhattan indoor scenes, and has high accuracy and precision for indoor layout reconstruction.

[0124] The present embodiment discloses segmenting the entire set of candidate layout panoramas to obtain initial candidate walls; using the initial candidate walls as candidate labels to obtain the candidate labeling costs corresponding to the initial plan; obtaining the adjacent point label energy items corresponding to the initial plan; and determining the label assignment energy function corresponding to the initial plan based on the candidate labeling costs and the adjacent point label energy items. The initial plan is assigned labels by the label assignment function to obtain the target plan label; the normal of the initial plan is migrated based on the target plan label to obtain a regularized plan. In order to further improve the estimated unsatisfactory initial plan, the present embodiment proposes a plane regularization method for multi-label graph cuts to fully utilize the entire set of candidate layout panoramas estimated by the pre-trained model. The present embodiment first segments the initial initial plan into dense two-dimensional points, and attempts to refine the position of each point through multi-label graph cuts. Specifically, the present embodiment regards the entire set of candidate layout panoramas as labels, and refines the two-dimensional points by solving the multi-label graph cut problem. At the same time, a label is assigned to each two-dimensional point by constructing a label assignment energy function, and a candidate wall is assigned to each point of the initial layout by moving each two-dimensional point along the normal of the corresponding line to the specified label, thereby obtaining a regularized plan view with a reasonable geometric shape, which is more similar to the real layout, thereby effectively improving the accuracy of the target layout view containing each panorama. At the same time, the method proposed in this embodiment does not require a ground truth label, and does not assume that adjacent walls should be perpendicular to each other. This method can also handle non-Manhattan indoor scenes, and has high accuracy and precision for indoor layout reconstruction.

[0125] For example, in order to help understand the technical concept or technical principle of the housing space layout estimation method based on multi-view panorama and multi-label graph cut after the present embodiment is combined with the above-mentioned embodiment 1 and embodiment 2, please refer to Figure 7 , Figure 7 The following is a brief flowchart of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts in this application, as follows:

[0126] First, given a set of target houses corresponding to known camera extrinsics The calibrated panorama, that is, the indoor multi-view panorama ,The goal of the proposed method is to estimate the accurate and complete layout of each view At this time, if Figure 7 As shown, we can first use the intersection layout prediction model (Pre-trained Model) pre-trained on other single-view panoramic datasets Estimate each panorama 2D layout, i.e. two-dimensional layout view .

[0127] Then by using the projection function and camera extrinsics The estimated 2D layout Project all lines between the walls and the floor onto the ground to generate a full set of candidate layout panoramas .

[0128] Then, a geometry-aware ray-casting method is applied to estimate The initial floor plan of each panorama of the layout candidate .

[0129] Second, by using the full set of candidate layouts Regularize the initial plane graph and use multi-label graph cuts to obtain an accurate and complete regularized plane graph .

[0130] In this process, the initial floor plan and the entire set of candidate layout panoramas of each panorama are projected into the bird's-eye view (BEV), and the latter layout is segmented into walls and used as candidate labels. At the same time, an energy equation is constructed to assign a label to each 2D point after the initial floor plan segmentation. By moving each 2D point along the normal of the corresponding line to the specified label, a candidate wall is assigned to each point of the initial layout, so that the 2D points in the initial floor plan are refined by solving the multi-label graph cut problem to obtain a regularized floor plan with a reasonable geometry.

[0131] Finally, by regularizing the plane Perform a panoramic geometric transformation (Transform) to convert it into the final layout with panoramic geometric relationship of the ceiling height estimated by the pre-trained model .

[0132] The specific effect of the method proposed in this embodiment can be as follows Figure 8 As shown, Figure 8 Schematic diagram showing the comparison of layout estimation effects of the housing space layout estimation method based on multi-view panorama and multi-label graph cut in this application, wherein Figure 8 (a) is a schematic diagram of the layout estimation of the HorizonNet model. Figure 8 (b) is a schematic diagram of the layout estimation corresponding to the improved method proposed in this application. Figure 8 As shown in Figure 2, the layout estimated by the pre-trained Horizon Net model (a) is affected by unseen scenes (the first row of red lines) and non-Manhattan scenes (the second row of red lines), and is quite different from the actual layout of the house. Figure 8 (b) It can be seen that the house layout estimated by the method proposed in this application is very similar to the ground truth of the house layout (i.e., the green line in the figure). Therefore, the accuracy of the house spatial layout estimation method proposed in this application is significantly improved.

[0133] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the housing space layout estimation method based on multi-view panorama and multi-label graph cuts of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0134] This application also provides a housing space layout estimation device based on multi-view panorama and multi-label graph cuts, please refer to Fig. 9 , Fig. 9 : This is a schematic diagram of the module structure of a housing space layout estimation device based on multi-view panorama and multi-label graph cuts according to an embodiment of the present application. In this embodiment, the housing space layout estimation device based on multi-view panorama and multi-label graph cuts includes:

[0135] The initial layout estimation module T1 is used to perform two-dimensional layout prediction on the indoor multi-view panoramic image of the target house to obtain a two-dimensional layout view;

[0136] A layout projection module T2, used for performing preset ray projection based on the two-dimensional layout view to obtain an initial plan view;

[0137] A layout optimization module T3 is used to perform a preset multi-label graph cut regularization on the initial plan view to obtain a regularized plan view;

[0138] The layout conversion module T4 is used to perform panoramic geometric conversion on the regularized plan view to obtain a target layout view corresponding to the target house.

[0139] As an implementable method, in this embodiment, the initial layout estimation module T1 is also used to predict plane intersections of the indoor multi-view panoramic image of the target house to obtain an initial layout view;

[0140] The initial layout estimation module T1 is further used to perform post-processing vectorization on the initial layout view to obtain a two-dimensional layout view.

[0141] As an implementable method, in this embodiment, the layout projection module T2 is further used to project the two-dimensional layout view onto the ground to obtain a full set of candidate layout panoramas;

[0142] The layout projection module T2 is further used to aggregate the candidate layout panorama set along several ray directions through a preset ray projection function to obtain an initial plan view.

[0143] As an implementable method, in this embodiment, the layout projection module T2 is also used to obtain the current camera external parameters;

[0144] The layout projection module T2 is further used to perform preliminary line projection on the two-dimensional layout view through a projection function and the current camera external parameters to obtain an initial layout panorama;

[0145] The layout projection module T2 is further used to split the initial layout panorama to obtain a complete set of candidate layout panoramas.

[0146] As an implementable method, in this embodiment, the layout optimization module T3 is further used to segment the candidate layout panorama set to obtain initial candidate walls;

[0147] The layout optimization module T3 is further used to perform multi-label graph cutting on the initial plan view based on the initial candidate wall to obtain a target plan view label;

[0148] The layout optimization module T3 is further used to perform normal migration on the initial plan view based on the target plan view label to obtain a regularized plan view.

[0149] As an implementable method, in this embodiment, the layout optimization module T3 is further used to construct a label allocation energy function corresponding to the initial plan view based on the initial candidate wall;

[0150] The layout optimization module T3 is further used to assign labels to the initial plan view through the label assignment function to obtain target plan view labels.

[0151] As an implementable method, in this embodiment, the layout optimization module T3 is further used to use the initial candidate wall as a candidate label to obtain the candidate label cost corresponding to the initial plan view;

[0152] The layout optimization module T3 is also used to obtain the adjacent point label energy items corresponding to the initial plane graph;

[0153] The layout optimization module T3 is further used to determine the label allocation energy function corresponding to the initial plan view according to the candidate label cost and the adjacent point label energy item.

[0154] The housing space layout estimation device based on multi-view panorama and multi-label graph cuts provided by the present application adopts the housing space layout estimation method based on multi-view panorama and multi-label graph cuts in the above-mentioned embodiment, which can solve the technical problem of housing space layout estimation based on multi-view panorama and multi-label graph cuts. Compared with the prior art, the beneficial effects of the housing space layout estimation device based on multi-view panorama and multi-label graph cuts provided by the present application are the same as the beneficial effects of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts provided by the above-mentioned embodiment, and other technical features in the housing space layout estimation device based on multi-view panorama and multi-label graph cuts are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0155] The present application provides a house space layout estimation device based on multi-view panorama and multi-label graph cuts, and the house space layout estimation device based on multi-view panorama and multi-label graph cuts includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the house space layout estimation method based on multi-view panorama and multi-label graph cuts in the above-mentioned embodiment one.

[0156] Reference below Fig.10 , which shows a schematic diagram of the structure of a housing space layout estimation device based on multi-view panorama and multi-label graph cuts suitable for implementing the embodiment of the present application. The housing space layout estimation device based on multi-view panorama and multi-label graph cuts in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The house space layout estimation device based on multi-view panorama and multi-label graph cuts shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0157] like Fig.10As shown, the housing space layout estimation device based on multi-view panorama and multi-label graph cuts may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: ReadOnly Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: RandomAccess Memory) 1004. In the random access memory 1004, various programs and data required for the operation of the housing space layout estimation device based on multi-view panorama and multi-label graph cuts are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the housing space layout estimation device based on multi-view panorama and multi-label graph cuts to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a housing space layout estimation device based on multi-view panorama and multi-label graph cuts with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0158] In particular, according to the embodiments disclosed in the present application, the process described with reference to the flowchart above can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a house space layout estimation program product based on multi-view panorama and multi-label graph cuts, which includes a house space layout estimation program based on multi-view panorama and multi-label graph cuts carried on a computer-readable medium, and the house space layout estimation program based on multi-view panorama and multi-label graph cuts contains program code for executing the method shown in the flowchart. In such an embodiment, the house space layout estimation program based on multi-view panorama and multi-label graph cuts can be downloaded and installed from the network through a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the house space layout estimation program based on multi-view panorama and multi-label graph cuts is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0159] The housing space layout estimation device based on multi-view panorama and multi-label graph cuts provided by the present application adopts the housing space layout estimation method based on multi-view panorama and multi-label graph cuts in the above-mentioned embodiment, which can solve the technical problem of housing space layout estimation based on multi-view panorama and multi-label graph cuts. Compared with the prior art, the beneficial effects of the housing space layout estimation device based on multi-view panorama and multi-label graph cuts provided by the present application are the same as the beneficial effects of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts provided by the above-mentioned embodiment, and the other technical features in the housing space layout estimation device based on multi-view panorama and multi-label graph cuts are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0160] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0161] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0162] The present application provides a storage medium having computer-readable program instructions stored thereon (i.e., a house space layout estimation program based on multi-view panorama and multi-label graph cuts), and the computer-readable program instructions are used to execute the house space layout estimation method based on multi-view panorama and multi-label graph cuts in the above-mentioned embodiment.

[0163] The storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: RandomAccess Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0164] The above-mentioned storage medium can be included in the housing space layout estimation device based on multi-view panorama and multi-label graph cut; or it can exist independently without being assembled into the housing space layout estimation device based on multi-view panorama and multi-label graph cut.

[0165] The above-mentioned storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the house space layout estimation device based on multi-view panorama and multi-label graph cuts, the house space layout estimation device based on multi-view panorama and multi-label graph cuts enables: house space layout estimation based on multi-view panorama and multi-label graph cuts.

[0166] The program code for estimating the spatial layout of a house based on multi-view panorama and multi-label graph cuts for performing the operations of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0167] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and housing space layout estimation program product based on multi-view panorama and multi-label graph cut according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0168] The modules involved in the embodiments of the present application may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0169] The readable storage medium provided by the present application is a storage medium, which stores computer-readable program instructions for executing the above-mentioned housing space layout estimation method based on multi-view panorama and multi-label graph cuts (i.e., housing space layout estimation program based on multi-view panorama and multi-label graph cuts), which can solve the technical problem of improving the authenticity and reliability of housing space layout estimation. Compared with the prior art, the beneficial effects of the storage medium provided by the present application are the same as the beneficial effects of the housing space layout estimation method based on multi-view panorama and multi-label graph cuts provided by the above-mentioned embodiment, and will not be repeated here.

[0170] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A housing space layout estimation method based on multi-view panorama and multi-label graph cut, characterized in that: The method comprises: Perform two-dimensional layout prediction on the indoor multi-view panoramic image of the target house to obtain a two-dimensional layout view; Perform preset ray projection based on the two-dimensional layout view to obtain an initial plan view; Performing a preset multi-label graph cut regularization on the initial plane graph to obtain a regularized plane graph; Performing panoramic geometric transformation on the regularized plan view to obtain a target layout view corresponding to the target house; The step of performing two-dimensional layout prediction on the indoor multi-view panoramic image of the target house to obtain a two-dimensional layout view includes: Using a single-view panoramic image layout prediction model and an indoor multi-view image dataset to generate pre-training of a two-dimensional layout of the intersection of a ground-wall plane and a top-wall plane for each image, obtaining an intersection layout prediction model, and estimating a two-dimensional layout of an indoor multi-view panoramic image of a target house based on the pre-trained intersection layout prediction model to obtain an initial layout view; Post-processing and vectorizing the initial layout view to obtain a two-dimensional layout view; The step of performing preset ray projection based on the two-dimensional layout view to obtain an initial plan view comprises: Projecting the two-dimensional layout view onto the ground to obtain a full set of candidate layout panoramas; The entire candidate layout panorama set is aggregated along several ray directions through a preset ray projection function to obtain an initial plan view.

2. The method according to claim 1, characterized in that The step of projecting the two-dimensional layout view onto the ground to obtain a full set of candidate layout panoramas includes: Get the current camera external parameters; Performing preliminary line projection on the two-dimensional layout view by using a projection function and the current camera extrinsic parameters to obtain an initial layout panorama; The initial layout panorama is split to obtain a complete set of candidate layout panoramas.

3. The method according to claim 2, characterized in that The step of performing preset multi-label graph cut regularization on the initial plane graph to obtain a regularized plane graph comprises: Segmenting the entire candidate layout panorama to obtain initial candidate walls; Performing a multi-label graph cut on the initial plan view based on the initial candidate walls to obtain a target plan view label; The normal of the initial plane map is migrated based on the target plane map label to obtain a regularized plane map.

4. The method according to claim 3, characterized in that The step of performing a multi-label graph cut on the initial plan view based on the initial candidate wall to obtain a target plan view label comprises: Constructing a label allocation energy function corresponding to the initial plane graph based on the initial candidate wall; The label assignment function is used to assign labels to the initial plan view to obtain target plan view labels.

5. The method according to claim 4, characterized in that The step of constructing a label allocation energy function corresponding to the initial plan view based on the initial candidate wall comprises: Taking the initial candidate wall as a candidate label, obtaining the candidate label cost corresponding to the initial plane map; Obtaining adjacent point label energy items corresponding to the initial plane graph; A label allocation energy function corresponding to the initial plane graph is determined according to the candidate label cost and the adjacent point label energy item.

6. A housing space layout estimation device based on multi-view panorama and multi-label graph cut, characterized in that: The device comprises: An initial layout estimation module is used to predict the 2D layout of the indoor multi-view panoramic image of the target house and obtain a 2D layout view; A layout projection module, used for performing preset ray projection based on the two-dimensional layout view to obtain an initial plan view; A layout optimization module, used for performing a preset multi-label graph cut regularization on the initial plan view to obtain a regularized plan view; A layout conversion module, used for performing panoramic geometric conversion on the regularized plan view to obtain a target layout view corresponding to the target house; The initial layout estimation module is further used to use the single-view panoramic image layout prediction model and the indoor multi-view image data set to pre-train the two-dimensional layout of each image based on the intersection of the ground wall plane and the top wall plane, obtain the intersection layout prediction model, and estimate the two-dimensional layout of the indoor multi-view panoramic image of the target house based on the pre-trained intersection layout prediction model to obtain an initial layout view; post-process the initial layout view for vectorization to obtain a two-dimensional layout view; The layout projection module is further used to project the two-dimensional layout view onto the ground to obtain a full set of candidate layout panoramas; and to aggregate the full set of candidate layout panoramas along several ray directions through a preset ray projection function to obtain an initial plan view.

7. A housing space layout estimation device based on multi-view panorama and multi-label graph cut, characterized in that: The device includes: a memory, a processor, and a house space layout estimation program based on multi-view panorama and multi-label graph cuts stored in the memory and executable on the processor, wherein the house space layout estimation program based on multi-view panorama and multi-label graph cuts is configured to implement the steps of the house space layout estimation method based on multi-view panorama and multi-label graph cuts as described in any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores a house space layout estimation program based on multi-view panorama and multi-label graph cuts. When the house space layout estimation program based on multi-view panorama and multi-label graph cuts is executed by a processor, the steps of the house space layout estimation method based on multi-view panorama and multi-label graph cuts as described in any one of claims 1 to 5 are implemented.

Images