House planar type drawing generation method, device, equipment and medium

Abstract

Embodiments of the present application provide a house layout generation method, device and equipment and medium. By obtaining panoramic images and point cloud plan views of each space object in a target house, the panoramic pixel coordinates and three-dimensional point cloud coordinates corresponding to the target space position can be determined; based on the mapping relationship between the panoramic pixel coordinates and the three-dimensional point cloud coordinates, the panoramic pixel coordinates and the three-dimensional point cloud coordinates are mapped to each other, and the positions of walls, doors and windows in the panoramic images and the positions of contour lines in the point cloud plan views are corrected according to the panoramic pixel coordinates and the three-dimensional point cloud coordinates before and after the mapping, which helps to determine accurate panoramic images and point cloud data; and according to the corrected panoramic images and point cloud plan views, the doors and windows in the panoramic images can also be mapped to the point cloud plan views to obtain door outlines and window outlines in the point cloud plan views. The house layout information obtained in this way is richer and more complete.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, and in particular to a method, apparatus, device and medium for generating house floor plans. Background Technology

[0002] In existing floor plan generation solutions, 3D point cloud data and panoramic images of each room in a house are typically collected. The 3D point cloud data of each room is then fused based on the panoramic image, i.e., the point cloud data of adjacent rooms are stitched together to obtain a 3D point cloud model of the entire house. Further, the obtained 3D point cloud model is divided into horizontal sections to obtain a 2D floor plan of the entire house.

[0003] However, in the existing solutions, on the one hand, the floor plan obtained by simply cutting the 3D point cloud model into horizontal sections cannot accurately reflect the details inside the house; on the other hand, since the obtained panoramic image and 3D point cloud data also have certain errors relative to the actual situation of the house, the accuracy of the obtained floor plan information is difficult to guarantee. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for generating floor plans of a house, which corrects the panoramic data and point cloud data corresponding to the walls, doors, and windows of each spatial object in the target house during the process of generating the floor plan of the target house, so as to obtain a floor plan that matches the actual situation of the target house based on the corrected panoramic data and point cloud data.

[0005] This application provides a method for generating a house floor plan, comprising: acquiring a panoramic view and a point cloud plan view corresponding to each spatial object in a target house, wherein the point cloud plan view is generated from point cloud data corresponding to each spatial object, the panoramic view includes walls, doors, and / or windows of each spatial object, and the point cloud plan view includes outlines corresponding to each wall; determining, for the target spatial location in each spatial object, corresponding first panoramic pixel coordinates from the panoramic view and corresponding first three-dimensional point cloud coordinates from the point cloud data; mapping the first panoramic pixel coordinates to second three-dimensional point cloud coordinates in a three-dimensional point cloud coordinate system according to the relative pose relationship between the devices acquiring the panoramic view and the point cloud data; and The first three-dimensional point cloud coordinates are mapped to the second panoramic pixel coordinates in the panoramic pixel coordinate system. Based on the panoramic pixel coordinates and the three-dimensional point cloud coordinates before and after mapping, the position information of the walls, doors, and / or windows of each spatial object in the panoramic image and the outlines in the point cloud planar image are corrected respectively. Based on the corrected panoramic image and the point cloud planar image, the doors and / or windows of each spatial object in the panoramic image are mapped to the corresponding outlines in the point cloud planar image to obtain a point cloud planar image containing door outlines and window outlines. The door outlines and window outlines mapped onto the point cloud planar image are marked, and the marked point cloud planar image is used as the floor plan of the target house for display.

[0006] In one optional embodiment, obtaining panoramic images and point cloud planar images corresponding to each spatial object in the target house includes: obtaining panoramic data and point cloud data corresponding to each spatial object in the target house; performing three-dimensional real-world spatial rendering on the panoramic data to obtain panoramic images corresponding to each spatial object; generating a three-dimensional point cloud model corresponding to the target house based on the point cloud data; and performing planar projection on the three-dimensional point cloud model to obtain point cloud planar images corresponding to each spatial object.

[0007] In one optional embodiment, mapping the first panoramic pixel coordinates to a second three-dimensional point cloud coordinate in a three-dimensional point cloud coordinate system based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data includes: mapping the first panoramic pixel coordinates to a spherical space according to the mapping relationship between panoramic pixel coordinates and spherical coordinates to obtain the corresponding first spherical coordinates; and mapping the first spherical coordinates to the three-dimensional point cloud coordinate system according to the relative pose relationship between the devices acquiring the panoramic image and point cloud data, combined with the mapping relationship between spherical coordinates and three-dimensional point cloud coordinates, to obtain the corresponding second three-dimensional point cloud coordinates.

[0008] In one optional embodiment, mapping the first three-dimensional point cloud coordinates to a second panoramic pixel coordinate in the panoramic pixel coordinate system based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data includes: mapping the first three-dimensional point cloud coordinates to a spherical space based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data, combined with the mapping relationship between spherical coordinates and three-dimensional point cloud coordinates, to obtain the corresponding second spherical coordinates; and mapping the second spherical coordinates to the panoramic space based on the mapping relationship between panoramic pixel coordinates and spherical coordinates, to obtain the corresponding second panoramic pixel coordinates.

[0009] In one optional embodiment, the target spatial location corresponds to at least one first panoramic pixel coordinate and at least one first three-dimensional point cloud coordinate. Based on the panoramic pixel coordinates and three-dimensional point cloud coordinates before and after mapping, the position information of the walls, doors, and / or windows of each spatial object in the panoramic image, as well as the contour lines in the point cloud planar image, are corrected respectively. This includes: determining accurate first panoramic pixel coordinates based on at least one second panoramic pixel coordinate obtained by mapping at least one first three-dimensional point cloud coordinate corresponding to the target spatial location; correcting the position information of the walls, doors, and / or windows of each spatial object in the panoramic image based on the accurate first panoramic pixel coordinates; determining accurate first three-dimensional point cloud coordinates based on at least one second three-dimensional point cloud coordinate obtained by mapping at least one first panoramic pixel coordinate corresponding to the target spatial location; and correcting the contour lines in the point cloud planar image based on the accurate first three-dimensional point cloud coordinates.

[0010] In an optional embodiment, before mapping the doors and / or windows of each spatial object in the panoramic image onto the corresponding contour lines in the point cloud planar image, the method further includes: determining the panoramic pixel coordinates corresponding to the doors and / or windows of each spatial object based on the panoramic image; mapping the panoramic pixel coordinates corresponding to the doors and / or windows of each spatial object onto the three-dimensional point cloud model according to the mapping relationship between the panoramic pixel coordinates and the three-dimensional point cloud coordinates, thereby obtaining the corresponding three-dimensional point cloud coordinates; and performing a planar projection on the three-dimensional point cloud model to map the doors and / or windows of each spatial object in the panoramic image onto the corresponding contour lines in the point cloud planar image.

[0011] In an optional embodiment, the target house includes an open space. Before projecting the three-dimensional point cloud model onto a plane, the method further includes: determining the panoramic pixel coordinates corresponding to the open space in the target house based on the panoramic image; determining the three-dimensional point cloud coordinates corresponding to the open space in the three-dimensional point cloud model based on the mapping relationship between the panoramic pixel coordinates and the three-dimensional point cloud coordinates; correspondingly, the point cloud planar image obtained by projecting the three-dimensional point cloud model onto a plane also includes the outline of the open space corresponding to the open space.

[0012] In an optional embodiment, the panoramic image is editable, and the panoramic image contains target doors and / or target windows whose horizontal widths do not meet the requirements. Before projecting the three-dimensional point cloud model onto a plane, the method further includes: responding to an operation on adjusting the horizontal width of the target doors and / or target windows in the panoramic image, adjusting the horizontal width of the target doors and / or target windows; and mapping the panoramic pixel coordinates corresponding to the adjusted horizontal width to the three-dimensional point cloud model according to the mapping relationship between panoramic pixel coordinates and three-dimensional point cloud coordinates, thereby obtaining the three-dimensional point cloud coordinates corresponding to the target doors and / or target windows.

[0013] In an optional embodiment, the point cloud plan view is editable, and the outlines on the point cloud plan view include target outlines that do not correspond to the wall position. The method further includes: in response to an adjustment operation on the target outlines on the point cloud plan view, adjusting the shape and / or position of the target outlines to make the target outlines correspond to the wall position.

[0014] In one optional embodiment, adjusting the shape and / or position of the target contour line in response to an adjustment operation on the point cloud planar graph includes: determining an initial position corresponding to the selected target contour line on the point cloud planar graph; in response to a movement operation performed on the selected target contour line, obtaining the target position corresponding to the termination of the movement operation, and moving the selected target contour line from the initial position to the target position; and / or in response to a shape adjustment operation performed on the selected target contour line, obtaining the target shape corresponding to the selected target contour line at the end of the shape adjustment operation, as the shape corresponding to the selected target contour line.

[0015] This application embodiment also provides a house floor plan generation device, including: an acquisition module, used to acquire panoramic images and point cloud plan views corresponding to each spatial object in a target house, wherein the point cloud plan view is generated from point cloud data corresponding to each spatial object, the panoramic image includes walls, doors and / or windows of each spatial object, and the point cloud plan view includes outlines corresponding to each wall; a determination module, used to determine the corresponding first panoramic pixel coordinates from the panoramic image and the corresponding first three-dimensional point cloud coordinates from the point cloud data for the target spatial position in each spatial object; and a first mapping module, used to map the first panoramic pixel coordinates to second three-dimensional point cloud coordinates in a three-dimensional point cloud coordinate system according to the relative pose relationship between the devices acquiring the panoramic image and the point cloud data. The system includes a first three-dimensional point cloud coordinate system and a second panoramic pixel coordinate system. A correction module is used to correct the position information of the walls, doors, and / or windows of each spatial object in the panoramic image and the outlines in the point cloud planar image, based on the panoramic pixel coordinates and the three-dimensional point cloud coordinates before and after mapping. A second mapping module is used to map the doors and / or windows of each spatial object in the panoramic image onto the corresponding outlines in the point cloud planar image, based on the corrected panoramic image and the point cloud planar image, to obtain a point cloud planar image containing door and window outlines. A marking module is used to mark the door and window outlines mapped onto the point cloud planar image, and to use the marked point cloud planar image as the floor plan corresponding to the target house for display.

[0016] This application also provides a computer device, including a processor and a memory, which, when the processor executes a computer program, are used to implement any step of the method.

[0017] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps of the method.

[0018] In this embodiment, by acquiring panoramic images and point cloud floor plans corresponding to each spatial object in the target house, the first panoramic pixel coordinates can be determined from the panoramic image for the target spatial location in each spatial object, and the first three-dimensional point cloud coordinates can be determined from the point cloud data that generated the point cloud floor plan. Furthermore, based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data, the first panoramic pixel coordinates are mapped to second three-dimensional point cloud coordinates in the three-dimensional point cloud coordinate system, and the first three-dimensional point cloud coordinates are mapped to second panoramic pixel coordinates in the panoramic pixel coordinate system. Based on this, according to the panoramic pixel coordinates and three-dimensional point cloud coordinates before and after mapping, the position information of walls, doors, and windows in the panoramic image can be corrected, as well as the outline position of walls in the point cloud floor plan. This method helps to determine accurate panoramic images and point cloud data from multiple panoramic images and point cloud data acquired from the same spatial location from multiple angles, resulting in a more accurate floor plan.

[0019] In addition, based on the corrected panoramic image and point cloud plan view, the doors and windows in the panoramic image are mapped onto the corresponding contour lines in the point cloud plan view and marked. The resulting floor plan not only includes the contour lines of the walls of each spatial object, but also the contour lines of the doors and windows, making the floor plan information richer and more complete. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 A flowchart illustrating a method for generating floor plans provided in this application embodiment;

[0022] Figure 2a A schematic diagram of a three-dimensional point cloud model corresponding to a target house provided in an embodiment of this application;

[0023] Figure 2b A point cloud plan view lacking point cloud data is provided as an embodiment of this application;

[0024] Figure 2c A point cloud planar view after supplementing point cloud data is provided in an embodiment of this application;

[0025] Figure 2d A point cloud planar view with outlines is provided for embodiments of this application;

[0026] Figure 2e Another point cloud planar view with outlines provided for embodiments of this application;

[0027] Figure 2f A partial panoramic view with editing functionality is provided as an embodiment of this application;

[0028] Figure 3 This is a schematic diagram of a floor plan generation device provided in an embodiment of this application;

[0029] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] To address the problem of inaccurate house floor plan generation in existing methods, this application provides a house floor plan generation method. This method acquires panoramic and point cloud floor plans corresponding to various spatial objects within a target house. Based on the relative pose relationships between the devices acquiring the panoramic and point cloud data, it maps the corresponding panoramic pixel coordinates and 3D point cloud coordinates of the target spatial locations within the target house. Using the mapped panoramic pixel coordinates and 3D point cloud coordinates, it corrects the positional information of walls, doors, and / or windows in the panoramic image, as well as the outlines of the corresponding walls in the point cloud floor plan. Finally, based on the corrected panoramic image, it maps the doors and / or windows onto the corresponding outlines in the point cloud floor plan, resulting in accurate and comprehensive floor plan information.

[0032] The steps of the house floor plan generation method provided in this application embodiment will be described in detail below with reference to the accompanying drawings. Figure 1 A flowchart of the method for generating floor plans provided in the embodiments of this application is shown below. Figure 1 As shown, the method includes:

[0033] S1. Obtain the panoramic view and point cloud plan view corresponding to each spatial object in the target house. The point cloud plan view is generated from the point cloud data corresponding to each spatial object. The panoramic view includes the walls, doors and / or windows of each spatial object, and the point cloud plan view includes the outline of each wall.

[0034] S2. For the target spatial location in each spatial object, determine the corresponding first panoramic pixel coordinates from the panoramic image and the corresponding first three-dimensional point cloud coordinates from the point cloud data.

[0035] S3. Based on the relative pose relationship between the devices acquiring panoramic images and point cloud data, the first panoramic pixel coordinates are mapped to the second three-dimensional point cloud coordinates in the three-dimensional point cloud coordinate system, and the first three-dimensional point cloud coordinates are mapped to the second panoramic pixel coordinates in the panoramic pixel coordinate system.

[0036] S4. Based on the panoramic pixel coordinates and 3D point cloud coordinates before and after mapping, correct the position information of walls, doors and / or windows of each spatial object in the panoramic image, as well as the outlines in the point cloud planar image.

[0037] S5. Based on the corrected panoramic image and point cloud planar image, map the doors and / or windows of each spatial object in the panoramic image onto the corresponding contour lines in the point cloud planar image to obtain a point cloud planar image containing the door contours and window contours.

[0038] S6. Mark the door and window outlines mapped onto the point cloud plan view, and use the marked point cloud plan view as the floor plan of the target house for display.

[0039] In this application embodiment, the specific method of obtaining the panoramic image and point cloud plan view corresponding to each spatial object in the target house is not limited. In one optional embodiment, a panoramic camera can be used to take pictures of each spatial object in sequence to obtain the panoramic image corresponding to each spatial object, and a laser scanning device can be used to perform laser scanning on each spatial object to obtain the point cloud plan view corresponding to each spatial object. In another optional embodiment, a panoramic camera and a laser scanning device can be used simultaneously to take panoramic pictures and perform laser scanning on each spatial object to obtain the panoramic image and point cloud plan view corresponding to the target house. Further optionally, when using a panoramic camera and a laser scanning device to take panoramic pictures and perform laser scanning on each spatial object at the same time, the panoramic camera and the laser scanning device can perform panoramic pictures and laser scanning on each spatial object according to a preset relative pose relationship. The specific data corresponding to the relative pose relationship is not limited in this application embodiment.

[0040] Based on the above, since there is a relative pose relationship between the panoramic camera and the laser scanning device, there is a corresponding mapping relationship between the panoramic pixel coordinates corresponding to the panoramic image and the three-dimensional point cloud coordinates corresponding to the point cloud planar image. Therefore, for the target spatial location in the target house, in order to obtain its accurate position information in the panoramic image and the point cloud planar image, the first panoramic pixel coordinates corresponding to the target spatial location can be determined from the panoramic image, and the first three-dimensional point cloud coordinates corresponding to the target spatial location can be determined from the point cloud data that generated the point cloud planar image. By mapping the panoramic pixel coordinates of the target spatial location in the panoramic image and the three-dimensional point cloud coordinates in the point cloud data, the first three-dimensional point cloud coordinates are mapped to second panoramic pixel coordinates, and the first panoramic pixel coordinates are also mapped to second three-dimensional point cloud coordinates. This allows for the correction of the position information of the walls, doors, and / or windows of each spatial object in the panoramic image based on the mapped second panoramic pixel coordinates, and the correction of the contour lines in the point cloud planar image based on the mapped second three-dimensional point cloud coordinates.

[0041] Furthermore, based on the corrected panoramic image and point cloud plan view, the doors and / or windows of each spatial object in the panoramic image can be mapped onto the corresponding contour lines in the point cloud plan view to obtain a point cloud plan view containing door and window contours. The door and window contours mapped onto the point cloud plan view are then marked, and the marked point cloud plan view is used as the floor plan corresponding to the target house. In this way, not only can floor plans be automatically generated, improving the efficiency of floor plan generation, but detailed information such as doors and windows in the house can also be mapped onto the floor plan, making the generated floor plan information richer and more complete, facilitating a quick and accurate understanding of the house information through the floor plan.

[0042] In this embodiment, the specific method for generating panoramic images and point cloud planar images of each spatial object is not limited. Optionally, the panoramic device and laser scanning device can directly generate panoramic images and point cloud planar images corresponding to each spatial object during the process of shooting and scanning each spatial object. In another optional embodiment, panoramic data and point cloud data corresponding to each spatial object can be obtained first, and then three-dimensional real-world spatial rendering can be performed on the panoramic data to obtain panoramic images corresponding to each spatial object; and a three-dimensional point cloud model corresponding to the target house can be generated based on the point cloud data; further, the three-dimensional point cloud model can be projected onto a plane to obtain point cloud planar images corresponding to each spatial object. Further optionally, when generating point cloud planar images corresponding to each spatial object, the outline lines of each spatial object on the point cloud planar image can also be generated based on the point cloud data corresponding to the walls of each spatial object. These outline lines are used to identify the house type corresponding to the target house.

[0043] In this embodiment, the specific method of mapping panoramic data and point cloud data is not limited. Optionally, the first panoramic pixel coordinates can be mapped to the second three-dimensional point cloud coordinates, and vice versa, directly based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data. Alternatively, the first panoramic pixel coordinates can be mapped to intermediate coordinates first, and then the intermediate coordinates can be mapped to the second three-dimensional point cloud coordinates, or vice versa, using the relative pose relationship and an intermediate coordinate system. In this embodiment, the mapping between panoramic pixel coordinates and three-dimensional point cloud coordinates using an intermediate coordinate system is used as an example. Here, the specific type of intermediate coordinate system is not limited, nor is the specific method used in the coordinate mapping process. The mapping method used will vary depending on the intermediate coordinate system and the relative pose relationship.

[0044] Optionally, using a spherical coordinate system as the intermediate coordinate system, in the process of mapping the first panoramic pixel coordinates to the second 3D point cloud coordinates, the first panoramic pixel coordinates can be mapped to the spherical space according to the mapping relationship between panoramic pixel coordinates and spherical coordinates to obtain the corresponding first spherical coordinates. Then, according to a preset pose relationship and combined with the mapping relationship between spherical coordinates and 3D point cloud coordinates, the first spherical coordinates can be mapped to the 3D point cloud coordinate system to obtain the corresponding second 3D point cloud coordinates. Correspondingly, in the process of mapping the first 3D point cloud coordinates to the second panoramic pixel coordinates, the first 3D point cloud coordinates can be mapped to the spherical space according to a preset pose relationship and combined with the mapping relationship between spherical coordinates and 3D point cloud coordinates to obtain the corresponding second spherical coordinates. Furthermore, according to the mapping relationship between panoramic pixel coordinates and spherical coordinates, the second spherical coordinates can be mapped to the panoramic space to obtain the corresponding second panoramic pixel coordinates.

[0045] The following example illustrates the coordinate mapping process by mapping the panoramic pixel coordinates and 3D point cloud coordinates corresponding to the outlines of doors and / or windows of various spatial objects.

[0046] 1. Map the panoramic pixel coordinates corresponding to the outlines of the door and window to 3D point cloud coordinates. .

[0047] Optionally, based on the mapping relationship between panoramic pixel coordinates and spherical coordinates, the panoramic pixel coordinates corresponding to the outlines of the door and window can be mapped to spherical space to obtain the corresponding spherical coordinates. Further, based on the relative pose relationship between the panoramic camera and the laser scanning device, and combined with the mapping relationship between spherical coordinates and 3D point cloud coordinates, the spherical coordinates corresponding to the outlines of the door and window can be mapped to the 3D point cloud coordinate system. Optionally, when mapping the panoramic pixel coordinates corresponding to the outlines of the door and window to spherical coordinates, the pixel coordinates at the upper left corner of the panoramic pixel coordinates can be taken as the origin. Assuming the length and width of the panoramic image are H and W respectively, and the pixel coordinates corresponding to each pixel point are Pixel(x, y), then the longitude Lon and latitude Lat corresponding to the spherical coordinates after mapping each panoramic pixel coordinate are as follows:

[0048] Lon = (x / W - 0.5) * 360;

[0049] Lat = (0.5 – y / H) * 180;

[0050] Furthermore, establishing the origin O1(0, 0, 0) of the spherical coordinate system, and assuming the radius of the spherical coordinate system is R, the spherical coordinates (X, Y, Z) of each panoramic pixel after mapping are as follows:

[0051] X = R * cos(Lon) * cos(Lat);

[0052] Y = R * sin(Lat);

[0053] Z = R * sin(Lon) * cos(Lat);

[0054] Furthermore, when mapping from the spherical coordinate system to the three-dimensional point cloud coordinate system, the mapping can be performed using the spherical coordinates P = Q*(X+x0, Y+y0, Z+z0) corresponding to the door and window after rotation and translation transformations when the laser scanning device scans them. Here, x0, y0, and z0 are the origin O2(x0, y0, z0) of the three-dimensional point cloud coordinate system, RotationY is the rotation angle of the laser scanning device around the Y-axis of the world coordinate system, and Q is a quaternion obtained through the system function Quaternion.AngleAxis(RotationY, Y) to represent the rotation and displacement of the laser scanning device.

[0055] Optionally, when determining the three-dimensional point cloud coordinates corresponding to the door and window outlines, the three-dimensional point cloud coordinates corresponding to a specified spatial location in each spatial object can be used as reference coordinates. Based on the relationship between spherical coordinates and reference coordinates, the three-dimensional point cloud coordinates corresponding to the door and window outlines can be determined respectively. In this embodiment, the specific location of the specified spatial location within the target house is not limited. Optionally, the three-dimensional point cloud coordinates of the corresponding wall outline in each spatial object can be used as reference coordinates. Further, the reference coordinates are mapped to the corresponding reference spherical coordinate set, and the focus of the ray from the origin O1 to point P in the spherical coordinate system with the reference spherical coordinates is determined. The three-dimensional point cloud coordinates corresponding to this focus are used as the three-dimensional point cloud coordinates corresponding to the door or window outline. Alternatively, the spherical coordinates corresponding to a known object in the target house can be used as reference spherical coordinates. For example, if the spherical coordinates corresponding to the ground are used as reference spherical coordinates, the focus of the ray from the origin O1 to point P with the reference spherical coordinates, i.e., the focus with the plane containing the ground, can be determined. The three-dimensional point cloud coordinates corresponding to this focus are used as the three-dimensional point cloud coordinates corresponding to the door or window outline.

[0056] 2. Map the 3D point cloud coordinates corresponding to the outlines of the door and window to panoramic pixel coordinates. .

[0057] Alternatively, the spherical coordinates P(X, Y, Z) corresponding to the 3D point cloud coordinates (x0, y0, z0) can be determined using the following formula:

[0058] P = -Q(X-x0, Y-y0, Z-z0);

[0059] Furthermore, the panoramic pixel coordinates (W, H) corresponding to the spherical coordinates P(X, Y, Z) can be determined using the following formula:

[0060] Lon = Atan2(Z, X);

[0061] Lat = Asin(Y / R);

[0062] X = (Lon / 360 + 0.5) * W;

[0063] Y = (0.5 – Lat / 180) * H;

[0064] The methods for establishing the spherical coordinate system and the panoramic pixel coordinate system, as well as for obtaining the rotation and displacement of the laser scanning device, can be found in the above embodiments and will not be repeated here. Based on the above, the mutual mapping between the panoramic pixel coordinates and the 3D point cloud coordinates corresponding to the target spatial position in each spatial object can be achieved.

[0065] Because panoramic cameras and laser scanning cameras may take multiple shots and scans of different spatial objects, or may take shots and scans of the same target spatial location from different points, the same target spatial location may correspond to at least one first panoramic pixel coordinate and at least one first three-dimensional point cloud coordinate. Accordingly, after implementing coordinate mapping through the above embodiments, at least one second panoramic pixel coordinate and at least one second three-dimensional point cloud coordinate can be obtained. Based on this, when correcting the position information of the walls, doors, and / or windows of each spatial object in the panoramic image and the contour lines in the point cloud planar image according to the second panoramic pixel coordinates and the second three-dimensional point cloud coordinates, the accurate first panoramic pixel coordinates can be determined by mapping at least one second panoramic pixel coordinate corresponding to at least one first three-dimensional point cloud coordinate of each spatial object, so as to correct the position information of the walls, doors, and / or windows of each spatial object in the panoramic image according to the accurate first panoramic pixel coordinates; and the accurate first three-dimensional point cloud coordinates can be determined by mapping at least one second three-dimensional point cloud coordinate corresponding to at least one first panoramic pixel coordinate of each spatial object in the point cloud planar image, so as to correct the contour lines of each spatial object in the point cloud planar image according to the accurate first three-dimensional point cloud coordinates.

[0066] Optionally, when correcting the position information of walls, doors, and / or windows of each spatial object in the panoramic image and the outlines in the point cloud planar image, if there is a panoramic pixel coordinate in the first panoramic pixel coordinate that is the same as that in the second panoramic pixel coordinate, then the first panoramic pixel coordinate is taken as the accurate panoramic pixel coordinate; if there is a three-dimensional point cloud coordinate in the first three-dimensional point cloud coordinate that is the same as that in the second three-dimensional point cloud coordinate, then the first three-dimensional point cloud coordinate is taken as the accurate three-dimensional point cloud coordinate. This is not limited to this; for example, if there is a first panoramic pixel coordinate in the first panoramic pixel coordinate and the second panoramic pixel coordinate with an error within a preset difference range, then the first panoramic pixel coordinate is taken as the accurate panoramic pixel coordinate; if there is a first three-dimensional point cloud coordinate in the first three-dimensional point cloud coordinate and the second three-dimensional point cloud coordinate with an error within a preset difference range, then the first three-dimensional point cloud coordinate is taken as the accurate three-dimensional point cloud coordinate. Based on this, the panoramic image can be updated according to the accurate first panoramic pixel coordinate, and the point cloud data can be updated according to the accurate first three-dimensional point cloud coordinate, so as to generate a floor plan corresponding to the target house based on the updated panoramic image and point cloud data.

[0067] In this embodiment, before generating the floor plan of the target house, to obtain richer floor plan information, this embodiment can also determine the panoramic pixel coordinates corresponding to the doors and / or windows of each spatial object based on the panoramic image. Then, according to the mapping relationship between panoramic pixel coordinates and 3D point cloud coordinates, the panoramic pixel coordinates corresponding to the doors and / or windows of the spatial objects are mapped to the 3D point cloud model to obtain the corresponding 3D point cloud coordinates. Further, a planar projection is performed on the 3D point cloud model to map the doors and / or windows of each spatial object in the panoramic image onto the corresponding contour lines in the point cloud planar image, obtaining the corresponding point cloud planar image. The point cloud planar image obtained in this way includes not only the wall contour lines corresponding to the walls, but also the door contour lines corresponding to the doors and the window contour lines corresponding to the windows. Based on these contour lines, the floor plan information of the target house can be clearly obtained. For the methods of determining the panoramic pixel coordinates and 3D point cloud coordinates corresponding to the doors and / or windows of each spatial object, and the process of converting panoramic pixel coordinates to 3D point cloud coordinates, please refer to the above embodiments, which will not be repeated here.

[0068] In this embodiment, the target house may further include an open space. To better understand the floor plan information of the target house, optionally, before projecting the 3D point cloud model onto a plane, the panoramic pixel coordinates corresponding to the open space in the target house can be determined based on the panoramic image, and the 3D point cloud coordinates corresponding to the open space in the 3D point cloud model can be determined based on the mapping relationship between the panoramic pixel coordinates and the 3D point cloud coordinates. Based on this, after projecting the 3D point cloud model onto a plane, the resulting point cloud planar image also includes the outline of the open space corresponding to the open space. Optionally, the projection area corresponding to the 3D point cloud coordinates of the target open space on the point cloud planar image can be determined first, and then the outline of the projection area can be used as the outline of the open space.

[0069] In this embodiment, since the target house may include objects such as furniture, the laser scanning device may encounter blind spots in the point cloud data when scanning the target house, such as failing to scan walls, or misidentifying scanned furniture or other objects as walls. Therefore, the point cloud data acquired by the laser scanning device may have issues such as missing, redundant, or incorrect data. Furthermore, the panoramic image acquired by the panoramic camera may also contain errors. For example, wall lines, door frames, and window frames of spatial objects are straight lines, but are rendered as curves in the panoramic image. To obtain accurate floor plan information before generating the floor plan, in this embodiment, in addition to correcting the acquired panoramic image and point cloud data by mapping panoramic pixel coordinates to 3D point cloud coordinates as described in the above embodiments, this embodiment can also correct the accuracy of the panoramic image and point cloud data in the following way, so as to obtain a floor plan that matches the actual information of the target house based on the post-corrected panoramic image and point cloud data.

[0070] In the target building, since doors, windows, and other objects are installed within the walls of the spatial objects, if the point cloud data for different locations is not differentiated, when projecting the 3D point cloud model onto a plane, the point cloud data corresponding to doors and windows will be projected onto the point cloud planar view along with the point cloud data corresponding to the walls. Only the outlines of the walls will be visible on the point cloud planar view; the outlines of doors and windows cannot be identified from these outlines. Therefore, to distinguish the outlines of doors and windows on the wall outlines, the outlines of doors and windows within the spatial objects need to be marked separately on the point cloud planar view. Optionally, the panoramic pixel coordinates corresponding to the door and window in the spatial object can be determined from the panoramic image. Then, based on the mapping relationship between the panoramic pixel coordinates and the three-dimensional point cloud coordinates, the point cloud data corresponding to the door and window in the spatial object in the three-dimensional point cloud model can be determined. The corresponding identification information is marked on the point cloud data corresponding to the door and window. After the three-dimensional point cloud model is projected onto a plane, the outlines of the door and window are marked on the corresponding point cloud planar map according to the identification information corresponding to the door and window.

[0071] In this embodiment of the application, to correct the panoramic image, the panoramic image provided in this embodiment supports editing functions. Workers can select and adjust any non-compliant wall lines, door frames, or window frames in the panoramic image to obtain a compliant panoramic image. Therefore, if there are target doors and / or windows in the panoramic image with non-compliant horizontal widths, workers can select the target doors and / or windows in the panoramic image and perform horizontal width adjustment operations on them. Figure 2eAs shown, staff can also manually adjust the outlines of doors and windows. Since the projected outlines of doors and windows on the point cloud map correspond to line segments, the line segment corresponding to the door outline should correspond to the horizontal width of the door, and the line segment corresponding to the window outline should correspond to the horizontal width of the window. Therefore, the outlines of doors and windows can be adjusted by adjusting the horizontal width of the doors and windows in the panoramic view.

[0072] Furthermore, in response to an adjustment operation on the horizontal width of the target door and / or target window in the panoramic image, the corresponding 3D point cloud coordinates in the 3D point cloud model can be updated synchronously based on the panoramic pixel coordinates corresponding to the adjusted target door and / or target window, so that after the 3D point cloud model is projected onto a plane, the outline of the target door in the point cloud plane image corresponds to the horizontal width of the target door in the panoramic image, and / or the outline of the target window corresponds to the horizontal width of the target window in the panoramic image.

[0073] It should be noted that the embodiments of this application are not limited to the above-described method when determining the spatial positions of doors and windows in each spatial object. Since doors and windows are installed on walls in each spatial object, the spatial coordinates corresponding to any wall in the same coordinate system include the spatial coordinates corresponding to the doors and / or windows installed on them. Based on this, the embodiments of this application can also map the panoramic pixel coordinates corresponding to walls, doors, and windows in each spatial object to spherical coordinates, and map the 3D point cloud coordinates of the corresponding walls in the 3D point cloud model to spherical coordinates in the same spherical coordinate system. Furthermore, based on the above two mapping results, the panoramic pixel coordinates and 3D point cloud coordinates corresponding to the target walls in the spherical coordinate system can be determined. Then, according to the relationship between the doors and windows and the walls, the 3D point cloud coordinates corresponding to the target doors and / or windows on the target walls can be determined. Subsequently, after performing planar projection on the 3D point cloud model, the door outlines and / or window outlines corresponding to the target doors and / or windows on the point cloud planar map are obtained.

[0074] However, floor plans obtained solely through the above methods may contain incomplete information. For example, when scanning various spatial objects, the laser scanning device may fail to detect parts of the walls obscured by furniture such as tables, chairs, wardrobes, and beds. Consequently, when generating a 3D point cloud model based on the point cloud data, there may be no corresponding point cloud data for the spatial location corresponding to this part of the wall. Alternatively, the point cloud data corresponding to the scanned furniture such as tables, chairs, wardrobes, and beds may be identified as the point cloud data corresponding to this part of the wall, leading to errors in the point cloud data and an inaccurate 3D point cloud model. Therefore, the generated 3D point cloud model and the projected point cloud plan are also inaccurate.

[0075] Figure 2a A 3D point cloud model obtained by scanning various spatial objects in a building using a laser scanning device, such as... Figure 2a As shown, a 3D point cloud model can identify the overall outline of the walls of each spatial object, but the actual information of each spatial object may not be fully represented in the 3D point cloud model. Therefore, the point cloud plan view obtained by projecting from the 3D point cloud model may not match the actual floor plan structure corresponding to each spatial object. For example, point cloud data that should be projected onto the point cloud plan view in the 3D point cloud model may not be projected onto the point cloud plan view; another example is that spatial locations without point cloud data in the 3D point cloud model may have corresponding projection areas on the point cloud plan view; yet another example is that although the point cloud data corresponding to the wall in the 3D point cloud model is projected onto the point cloud plan view, its corresponding outline does not correspond to the actual position of the wall, and so on.

[0076] In response to potential errors, the following embodiments will describe the correction methods for erroneous point clouds under different conditions in conjunction with the accompanying drawings.

[0077] Scenario 1: Projection information of point cloud data is "missing" on the point cloud plan view .

[0078] Figure 2b To obtain a planar view of the point cloud obtained by projecting a 3D point cloud model onto a plane, such as... Figure 2b As shown, the point cloud plan view contains at least one missing section, which does not meet the requirements of the house's layout and type. Figure 2bTaking the missing portion of the symmetrical rectangle as an example, this could be due to walls being blocked by other objects and not being scanned by the laser scanning device. Therefore, when generating a 3D point cloud model from the scanned point cloud data, a "gap" phenomenon occurs, and consequently, the point cloud planar diagram projected from this 3D point cloud model also shows a "gap." To address this situation, in this embodiment, before generating the outline of each spatial object on the point cloud planar diagram based on the point cloud data corresponding to the walls in each spatial object, the panoramic pixel coordinates corresponding to the walls of each spatial object can be determined based on the panoramic image. Furthermore, based on the mapping relationship between the panoramic pixel coordinates and the 3D point cloud coordinates, the corresponding 3D point cloud coordinates in the 3D point cloud model can be determined, and then it can be determined whether there is corresponding point cloud data at the spatial location corresponding to the 3D point cloud coordinates. If there is a first spatial location in the 3D point cloud model that lacks point cloud data corresponding to the 3D point cloud coordinates, then point cloud data is added at the corresponding first spatial location in the 3D point cloud model, and the added point cloud data is projected onto the point cloud planar diagram to correct the wall outline in the point cloud planar diagram. Furthermore, other "gap" parts corresponding to each spatial object in the 3D point cloud model can be corrected, and point cloud data can be added to the corresponding spatial locations in the 3D point cloud model. Based on the 3D point cloud model after adding point cloud data, a point cloud plan that matches the actual floor plan information of each spatial object can be projected.

[0079] Figure 2c This is a planar view of the point cloud obtained by projecting the 3D point cloud model after supplementing the point cloud data, such as... Figure 2c As shown, after supplementing the 3D point cloud model with point cloud data, the resulting point cloud planar graphic projected from the 3D point cloud model conforms to the actual apartment layout requirements. Based on this, the outlines of the walls of each spatial object can be generated on this point cloud planar graphic. The point cloud planar graphic after generating the outlines is shown below. Figure 2d As shown, the layout of the target house is clearly demonstrated by the outline of the walls of each spatial object.

[0080] In this embodiment, the point cloud plan view supports editing. If there are "missing" outlines of the corresponding walls in the point cloud plan view, the operator can add outlines to the missing outlines on the point cloud plan view. Optionally, the operator can determine the target location of the actual outline in the point cloud plan view based on the panoramic view, select the target location in the point cloud plan view, and add outlines to the target location. Further, in response to the outline addition operation performed on the target location, an outline can be added to the corresponding target location on the point cloud plan view. In this embodiment, the specific method of adding outlines at the target location is not limited. Optionally, in response to the target location selected by the operator and the operation of adding outlines to the target location, the outline can be directly displayed on the point cloud plan view to supplement the missing outlines; or, the operator can select the outline component from the toolbar for editing the point cloud plan view and drag the outline component to the target location on the point cloud plan view to supplement the missing outlines.

[0081] In this embodiment, the method of automatically adding new contour lines is not limited to the initial position of the new contour line on the point cloud planar map. The initial position can be any position within a preset range from the target position, or it can be directly displayed at the target position. If the initial position is not at the target position, the operator can further perform movement and shape adjustment operations on the new contour line to move it to the target position and connect it with the end of the existing contour line. Furthermore, when the operator selects a contour line component to add a contour line, the method of adding the contour line component is not limited. Optionally, the operator can first drag the selected contour line component to any position on the point cloud planar map, and then move and adjust the shape of the dragged contour line component to add it to the target position and connect it with the end of the existing contour line. Further optionally, in order to quickly supplement missing contour lines, this embodiment provides a contour line endpoint snapping function, that is, when the operator moves the contour line to a preset range at the target position, the two ends of the new contour line can be automatically connected to the ends of the existing contour line, which helps to improve the speed and accuracy of contour line addition. For details on how to perform movement-level shape adjustments on the outline, please refer to the following examples, which will not be described in detail here.

[0082] Scenario 2: There is "redundant" projection information on the point cloud planar map. .

[0083] according to Figure 2d As can be seen, the target house comprises three spatial objects, namely... Figure 2dThe rectangles in the upper middle section correspond to spatial objects, and the rectangles on the left and right sides of the lower section correspond to spatial objects respectively. Because laser scanning equipment scans different spatial objects or the same spatial location from different points, and because the point cloud data acquired by the laser scanning equipment at different points for the same spatial location may contain slight errors, this can lead to inaccurate outlines projected onto the point cloud plan view from the point cloud data corresponding to the walls between adjacent spatial objects. For example, as... Figure 2d As shown, the upper rectangular space object and the two lower rectangular space objects are adjacent space objects, and they should be separated by only one wall.

[0084] However, in Figure 2d As shown in the point cloud planar image, two contour lines separate the upper rectangular spatial object from the two lower rectangular spatial objects, indicating a separation between them, which does not meet the actual requirements. Therefore, after projecting the 3D point cloud model onto a plane, we can first determine the target projection contours of each spatial object on the point cloud planar image, and then determine the relative positional relationship between each spatial object based on the panoramic image. Based on this relative positional relationship, we can determine whether the target projection contours of each spatial object on the point cloud planar image are consistent with the actual house layout requirements. If they are inconsistent, we can adjust the position of the target projection contours of each spatial object on the point cloud planar image according to the relative positional relationship between them, aligning the edges of the target projection contours of each spatial object to obtain the contour lines corresponding to the actual wall positions.

[0085] In this embodiment, the specific method of adjusting the position of the target projection contour on the point cloud plan based on the relative positional relationship of each spatial object is not limited. Optionally, the position of each target projection contour on the point cloud plan can be automatically adjusted according to the relative positional relationship of each spatial object when projecting the three-dimensional point cloud model. In another optional embodiment, no adjustment is made when projecting the three-dimensional point cloud model. The operator can manually adjust the contour lines on the point cloud plan that do not correspond to the wall position to obtain the contour lines that correspond to the wall position of each spatial object. Figure 2f The point cloud planar view after adjusting the outline position, such as Figure 2f As shown, after the adjustment, the wall used to separate the upper rectangular space object from the two lower rectangular space objects corresponds to only one or two outline lines, which meets the actual requirements.

[0086] Because laser scanning equipment may introduce errors when scanning various spatial objects, the 3D point cloud model generated from the acquired point cloud data may contain instances where non-wall locations within the spatial objects are identified as walls. This means that the point cloud planar image obtained by projecting the 3D point cloud model may contain "redundant" outlines. Therefore, in addition to supplementing the point cloud data, this embodiment of the application also provides a method to correct the point cloud data by deleting the corresponding point cloud data in the 3D point cloud model when non-wall locations within the spatial objects are identified as walls. Optionally, the panoramic pixel coordinates corresponding to non-wall locations within each spatial object can be determined based on the panoramic image. Furthermore, the corresponding 3D point cloud coordinates in the 3D point cloud model can be determined based on the mapping relationship between the panoramic pixel coordinates and the 3D point cloud coordinates. This allows for the determination of whether point cloud data corresponding to a wall shape exists at the second spatial location corresponding to the 3D point cloud coordinates in the 3D point cloud model. If so, the corresponding point cloud data is deleted to obtain point cloud data that matches the actual floor plan information. In this embodiment, the specific method of deleting the point cloud shape at the second spatial location is not limited. Optionally, the three-dimensional point cloud coordinates corresponding to the second spatial location can be directly deleted, or the three-dimensional point cloud coordinates that originally corresponded to the wall shape at the second spatial location can be adjusted to the three-dimensional point cloud coordinates that correspond to the ground shape to correct the point cloud data.

[0087] In this embodiment, the timing of deleting point cloud data at the second spatial location is not limited. Optionally, the deletion action can be performed before or during the planar projection of the 3D point cloud model. In this way, the contour line obtained on the projected point cloud planar diagram is the corrected contour line. In another optional embodiment, the point cloud data corresponding to the second spatial location can also be deleted after the planar projection of the 3D point cloud model. If the second spatial location is determined, the point cloud data at the second spatial location and the corresponding contour line on the point cloud planar diagram can also be deleted. Regarding the deletion of point cloud data after planar projection of a 3D point cloud model, this application embodiment does not limit the specific method of performing the deletion action. Optionally, when the second spatial position is determined by the above method, the point cloud data at the second spatial position in the 3D point cloud model can be automatically deleted, and the corresponding contour line can be deleted on the point cloud planar diagram based on the projection relationship of the 3D point cloud coordinates. In another optional embodiment, the operator can also determine the target position on the point cloud planar diagram where the contour line should not exist based on the panoramic view, and delete the contour line corresponding to the target position on the point cloud planar diagram by manual deletion. Based on this, when the wheel line corresponding to the target position on the point cloud planar diagram is deleted, the point cloud data at the corresponding spatial position in the 3D point cloud model is automatically deleted based on the projection relationship of the 3D point cloud coordinates.

[0088] Scenario 3: The point cloud planar plot contains "incorrect" projection information. .

[0089] In this embodiment, "erroneous" projection information refers to the situation where point cloud data corresponding to walls, doors, windows, and other objects in various spatial objects have been scanned, but the 3D point cloud model contains point cloud data that is inconsistent with the actual shape of the walls, doors, windows, and other objects in the spatial objects, and / or, in the point cloud planar diagram obtained after projecting the 3D point cloud model onto a plane, there is projection information that is inconsistent with the projected shape of the walls, doors, windows, and other objects in the spatial objects. For example, the walls in the spatial objects are straight, but the point cloud shape corresponding to the walls in the 3D point cloud model is curved. In this case, it is necessary to adjust the contour lines that do not correspond to the wall positions.

[0090] In one optional embodiment, a worker can select a target contour line that does not correspond to the wall position from the point cloud plan view and move it. When a target contour line is selected on the point cloud plan view, the initial position of the selected target contour line on the point cloud plan view can be determined. Furthermore, when a movement operation is performed on the selected target contour line, the target position corresponding to the termination of the movement operation can be obtained, and the selected target contour line can be moved from the initial position to the target position to adjust the position of the target contour line. Herein, the movement operation refers to moving the point cloud data corresponding to at least one end of the contour line and the point cloud data between the two ends from the initial position to the target position.

[0091] In another optional embodiment, the operator can also adjust the shape of the target contour line whose point cloud shape does not meet the requirements. In response to the operation of shape adjustment on any part of the selected target contour line, the operator can obtain the initial shape of the target contour line corresponding to the adjusted part, and the target shape of the target contour line corresponding to the adjusted part when the adjustment operation ends. The target contour line of the adjusted part is adjusted from the initial shape to the target shape, and the overall shape of the selected target contour line after adjustment is taken as the shape corresponding to the selected target contour line. Here, the shape adjustment operation means that the point cloud data corresponding to the two ends of the contour line is not moved, but the point cloud data between the two ends of the contour line is moved. For example, the originally curved contour line is adjusted to a straight contour line, but it is not limited to this.

[0092] Through the above methods, erroneous point cloud data and incorrect outlines on the point cloud plan can be corrected before, during, or after obtaining the point cloud plan, resulting in more accurate floor plan information. Of course, to more clearly and intuitively understand the layout of the target house and the function of each spatial object from the floor plan, this application embodiment also supports marking the types of each spatial object in the target house on the point cloud plan, thereby obtaining a more complete and richer floor plan. Optionally, when the point cloud plan is obtained, the types of each spatial object in the target house can be determined based on the panoramic view, and the corresponding projection areas of each spatial object can be marked with type in the point cloud plan. In this application embodiment, the specific method of marking the projection areas of each spatial object with type in the point cloud plan is not limited. Optionally, the panoramic pixel coordinates and the type of each spatial object can be determined from the panoramic view; further, based on the mapping relationship between the panoramic pixel coordinates and the three-dimensional point cloud coordinates and the projection relationship of the three-dimensional point cloud coordinates, the projection area corresponding to each spatial object is determined in the point cloud plan, and the corresponding type is automatically marked in each projection area. In another alternative embodiment, staff can also directly determine the type of each spatial object based on the panoramic image, and then, based on the projection relationship, directly determine the projection area corresponding to each type of spatial object on the point cloud plan view. Based on this, staff can perform a marking operation on each area to mark the type of each spatial object on the point cloud plan view. Furthermore, in response to the marking operation performed on each projection area on the point cloud plan view, the type of each spatial object can be marked on the point cloud plan view.

[0093] Based on the above, once accurate point cloud data is obtained, the updated point cloud data can be input into the model making terminal, so that the model making terminal can create a 3D model and a floor plan model of the target house based on the updated point cloud data, and apply it to online house rental, sales, home decoration and other scenarios.

[0094] In this embodiment, by acquiring panoramic images and point cloud floor plans corresponding to each spatial object in the target house, the first panoramic pixel coordinates can be determined from the panoramic image for the target spatial location in each spatial object, and the first three-dimensional point cloud coordinates can be determined from the point cloud data that generated the point cloud floor plan. Furthermore, based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data, the first panoramic pixel coordinates are mapped to second three-dimensional point cloud coordinates in the three-dimensional point cloud coordinate system, and the first three-dimensional point cloud coordinates are mapped to second panoramic pixel coordinates in the panoramic pixel coordinate system. Based on this, according to the panoramic pixel coordinates and three-dimensional point cloud coordinates before and after mapping, the position information of walls, doors, and windows in the panoramic image can be corrected, as well as the outline position of walls in the point cloud floor plan. This method helps to determine accurate panoramic images and point cloud data from multiple panoramic images and point cloud data acquired from the same spatial location from multiple angles, resulting in a more accurate floor plan. In addition, based on the corrected panoramic image and point cloud plan view, the doors and windows in the panoramic image are mapped onto the corresponding contour lines in the point cloud plan view and marked. The resulting floor plan not only includes the contour lines of the walls of each spatial object, but also the contour lines of the doors and windows, making the floor plan information richer and more complete.

[0095] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps S1 to S6 can be device A; or the execution subject of step S1 can be device A, and the execution subject of steps S2 and S6 can be device B; and so on.

[0096] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as S1, S2, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0097] Based on the above, this application also provides a floor plan generation device, which can be implemented as a virtual device in a Communication Control Unit (CCU), such as an application program. Figure 3As shown, the floor plan generation device includes: an acquisition module 301, a determination module 302, a first mapping module 303, a correction module 304, a second mapping module 305, and a marking module 306; wherein, the acquisition module 301 is used to acquire panoramic images and point cloud floor plans corresponding to each spatial object in the target house. The point cloud floor plan is generated from the point cloud data corresponding to each spatial object. The panoramic image includes the walls, doors, and / or windows of each spatial object, and the point cloud floor plan includes the outlines corresponding to each wall; the determination module 302 is used to determine the corresponding first panoramic pixel coordinates from the panoramic image and the corresponding first three-dimensional point cloud coordinates from the point cloud data for the target spatial position in each spatial object; the first mapping module 303 is used to map the first panoramic pixels according to the relative pose relationship between the devices acquiring the panoramic image and the point cloud data. The coordinates are mapped to the second three-dimensional point cloud coordinates in the three-dimensional point cloud coordinate system, and the first three-dimensional point cloud coordinates are mapped to the second panoramic pixel coordinates in the panoramic pixel coordinate system; the correction module 304 is used to correct the position information of the walls, doors and / or windows of each spatial object in the panoramic image and the outlines in the point cloud planar image according to the panoramic pixel coordinates and three-dimensional point cloud coordinates before and after mapping; the second mapping module 305 is used to map the doors and / or windows of each spatial object in the panoramic image to the corresponding outlines in the point cloud planar image according to the corrected panoramic image and point cloud planar image, so as to obtain a point cloud planar image containing the door outlines and window outlines; the marking module 306 is used to mark the door outlines and window outlines mapped to the point cloud planar image, and use the marked point cloud planar image as the floor plan of the target house for display.

[0098] In an optional embodiment, when the acquisition module 301 acquires the panoramic view and point cloud plan view corresponding to each spatial object in the target house, it is used to: acquire the panoramic data and point cloud data corresponding to each spatial object in the target house; perform three-dimensional real-world spatial rendering on the panoramic data to obtain the panoramic view corresponding to each spatial object; and generate a three-dimensional point cloud model corresponding to the target house based on the point cloud data; and perform planar projection on the three-dimensional point cloud model to obtain the point cloud plan view corresponding to each spatial object.

[0099] In an optional embodiment, when the first mapping module 303 maps the first panoramic pixel coordinates to the second three-dimensional point cloud coordinates in the three-dimensional point cloud coordinate system according to the relative pose relationship between the devices acquiring panoramic images and point cloud data, it is used to: map the first panoramic pixel coordinates to the spherical space according to the mapping relationship between panoramic pixel coordinates and spherical coordinates to obtain the corresponding first spherical coordinates; and map the first spherical coordinates to the three-dimensional point cloud coordinate system according to the relative pose relationship between the devices acquiring panoramic images and point cloud data, combined with the mapping relationship between spherical coordinates and three-dimensional point cloud coordinates, to obtain the corresponding second three-dimensional point cloud coordinates.

[0100] In an optional embodiment, when the first mapping module 303 maps the first three-dimensional point cloud coordinates to the second panoramic pixel coordinates in the panoramic pixel coordinate system based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data, it is used to: map the first three-dimensional point cloud coordinates to a spherical space based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data, combined with the mapping relationship between spherical coordinates and three-dimensional point cloud coordinates, to obtain the corresponding second spherical coordinates; and map the second spherical coordinates to the panoramic space based on the mapping relationship between panoramic pixel coordinates and spherical coordinates, to obtain the corresponding second panoramic pixel coordinates.

[0101] In an optional embodiment, the target spatial location corresponds to at least one first panoramic pixel coordinate and at least one first three-dimensional point cloud coordinate. When the correction module 304 corrects the position information of the walls, doors, and / or windows of each spatial object in the panoramic image and the contour lines in the point cloud planar image based on the panoramic pixel coordinates and three-dimensional point cloud coordinates before and after mapping, it is used to: determine the accurate first panoramic pixel coordinates based on at least one second panoramic pixel coordinate obtained by mapping at least one first three-dimensional point cloud coordinate corresponding to the target spatial location; correct the position information of the walls, doors, and / or windows of each spatial object in the panoramic image based on the accurate first panoramic pixel coordinates; determine the accurate first three-dimensional point cloud coordinates based on at least one second three-dimensional point cloud coordinate obtained by mapping at least one first panoramic pixel coordinate corresponding to the target spatial location; and correct the contour lines in the point cloud planar image based on the accurate first three-dimensional point cloud coordinates.

[0102] In an optional embodiment, before mapping the doors and / or windows of each spatial object in the panoramic image to the corresponding contour lines in the point cloud planar image, the second mapping module 305 is further configured to: determine the panoramic pixel coordinates corresponding to the doors and / or windows of each spatial object according to the panoramic image; map the panoramic pixel coordinates corresponding to the doors and / or windows of each spatial object to the three-dimensional point cloud model according to the mapping relationship between the panoramic pixel coordinates and the three-dimensional point cloud coordinates, thereby obtaining the corresponding three-dimensional point cloud coordinates; and perform planar projection on the three-dimensional point cloud model to map the doors and / or windows of each spatial object in the panoramic image to the corresponding contour lines in the point cloud planar image.

[0103] In an optional embodiment, the target house includes an open space. Before performing planar projection on the 3D point cloud model, the second mapping module 305 is further configured to: determine the panoramic pixel coordinates corresponding to the open space in the target house based on the panoramic image; determine the 3D point cloud coordinates corresponding to the open space in the 3D point cloud model based on the mapping relationship between the panoramic pixel coordinates and the 3D point cloud coordinates; accordingly, the point cloud planar image obtained by performing planar projection on the 3D point cloud model also includes the outline of the open space corresponding to the open space.

[0104] In an optional embodiment, the panoramic image is editable, and there are target doors and / or target windows in the panoramic image whose horizontal width does not meet the requirements. Before performing planar projection on the three-dimensional point cloud model, the second mapping module 305 is further configured to: respond to the horizontal width adjustment operation of the target doors and / or target windows in the panoramic image, adjust the horizontal width of the target doors and / or target windows; and map the panoramic pixel coordinates corresponding to the adjusted horizontal width to the three-dimensional point cloud model according to the mapping relationship between panoramic pixel coordinates and three-dimensional point cloud coordinates to obtain the three-dimensional point cloud coordinates corresponding to the target doors and / or target windows.

[0105] In an optional embodiment, the point cloud plan view is editable, and the outlines on the point cloud plan view include target outlines that do not correspond to the wall position. The second mapping module 305 is further configured to: respond to an adjustment operation on the target outlines on the point cloud plan view, adjust the shape and / or position of the target outlines so that the target outlines correspond to the wall position.

[0106] In an optional embodiment, when the second mapping module 305 responds to an adjustment operation on the target contour line on the point cloud planar graph to adjust the shape and / or position of the target contour line, it is configured to: determine the initial position of the selected target contour line on the point cloud planar graph; respond to a movement operation performed on the selected target contour line, obtain the target position corresponding to the termination of the movement operation, and move the selected target contour line from the initial position to the target position; and / or respond to a shape adjustment operation performed on the selected target contour line, obtain the target shape corresponding to the selected target contour line at the end of the shape adjustment operation, and use it as the shape corresponding to the selected target contour line.

[0107] It should be noted that the specific functions and implementation processes of each module in the above-mentioned device can be found in the above-mentioned method embodiments, and will not be repeated here.

[0108] This application also provides a computer device. Figure 4 A schematic diagram of the structure of a computer device, such as Figure 4 As shown, the computer device includes a processor 41 and a memory 42 storing a computer program; wherein the processor 41 and the memory 42 may be one or more.

[0109] Memory 42 is primarily used to store computer programs that can be executed by processor 41, causing processor 41 to control the computer device to perform corresponding functions, actions, or tasks. In addition to storing computer programs, memory 42 can also be configured to store various other data to support operations on the computer device. Examples of this data include instructions for any application program or method used to operate on the computer device.

[0110] The memory 42 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0111] In this embodiment, the implementation of processor 41 is not limited; it may be, for example, but not limited to, a CPU, GPU, or MCU. Processor 41 can be considered a control system for a computer device, capable of executing computer programs stored in memory 42 to control the computer device to perform corresponding functions, actions, or tasks. It is worth noting that, depending on the implementation of the computer device and the context in which it operates, the required functions, actions, or tasks may differ; correspondingly, the computer programs stored in memory 42 will also differ, and processor 41 can control the computer device to perform different functions and complete different actions or tasks by executing different computer programs.

[0112] In some alternative embodiments, such as Figure 4 As shown, the computer device may also include other components such as a display 43, a power supply component 44, and a communication component 45. Figure 4 The diagram only shows some components and does not mean that the computer device includes only these components. Figure 4 The components shown are optional. To meet different application needs, computer devices may also include other components, such as those required for voice interaction. Figure 4 As shown, the computer device may also include an audio component 46. The specific components that a computer device may include depend on the product form of the computer device and are not limited here.

[0113] In this embodiment of the application, when the processor 41 executes the computer program in the memory 42, it is used to: acquire panoramic images and point cloud planar images corresponding to each spatial object in the target house, wherein the point cloud planar images are generated from the point cloud data corresponding to each spatial object, the panoramic images include the walls, doors and / or windows of each spatial object, and the point cloud planar images include the outlines corresponding to each wall; for the target spatial location in each spatial object, determine the corresponding first panoramic pixel coordinates from the panoramic images and the corresponding first three-dimensional point cloud coordinates from the point cloud data; and map the first panoramic pixel coordinates to a second three-dimensional point cloud coordinate system according to the relative pose relationship between the devices acquiring the panoramic images and point cloud data. The system obtains 3D point cloud coordinates and maps the first 3D point cloud coordinates to a second panoramic pixel coordinate system. Based on the panoramic pixel coordinates and 3D point cloud coordinates before and after mapping, it corrects the position information of walls, doors, and / or windows of each spatial object in the panoramic image, as well as the outlines in the point cloud planar image. Based on the corrected panoramic image and point cloud planar image, it maps the doors and / or windows of each spatial object in the panoramic image to the corresponding outlines in the point cloud planar image, obtaining a point cloud planar image containing door and window outlines. It marks the door and window outlines mapped onto the point cloud planar image and uses the marked point cloud planar image as the floor plan of the target house for display.

[0114] In an optional embodiment, when the processor 41 acquires the panoramic view and point cloud plan view corresponding to each spatial object in the target house, it is configured to: acquire the panoramic data and point cloud data corresponding to each spatial object in the target house; perform three-dimensional real-world spatial rendering on the panoramic data to obtain the panoramic view corresponding to each spatial object; and generate a three-dimensional point cloud model corresponding to the target house based on the point cloud data; and perform planar projection on the three-dimensional point cloud model to obtain the point cloud plan view corresponding to each spatial object.

[0115] In an optional embodiment, when the processor 41 maps the first panoramic pixel coordinates to the second three-dimensional point cloud coordinates in the three-dimensional point cloud coordinate system according to the relative pose relationship between the devices acquiring panoramic images and point cloud data, it is configured to: map the first panoramic pixel coordinates to a spherical space according to the mapping relationship between panoramic pixel coordinates and spherical coordinates to obtain the corresponding first spherical coordinates; and map the first spherical coordinates to the three-dimensional point cloud coordinate system according to the relative pose relationship between the devices acquiring panoramic images and point cloud data, combined with the mapping relationship between spherical coordinates and three-dimensional point cloud coordinates, to obtain the corresponding second three-dimensional point cloud coordinates.

[0116] In an optional embodiment, when the processor 41 maps the first three-dimensional point cloud coordinates to the second panoramic pixel coordinates in the panoramic pixel coordinate system based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data, it is configured to: map the first three-dimensional point cloud coordinates to a spherical space based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data, combined with the mapping relationship between spherical coordinates and three-dimensional point cloud coordinates, to obtain the corresponding second spherical coordinates; and map the second spherical coordinates to the panoramic space based on the mapping relationship between panoramic pixel coordinates and spherical coordinates, to obtain the corresponding second panoramic pixel coordinates.

[0117] In an optional embodiment, the target spatial location corresponds to at least one first panoramic pixel coordinate and at least one first three-dimensional point cloud coordinate. When the processor 41 corrects the position information of the walls, doors, and / or windows of each spatial object in the panoramic image and the contour lines in the point cloud planar image based on the panoramic pixel coordinates and three-dimensional point cloud coordinates before and after mapping, the processor 41 is configured to: determine the accurate first panoramic pixel coordinates based on at least one second panoramic pixel coordinates obtained by mapping at least one first three-dimensional point cloud coordinate corresponding to the target spatial location; correct the position information of the walls, doors, and / or windows of each spatial object in the panoramic image based on the accurate first panoramic pixel coordinates; determine the accurate first three-dimensional point cloud coordinates based on at least one second three-dimensional point cloud coordinates obtained by mapping at least one first panoramic pixel coordinate corresponding to the target spatial location; and correct the contour lines in the point cloud planar image based on the accurate first three-dimensional point cloud coordinates.

[0118] In an optional embodiment, before mapping the doors and / or windows of each spatial object in the panoramic image to the corresponding contour lines in the point cloud planar image, the processor 41 is further configured to: determine the panoramic pixel coordinates corresponding to the doors and / or windows of each spatial object according to the panoramic image; map the panoramic pixel coordinates corresponding to the doors and / or windows of each spatial object to the three-dimensional point cloud model according to the mapping relationship between the panoramic pixel coordinates and the three-dimensional point cloud coordinates to obtain the corresponding three-dimensional point cloud coordinates; and perform planar projection on the three-dimensional point cloud model to map the doors and / or windows of each spatial object in the panoramic image to the corresponding contour lines in the point cloud planar image.

[0119] In an optional embodiment, the target house includes an open space. Before projecting the three-dimensional point cloud model onto a plane, the processor 41 is further configured to: determine the panoramic pixel coordinates corresponding to the open space in the target house based on the panoramic image; determine the three-dimensional point cloud coordinates corresponding to the open space in the three-dimensional point cloud model based on the mapping relationship between the panoramic pixel coordinates and the three-dimensional point cloud coordinates; accordingly, the point cloud planar image obtained by projecting the three-dimensional point cloud model onto a plane also includes the outline of the open space corresponding to the open space.

[0120] In an optional embodiment, the panoramic image is editable, and there are target doors and / or target windows in the panoramic image whose horizontal width does not meet the requirements. Before performing planar projection on the three-dimensional point cloud model, the processor 41 is further configured to: respond to the horizontal width adjustment operation of the target doors and / or target windows in the panoramic image, adjust the horizontal width of the target doors and / or target windows; and map the panoramic pixel coordinates corresponding to the adjusted horizontal width to the three-dimensional point cloud model according to the mapping relationship between panoramic pixel coordinates and three-dimensional point cloud coordinates to obtain the three-dimensional point cloud coordinates corresponding to the target doors and / or target windows.

[0121] In an optional embodiment, the point cloud plan view is editable, and the outline on the point cloud plan view includes a target outline that does not correspond to the position of the wall. The processor 41 is further configured to: in response to an adjustment operation on the target outline on the point cloud plan view, adjust the shape and / or position of the target outline to make the target outline correspond to the position of the wall.

[0122] In an optional embodiment, when the processor 41 adjusts the shape and / or position of the target contour line in response to an adjustment operation on the point cloud planar graph, it is configured to: determine the initial position of the selected target contour line on the point cloud planar graph; in response to a movement operation performed on the selected target contour line, obtain the target position corresponding to the termination of the movement operation, and move the selected target contour line from the initial position to the target position; and / or in response to a shape adjustment operation performed on the selected target contour line, obtain the target shape corresponding to the selected target contour line at the end of the shape adjustment operation, as the shape corresponding to the selected target contour line.

[0123] It should be noted that the specific functions of the processor in the above-mentioned computer device can be found in the above method embodiments, and will not be repeated here.

[0124] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can perform the steps that can be executed by a computer device in the above method embodiments.

[0125] The communication components in the above embodiments are configured to facilitate wired or wireless communication between the device housing the communication component and other devices. The device housing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0126] The display in the above embodiments includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0127] The power supply component in the above embodiments provides power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0128] The audio component in the above embodiments can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0129] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0131] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0132] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0134] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0135] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0136] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0137] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for generating house floor plans, characterized in that, include: Obtain panoramic view and point cloud plan view corresponding to each spatial object in the target house. The point cloud plan view is generated from the point cloud data corresponding to each spatial object. The panoramic view includes the walls, doors and / or windows of each spatial object. The point cloud plan view includes the outline of each wall. For the target spatial location in each spatial object, the corresponding first panoramic pixel coordinates are determined from the panoramic image, and the corresponding first three-dimensional point cloud coordinates are determined from the point cloud data. Based on the relative pose relationship between the devices acquiring panoramic images and point cloud data, the first panoramic pixel coordinates are mapped to the second three-dimensional point cloud coordinates in the three-dimensional point cloud coordinate system, and the first three-dimensional point cloud coordinates are mapped to the second panoramic pixel coordinates in the panoramic pixel coordinate system. Based on at least one second panoramic pixel coordinate obtained by mapping at least one first three-dimensional point cloud coordinate corresponding to the walls, doors and / or windows of each spatial object, the accurate first panoramic pixel coordinate is determined. Based on at least one second three-dimensional point cloud coordinate obtained by mapping at least one first panoramic pixel coordinate corresponding to the outline of each spatial object on the point cloud planar map, the accurate first three-dimensional point cloud coordinate is determined. Based on the accurate first panoramic pixel coordinates and the accurate first three-dimensional point cloud coordinates, the position information of the walls, doors and / or windows of each spatial object in the panoramic image and the outline lines in the point cloud planar image are corrected respectively. Based on the corrected panoramic image and point cloud planar image, the doors and / or windows of each spatial object in the panoramic image are mapped onto the corresponding contour lines in the point cloud planar image to obtain a point cloud planar image containing the door contours and window contours. as well as The outlines of doors and windows mapped onto the point cloud plan are marked, and the marked point cloud plan is used as the floor plan of the target house for display.

2. The method according to claim 1, characterized in that, Obtain panoramic views and point cloud floor plans corresponding to each spatial object in the target house, including: Acquire panoramic and point cloud data corresponding to each spatial object in the target house; The panoramic data is rendered in 3D real-world space to obtain panoramic images corresponding to each spatial object; and Based on the point cloud data, a three-dimensional point cloud model corresponding to the target house is generated; The three-dimensional point cloud model is projected onto a plane to obtain a point cloud planar diagram corresponding to each spatial object.

3. The method according to claim 2, characterized in that, Based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data, the first panoramic pixel coordinates are mapped to a second three-dimensional point cloud coordinate system, including: Based on the mapping relationship between panoramic pixel coordinates and spherical coordinates, the first panoramic pixel coordinates are mapped to the spherical space to obtain the corresponding first spherical coordinates; Based on the relative pose relationship between the devices acquiring panoramic images and point cloud data, and combined with the mapping relationship between spherical coordinates and three-dimensional point cloud coordinates, the first spherical coordinates are mapped to the three-dimensional point cloud coordinate system to obtain the corresponding second three-dimensional point cloud coordinates.

4. The method according to claim 2, characterized in that, Based on the relative pose relationship between the devices acquiring the panoramic image and point cloud data, the first three-dimensional point cloud coordinates are mapped to second panoramic pixel coordinates in the panoramic pixel coordinate system, including: Based on the relative pose relationship between the devices acquiring panoramic images and point cloud data, and combined with the mapping relationship between spherical coordinates and three-dimensional point cloud coordinates, the first three-dimensional point cloud coordinates are mapped to the spherical space to obtain the corresponding second spherical coordinates. Based on the mapping relationship between panoramic pixel coordinates and spherical coordinates, the second spherical coordinates are mapped onto the panoramic space to obtain the corresponding second panoramic pixel coordinates.

5. The method according to any one of claims 2-4, characterized in that, Based on accurate first panoramic pixel coordinates and accurate first 3D point cloud coordinates, the position information of walls, doors, and / or windows of each spatial object in the panoramic image, as well as the contour lines in the point cloud planar image, are corrected, including: Based on the accurate first panoramic pixel coordinates, correct the positional information of the walls, doors, and / or windows of each spatial object in the panoramic image; and Based on the accurate first three-dimensional point cloud coordinates, the outlines of each spatial object in the point cloud planar diagram are corrected.

6. The method according to claim 5, characterized in that, Before mapping the doors and / or windows of each spatial object in the panoramic image onto the corresponding contour lines in the point cloud planar image, the method further includes: Based on the panoramic image, determine the panoramic pixel coordinates corresponding to the doors and / or windows of each spatial object; Based on the mapping relationship between panoramic pixel coordinates and 3D point cloud coordinates, the panoramic pixel coordinates corresponding to the doors and / or windows of each spatial object are mapped to the 3D point cloud model to obtain the corresponding 3D point cloud coordinates. A planar projection is performed on the 3D point cloud model to map the doors and / or windows of each spatial object in the panoramic image onto the corresponding contour lines in the point cloud planar image.

7. The method according to claim 6, characterized in that, The target house includes open space, and the method further includes, before performing a planar projection on the 3D point cloud model: Based on the panoramic image, determine the panoramic pixel coordinates corresponding to the open space in the target house; Based on the mapping relationship between panoramic pixel coordinates and 3D point cloud coordinates, the 3D point cloud coordinates corresponding to the open space in the 3D point cloud model are determined; Accordingly, the point cloud planar diagram obtained by projecting the three-dimensional point cloud model onto a plane also includes the outline of the open space corresponding to the open space.

8. The method according to claim 6, characterized in that, The panoramic image is editable, and the panoramic image contains target doors and / or target windows whose horizontal width does not meet the requirements. Before performing planar projection on the 3D point cloud model, the method further includes: In response to an operation that adjusts the horizontal width of the target door and / or the target window in the panoramic view, the horizontal width of the target door and / or the target window is adjusted. Based on the mapping relationship between panoramic pixel coordinates and 3D point cloud coordinates, the panoramic pixel coordinates corresponding to the adjusted horizontal width are mapped to the 3D point cloud model to obtain the 3D point cloud coordinates corresponding to the target door and / or target window.

9. The method according to claim 6, characterized in that, The point cloud plan view is editable, and the outlines on the point cloud plan view include target outlines that do not correspond to the positions of the walls. The method further includes: In response to an adjustment operation on the target contour line on the point cloud planar view, the shape and / or position of the target contour line are adjusted so that the target contour line corresponds to the position of the wall.

10. The method according to claim 9, characterized in that, In response to an adjustment operation on the target contour line on the point cloud planar view, adjusting the shape and / or position of the target contour line includes: Determine the initial position of the selected target contour line on the point cloud planar graph; In response to a movement operation performed on the selected target contour line, the target position corresponding to the termination of the movement operation is obtained, and the selected target contour line is moved from the initial position to the target position. and / or In response to the shape adjustment operation performed on the selected target contour line, the target shape corresponding to the selected target contour line at the end of the shape adjustment operation is obtained as the shape corresponding to the selected target contour line.

11. A house floor plan generation device, characterized in that, include: The acquisition module is used to acquire panoramic images and point cloud plan views corresponding to each spatial object in the target house. The point cloud plan view is generated from the point cloud data corresponding to each spatial object. The panoramic image includes the walls, doors and / or windows of each spatial object, and the point cloud plan view includes the outlines of each wall. The determination module is used to determine the corresponding first panoramic pixel coordinates from the panoramic image and the corresponding first three-dimensional point cloud coordinates from the point cloud data for the target spatial position in each spatial object. The first mapping module is used to map the first panoramic pixel coordinates to the second three-dimensional point cloud coordinates in the three-dimensional point cloud coordinate system according to the relative pose relationship between the devices that acquire panoramic images and point cloud data, and to map the first three-dimensional point cloud coordinates to the second panoramic pixel coordinates in the panoramic pixel coordinate system. The correction module is used to determine accurate first panoramic pixel coordinates based on at least one second panoramic pixel coordinate obtained by mapping at least one first three-dimensional point cloud coordinates corresponding to the walls, doors, and / or windows of each spatial object; to determine accurate first three-dimensional point cloud coordinates based on at least one second three-dimensional point cloud coordinate obtained by mapping at least one first panoramic pixel coordinates corresponding to the contour lines of each spatial object on the point cloud planar diagram; and to correct the position information of the walls, doors, and / or windows of each spatial object in the panoramic diagram and the contour lines in the point cloud planar diagram based on the accurate first panoramic pixel coordinates and the accurate first three-dimensional point cloud coordinates, respectively. The second mapping module is used to map the doors and / or windows of each spatial object in the panoramic image to the corresponding contour lines in the point cloud planar image based on the corrected panoramic image and point cloud planar image, so as to obtain a point cloud planar image containing the door contours and window contours. The marking module is used to mark the door and window outlines mapped onto the point cloud plan view, and to use the marked point cloud plan view as the floor plan of the target house for display.

12. A computer device, characterized in that, include: A processor and memory, which, when the processor executes a computer program, are used to implement the steps of the method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic terminal device, the electronic terminal device is enabled to perform the steps of the method as described in any one of claims 1-10.

Images