Virtualization reconstruction method and device for indoor environment, electronic equipment and medium
By generating cubes at point cloud data locations and clustering simplified models, the problems of large amount of computation and high memory usage of three-dimensional models for reconstruction of indoor environments in the prior art are solved, and the rendering performance and user experience are improved.
Patent Information
- Application Number
- CN202411628571.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2025-03-07
AI Technical Summary
When reconstructing a three-dimensional model of an indoor environment, the existing technology has large computing volume and high memory usage, which affects the rendering performance and the operating performance of virtual reality systems.
By acquiring the point cloud model, establishing the target cuboid, and simplifying the model based on the clustering algorithm, reducing the number of triangular mesh surfaces to improve rendering performance.
On the basis of ensuring the authenticity of the reconstruction model, the model structure is simplified, the memory usage is reduced, and the rendering performance and user experience are improved.
Smart Images
Figure CN119579825A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of constructing a virtual environment, and in particular, to a method, device, electronic device and medium for virtualizing and reconstructing an indoor environment. Background Art
[0002] One of the main goals of immersive virtual reality is to create a virtual environment that can induce a strong sense of presence. Users wearing head-mounted displays are best not to be disturbed by external factors to maintain a high degree of immersion. However, users are actually in a real, chaotic environment and cannot see objects in the real environment. When users interact with virtual scenes, they can easily bump into objects in the real environment, which hinders interaction with the real environment. In order to enhance virtual interaction, these obstacles should be embedded in the virtual environment. Therefore, it is necessary to know the 3D structure of the real environment in order to align it with the virtual environment. In this way, users can truly feel the surrounding environment, which not only solves the problem of bumping with objects in the real environment, but also enhances the user's immersive experience.
[0003] The existing technology collects point cloud data of the indoor environment, extracts multiple semantic component units from the point cloud data to generate a scene information graph, generates a physical structure according to the component category, and assembles the physical structure and the models of the attached objects into a three-dimensional model of the indoor scene. The existing technology has a large amount of calculation in the reconstruction process, and the reconstructed indoor environment model has a high memory usage, which will affect the rendering performance of the computer and the operating performance of the virtual reality system, and affect the experience of virtual reality users. Summary of the invention
[0004] The purpose of the embodiments of the present application is to provide a method, device, electronic device and medium for virtual reconstruction of an indoor environment, which are used to solve the above-mentioned problems existing in the prior art. The reconstructed model can be simplified on the basis of ensuring the authenticity of the reconstructed model, and the number of triangular mesh surfaces can be reduced to improve rendering performance.
[0005] In a first aspect, a method for virtual reconstruction of an indoor environment is provided, and the method may include:
[0006] Acquire a point cloud model composed of a plurality of point cloud data corresponding to the room to be processed;
[0007] When each first cube in the configured first three-dimensional grid plane collides with any point cloud data in the point cloud model, a target cuboid is established with the point cloud data as a vertex, the distance from the point cloud data to a target plane as a target height, and the side length of the first cube as a target length and a target width; wherein the target plane is a ground plane or a ceiling plane of the point cloud model;
[0008] Based on multiple target cuboids, target cuboids with the same target height are clustered to obtain multi-category cuboid clusters;
[0009] For any type of cuboid cluster, create a bounding box of the cuboid cluster to obtain a mid-term virtual model consisting of a plurality of bounding boxes;
[0010] Based on the configured adjustment coefficients, multiple bounding boxes in the mid-term virtual model are adjusted to obtain a reconstructed target virtual model.
[0011] In a possible implementation, the process of determining the ground plane in the point cloud model includes:
[0012] A collision detection is performed on each second cube in the configured second three-dimensional grid plane and a plurality of point cloud data in the point cloud model to determine a ground plane in the point cloud model.
[0013] In a possible implementation, determining a ground plane in the point cloud model includes:
[0014] Acquire the number of point clouds where the plurality of second cubes collide with the point cloud data on six surfaces in the point cloud model;
[0015] The surface corresponding to the maximum number of point clouds after the point cloud data on the six surfaces collide is determined as the ground plane.
[0016] In a possible implementation, the adjustment coefficient includes: a scaling factor, a rotation angle, and a translation parameter;
[0017] Based on the configured adjustment coefficients, multiple bounding boxes in the mid-term virtual model are adjusted to obtain a reconstructed target virtual model, including:
[0018] The plurality of bounding boxes are scaled, translated and rotated according to the scaling factor, the rotation angle and the translation parameter to obtain the target virtual model.
[0019] In a possible implementation, two infrared laser positioning lighthouses are installed in the room to be processed;
[0020] The process of determining the scaling factor includes:
[0021] The scaling factor is determined according to a first distance between two infrared laser positioning lighthouses in the head mounted display and a second distance between two virtual infrared laser positioning lighthouses in the mid-term virtual model.
[0022] In a possible implementation, the process of determining the translation parameter includes:
[0023] The translation parameter is determined according to a first position of a first midpoint in a head-mounted display coordinate system and a second position of a second midpoint in a virtual model coordinate system; the first midpoint is the midpoint between two infrared laser positioning lighthouses in the head-mounted display, and the second midpoint is the midpoint between two virtual infrared laser positioning lighthouses in the mid-term virtual model.
[0024] In a possible implementation, the process of determining the rotation angle includes:
[0025] The rotation angle is determined according to a first vector between two infrared laser positioning lighthouses in the head mounted display coordinate system and a second vector between two virtual infrared laser positioning lighthouses in the virtual model coordinate system.
[0026] In a second aspect, a virtual reconstruction device for an indoor environment is provided, and the device may include:
[0027] An acquisition unit, used for acquiring a point cloud model composed of a plurality of point cloud data corresponding to the room to be processed;
[0028] a creation unit, configured to establish a target cuboid with the point cloud data as a vertex, the distance from the point cloud data to a target plane as a target height, and the side length of the first cube as a target length and a target width when each first cube in the configured first three-dimensional grid plane collides with any point cloud data in the point cloud model; wherein the target plane is a ground plane or a ceiling plane of the point cloud model;
[0029] A clustering unit, used for clustering target cuboids of the same target height based on multiple target cuboids to obtain multiple categories of cuboid clusters;
[0030] The creation unit is further used to create a bounding box of the cuboid cluster of any category to obtain a mid-term virtual model composed of a plurality of bounding boxes;
[0031] The adjustment unit is used to adjust the multiple bounding boxes in the mid-term virtual model based on the configured adjustment coefficient to obtain the reconstructed target virtual model.
[0032] In a third aspect, an electronic device is provided, the electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0033] Memory, used to store computer programs;
[0034] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.
[0035] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.
[0036] The present application provides a method for virtual reconstruction of an indoor environment, the method comprising: obtaining a point cloud model composed of a plurality of point cloud clusters corresponding to the indoor environment to be processed; when each first cube in the configured first three-dimensional grid plane collides with any point cloud data in the point cloud model, taking the point cloud data as a vertex, taking the distance from the point cloud data to the connecting plane of the point cloud cluster to which the point cloud data belongs as a target height, and taking the side length of the first cube as a target length and a target width to establish a target cuboid; based on a plurality of target cuboids, clustering the target cuboids of the same target height to obtain a plurality of categories of cuboid clusters; for any category of cuboid clusters, creating a bounding box of the cuboid cluster to obtain a mid-term virtual model composed of a plurality of bounding boxes; based on the configured adjustment coefficient, adjusting the plurality of bounding boxes in the mid-term virtual model to obtain a reconstructed target virtual model. The traditional point cloud-based three-dimensional reconstruction method is to generate a triangular mesh model based on the point cloud data. This method takes a long time to process and will generate a large amount of triangular mesh data, resulting in high memory usage, affecting the rendering performance of the computer and the running performance of the virtual reality system. The purpose of this application is to reconstruct a model that represents real objects in the room, so as to enhance the user's perception of the real physical space and thus enhance the user's immersive experience. Therefore, there is no need to reconstruct a particularly refined model. This application can reduce the time for point cloud data processing by generating cubes at the location of point cloud data, and further cluster and merge the cubes by clustering methods, simplify the model on the basis of ensuring the realism of reconstruction, and reduce the number of triangular mesh faces to improve rendering performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0038] Figure 1 A system architecture diagram of a virtual reconstruction method for an indoor environment provided in an embodiment of the present application;
[0039] Figure 2 A schematic diagram of a flow chart of a method for virtual reconstruction of an indoor environment provided in an embodiment of the present application;
[0040] Figure 3A schematic diagram of the structure of a virtual reconstruction device for an indoor environment provided in an embodiment of the present application;
[0041] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0043] The virtual reconstruction method of an indoor environment provided in the embodiment of the present application can be applied in Figure 1 In the system architecture shown in Figure 1 As shown, the system may include: a server and an RGB-D sensor. The server may be a physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The RGB-D sensor and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.
[0044] The RGB-D sensor is used to obtain multiple point cloud data in the room to be processed and send the multiple point cloud data to the server.
[0045] The server is used to obtain multiple point cloud data to execute a virtual reconstruction method of an indoor environment provided in the present application.
[0046] One of the main goals of immersive virtual reality is to create a virtual environment that can induce a strong sense of presence. Users wearing head-mounted displays are best not to be disturbed by external factors to maintain a high degree of immersion. However, users are actually in a real, chaotic environment and cannot see objects in the real environment. When users interact with virtual scenes, they can easily bump into objects in the real environment, which hinders interaction with the real environment. In order to enhance virtual interaction, these obstacles should be embedded in the virtual environment. Therefore, it is necessary to know the 3D structure of the real environment in order to align it with the virtual environment. In this way, users can truly feel the surrounding environment, which not only solves the problem of bumping with objects in the real environment, but also enhances the user's immersive experience.
[0047] The existing technology collects point cloud data of the indoor environment, extracts multiple semantic component units from the point cloud data to generate a scene information graph, generates a physical structure according to the component category, and assembles the physical structure and the models of the attached objects into a three-dimensional model of the indoor scene. The existing technology has a large amount of calculation in the reconstruction process, and the reconstructed indoor environment model has a high memory usage, which will affect the rendering performance of the computer and the operating performance of the virtual reality system, and affect the experience of virtual reality users.
[0048] Therefore, the present application provides a virtual reconstruction method for an indoor environment to solve the above-mentioned problems existing in the prior art. The reconstructed model can be simplified while ensuring its authenticity, thereby reducing the number of triangular mesh surfaces to improve rendering performance.
[0049] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.
[0050] Figure 2 The flowchart of a virtual reconstruction method of an indoor environment provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method may include:
[0051] Step S210: Obtain a point cloud model consisting of a plurality of point cloud data corresponding to the room to be processed.
[0052] Specifically, an RGB-D sensor device is used to collect point cloud data of an environment in a room to be processed where a user of a head mounted display (which may be HTC Vive) is located, to obtain a certain amount of point cloud data.
[0053] Step S220: performing collision detection on the plurality of first cubes in the configured first three-dimensional grid plane and the plurality of point cloud data in the point cloud model, and constructing an initial virtual model consisting of a plurality of target cuboids according to the detection result.
[0054] The first three-dimensional grid plane is composed of m×n first cubes, and the length and width of the first three-dimensional grid plane are greater than the length and width of the point cloud model.
[0055] Before executing step S220 , the bottom plane among the six surfaces in the point cloud model is determined.
[0056] Specifically, first, collision detection is performed on each second cube in the configured second three-dimensional grid plane and multiple point cloud data in the point cloud model; this process can be understood as: a second three-dimensional grid plane composed of a×b second cubes is slid through the point cloud model; wherein, the length and width of the second three-dimensional grid plane are greater than the length and width of the point cloud model.
[0057] Secondly, the number of point clouds that collide with the point cloud data on the six faces of the point cloud model is obtained;
[0058] Finally, the surface corresponding to the maximum number of point clouds after the point cloud data on the six surfaces collide is determined as the ground plane. This process can be understood as: during the sliding of the second three-dimensional mesh surface, the intersection area is judged by collision detection, and the side with the maximum intersection area between the point cloud model and the first mesh plane is taken as the ground plane.
[0059] It is understandable that since there are many objects on the ground plane, in the collision detection process, after the collision detection occurs on the ground plane, the number of point clouds that collide is the largest, so the surface corresponding to the largest number of point clouds is determined as the ground plane. The opposite surface of the ground plane is the ceiling plane.
[0060] After determining the ground plane and the ceiling plane, step S220 specifically includes:
[0061] Using multiple first cubes in the first grid plane, starting from a virtual plane where no collision occurs between the ground plane and the ceiling plane, slide through the point cloud model in the direction of the ceiling plane and the ground plane respectively. During the sliding process, determine whether each point cloud data in the point cloud model collides with the first cube:
[0062] After the multiple first cubes collide with any point cloud data in the point cloud model, when the multiple cubes move toward the ceiling plane, the distance from the point cloud data to the ceiling plane is used as the target height, and the side length of the first cube is used as the target length and the target width to establish a target cuboid. When the multiple cubes move toward the ground plane, the distance from the point cloud data to the ceiling plane is used as the target height, and the side length of the first cube is used as the target length and the target width to establish a target cuboid.
[0063] Each target cuboid created is marked with a unique ID, and the target height of the target cuboid is recorded;
[0064] The first cube that collides on the first three-dimensional grid plane is eliminated, which means that in the subsequent sliding process, the first cube does not need to perform collision detection again. Starting from the virtual plane between the ground plane and the ceiling plane, it slides through the point cloud model towards the ceiling plane and towards both sides of the ground plane, and stops detecting when reaching the ground plane or the ceiling plane.
[0065] After constructing multiple target cuboids, an initial virtual model consisting of multiple target cuboids is obtained.
[0066] It should be noted that a 3D grid plane refers to a surface defined by a set of polygons in a 3D space, which is used to represent the contour and topological structure of a 3D model surface. The side lengths of the first cube are smaller than the side lengths of the second cube, that is, the side lengths and volumes of the small cubes in the first grid plane are smaller than those in the second grid plane.
[0067] Step S230 : Based on the multiple target cuboids, cluster the target cuboids with the same target height to obtain multiple categories of cuboid clusters, and create a bounding box for any cuboid cluster.
[0068] Specifically, the memory usage of the initial virtual model composed of multiple target cuboids is still high. In order to further improve the rendering performance of the system, the DBSCAN clustering algorithm is used to cluster multiple target cuboids.
[0069] At the same time, in order to maintain the authenticity of the reconstruction, the height data of multiple target cuboids are used as clustering parameters for voxel classification. That is, target cuboids with the same height and the distance between target cuboids with the same height is less than the preset distance can be classified into the same category, thus obtaining a cuboid cluster. The height threshold of the clustering is set to λ.
[0070] After clustering is completed, a completely enclosed AABB-type bounding box is established for each category of rectangular clusters, and rectangular blocks of the same size as the bounding box are used to replace all rectangular clusters of this category. After all categories are replaced, a mid-term virtual model can be obtained.
[0071] Step S240: Based on the configured adjustment coefficients, multiple bounding boxes are adjusted to obtain a reconstructed virtual model.
[0072] Specifically, it is necessary to spatially align the mid-term virtual model with the real environment, that is, the environment in the room to be treated. The alignment is achieved by translating, rotating and scaling the mid-term virtual model to match the real environment. The translation parameters, rotation angles and scaling factors can be calculated through at least three pairs of matching points between the real environment and the mid-term virtual model.
[0073] That is, the adjustment coefficients include: scaling factor, rotation angle and translation parameter;
[0074] Since the ground plane is determined in the previous step, the Z-axis direction of the ground plane can be aligned. Therefore, only two pairs of matching points need to be found to complete the spatial alignment.
[0075] Taking HTC Vive as an example of a head mounted display, this application uses the positions of two infrared laser positioning lighthouses of HTC Vive as key points for spatial alignment.
[0076] That is to say, two infrared laser positioning lighthouses are installed at corresponding positions in the room to be treated, and the specific positions can be set at two diagonal positions on the ceiling side of the room to be treated.
[0077] A. The process of determining the scaling factor may include:
[0078] Get the first distance d between two infrared laser positioning lighthouses in HTC Vive r , and the second distance d between the two virtual infrared laser positioning beacons in the mid-term virtual model v .
[0079] The scaling factor S is calculated as:
[0080] s=d r / d v
[0081] B. The process of determining translation parameters includes:
[0082] Obtaining a first position of a midpoint between two infrared laser positioning lighthouses in the head mounted display in a head mounted display coordinate system;
[0083] and obtaining a second position of a midpoint between two virtual infrared laser positioning lighthouses in the mid-term virtual model in the virtual model coordinate system;
[0084] By aligning the first position and the second position, the translation parameters can be determined.
[0085] C. The process of determining the rotation angle includes:
[0086] Get the first vector v between the two infrared laser positioning lighthouses in the head mounted display coordinate system r ;
[0087] And, obtain the second vector v between the two virtual infrared laser positioning lighthouses in the virtual model coordinate system v ;
[0088] According to the first vector and the second vector, the rotation angle α is determined, and the process can be expressed as:
[0089]
[0090] Afterwards, the multiple bounding boxes in the mid-term virtual model and the six faces of the mid-term virtual model are first scaled according to the scaling factor, then translated according to the translation parameter, and finally rotated according to the rotation angle. This completes the spatial alignment of the mid-term virtual model with the head-mounted display coordinate system, and the point cloud model of the room to be processed and the positions of the real objects in the room to be processed are correspondingly displayed in the target virtual model.
[0091] The present application provides a method for virtual reconstruction of an indoor environment, the method comprising: obtaining a point cloud model composed of a plurality of point cloud data corresponding to the indoor environment to be processed; when each first cube in the configured first three-dimensional grid plane collides with any point cloud data in the point cloud model, establishing a target cuboid with the point cloud data as the vertex, the distance from the point cloud data to the target plane as the target height, and the side length of the first cube as the target length and the target width; based on the plurality of target cuboids, clustering the target cuboids of the same target height to obtain a plurality of cuboid clusters; for any category of cuboid clusters, creating a bounding box of the cuboid cluster to obtain a mid-term virtual model composed of a plurality of bounding boxes; based on the configured adjustment coefficient, adjusting the plurality of bounding boxes in the mid-term virtual model to obtain a reconstructed target virtual model. The traditional point cloud-based three-dimensional reconstruction method is to generate a triangular mesh model based on the point cloud data. This method takes a long time to process and will generate a large amount of triangular mesh data, resulting in high memory usage, affecting the rendering performance of the computer and the running performance of the virtual reality system. The purpose of this application is to reconstruct a model that represents real objects in the room, so as to enhance the user's perception of the real physical space and thus enhance the user's immersive experience. Therefore, there is no need to reconstruct a particularly refined model. This application can reduce the time for point cloud data processing by generating cubes at the location of point cloud data, and further cluster and merge the cubes by clustering methods, simplify the model on the basis of ensuring the realism of reconstruction, and reduce the number of triangular mesh faces to improve rendering performance.
[0092] Corresponding to the above method, the embodiment of the present application also provides a virtual reconstruction device for an indoor environment, such as Figure 3 As shown, the device comprises:
[0093] An acquisition unit 310 is used to acquire a point cloud model composed of a plurality of point cloud data corresponding to the room to be processed;
[0094] The creation unit 320 is used to establish a target cuboid with the point cloud data as a vertex, the distance from the point cloud data to a target plane as a target height, and the side length of the first cube as a target length and a target width when each first cube in the configured first three-dimensional grid plane collides with any point cloud data in the point cloud model; wherein the target plane is a ground plane or a ceiling plane of the point cloud model;
[0095] A clustering unit 330 is used to cluster target cuboids of the same target height based on the multiple target cuboids to obtain multiple categories of cuboid clusters;
[0096] The creation unit 320 is further used to create a bounding box of the cuboid cluster of any category to obtain a mid-term virtual model composed of a plurality of bounding boxes;
[0097] The adjustment unit 340 is used to adjust the multiple bounding boxes in the mid-term virtual model based on the configured adjustment coefficient to obtain a reconstructed target virtual model.
[0098] The functions of each functional unit of a virtual reconstruction device for an indoor environment provided in the above-mentioned embodiment of the present application can be realized through the above-mentioned method steps. Therefore, the specific working process and beneficial effects of each unit in a virtual reconstruction device for an indoor environment provided in the embodiment of the present application will not be repeated here.
[0099] The present application also provides an electronic device, such as Figure 4 As shown, it includes a processor 410 , a communication interface 420 , a memory 430 and a communication bus 440 , wherein the processor 410 , the communication interface 420 , and the memory 430 communicate with each other via the communication bus 440 .
[0100] Memory 430, for storing computer programs;
[0101] The processor 410 is used to execute the program stored in the memory 430 to implement the following steps:
[0102] Acquire a point cloud model composed of a plurality of point cloud data corresponding to the room to be processed;
[0103] When each first cube in the configured first three-dimensional grid plane collides with any point cloud data in the point cloud model, a target cuboid is established with the point cloud data as a vertex, the distance from the point cloud data to a target plane as a target height, and the side length of the first cube as a target length and a target width; wherein the target plane is a ground plane or a ceiling plane of the point cloud model;
[0104] Based on multiple target cuboids, target cuboids with the same target height are clustered to obtain multi-category cuboid clusters;
[0105] For any type of cuboid cluster, create a bounding box of the cuboid cluster to obtain a mid-term virtual model consisting of a plurality of bounding boxes;
[0106] Based on the configured adjustment coefficients, multiple bounding boxes in the mid-term virtual model are adjusted to obtain a reconstructed target virtual model.
[0107] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0108] The communication interface is used for communication between the above electronic device and other devices.
[0109] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0110] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0111] The implementation methods and beneficial effects of the components of the electronic device in the above embodiments to solve the problems can be seen in Figure 2 The various steps in the illustrated embodiment are implemented, therefore, the specific working process and beneficial effects of the electronic device provided by the embodiment of the present application are not repeated here.
[0112] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is executed on a computer, the computer executes a method for virtualizing and reconstructing an indoor environment as described in any of the above embodiments.
[0113] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute a method for virtualizing and reconstructing an indoor environment as described in any one of the above embodiments.
[0114] Those skilled in the art will appreciate that the embodiments in the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt a complete hardware embodiment, a complete software embodiment, or a form of an embodiment combining software and hardware. Moreover, the present application may adopt a form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0115] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0116] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0118] Unless otherwise defined, the technical terms or scientific terms used in this application should be understood by people with ordinary skills in the field to which the present invention belongs. "First", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect", "couple" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0119] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the present application embodiments are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application embodiments.
[0120] Obviously, those skilled in the art can make various changes and modifications to the embodiments in the present application without departing from the spirit and scope of the embodiments in the present application. Thus, if these modifications and variations of the embodiments in the present application are within the scope of the embodiments in the present application and their equivalents, the embodiments in the present application are also intended to include these modifications and variations.
Claims
1. A virtual reconstruction method for an indoor environment, characterized in that: The method comprises: Acquire a point cloud model composed of a plurality of point cloud data corresponding to the room to be processed; When each first cube in the configured first three-dimensional grid plane collides with any point cloud data in the point cloud model, a target cuboid is established with the point cloud data as a vertex, the distance from the point cloud data to a target plane as a target height, and the side length of the first cube as a target length and a target width; wherein the target plane is a ground plane or a ceiling plane of the point cloud model; Based on multiple target cuboids, target cuboids with the same target height are clustered to obtain multi-category cuboid clusters; For any type of cuboid cluster, create a bounding box of the cuboid cluster to obtain a mid-term virtual model consisting of a plurality of bounding boxes; Based on the configured adjustment coefficients, multiple bounding boxes in the mid-term virtual model are adjusted to obtain a reconstructed target virtual model.
2. The method according to claim 1, characterized in that The process of determining the ground plane in the point cloud model includes: A collision detection is performed on each second cube in the configured second three-dimensional grid plane and a plurality of point cloud data in the point cloud model to determine a ground plane in the point cloud model.
3. The method according to claim 2, characterized in that Determining a ground plane in the point cloud model includes: Acquire the number of point clouds where the plurality of second cubes collide with the point cloud data on six surfaces in the point cloud model; The surface corresponding to the maximum number of point clouds after the point cloud data on the six surfaces collide is determined as the ground plane.
4. The method according to claim 1, characterized in that The adjustment coefficients include: scaling coefficient, rotation angle and translation parameter; Based on the configured adjustment coefficients, multiple bounding boxes in the mid-term virtual model are adjusted to obtain a reconstructed target virtual model, including: The plurality of bounding boxes are scaled, translated and rotated according to the scaling factor, the rotation angle and the translation parameter to obtain the target virtual model.
5. The method according to claim 4, characterized in that Two infrared laser positioning lighthouses are installed in the room to be treated; The process of determining the scaling factor includes: The scaling factor is determined according to a first distance between two infrared laser positioning lighthouses in the head mounted display and a second distance between two virtual infrared laser positioning lighthouses in the mid-term virtual model.
6. The method according to claim 5, characterized in that The process of determining the translation parameters includes: The translation parameter is determined according to a first position of a first midpoint in a head-mounted display coordinate system and a second position of a second midpoint in a virtual model coordinate system; the first midpoint is the midpoint between two infrared laser positioning lighthouses in the head-mounted display, and the second midpoint is the midpoint between two virtual infrared laser positioning lighthouses in the mid-term virtual model.
7. The method according to claim 6, characterized in that The process of determining the rotation angle includes: The rotation angle is determined according to a first vector between two infrared laser positioning lighthouses in the head mounted display coordinate system and a second vector between two virtual infrared laser positioning lighthouses in the virtual model coordinate system.
8. A virtual reconstruction device for an indoor environment, characterized in that: The device comprises: An acquisition unit, used for acquiring a point cloud model composed of a plurality of point cloud data corresponding to the room to be processed; a creation unit, configured to establish a target cuboid with the point cloud data as a vertex, the distance from the point cloud data to a target plane as a target height, and the side length of the first cube as a target length and a target width when each first cube in the configured first three-dimensional grid plane collides with any point cloud data in the point cloud model; wherein the target plane is a ground plane or a ceiling plane of the point cloud model; A clustering unit, used for clustering target cuboids of the same target height based on multiple target cuboids to obtain multiple categories of cuboid clusters; The creation unit is further used to create a bounding box of the cuboid cluster of any category to obtain a mid-term virtual model composed of a plurality of bounding boxes; The adjustment unit is used to adjust the multiple bounding boxes in the mid-term virtual model based on the configured adjustment coefficient to obtain the reconstructed target virtual model.
9. An electronic device, characterized in that: The electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.