Indoor environment three-dimensional reconstruction method driven by panoramic image

Through the panoramic image driving method, image feature points are acquired using panoramic cameras and micro camera arrays, combined with the three-dimensional reconstruction algorithm, the problems of high cost and complex operation of indoor environments in the existing technology are solved, and high-precision three-dimensional indoor environments are realized Three-dimensional reconstruction.

CN118918259APending Publication Date: 2024-11-08BEIJING DAXI TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410982974.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the prior art, the three-dimensional reconstruction method of indoor environment is costly and complex in operation, making it difficult to achieve high-precision complex environment reconstruction.

Method used

The panoramic image driving method is adopted to acquire multi-angle environmental images through a panoramic camera, divide functional areas, and optimize the encoding and identification of images and camera layout. The image feature points are collected using a micro-camera array, and a three-dimensional three-dimensional model of the panoramic indoor environment is generated by combining a three-dimensional reconstruction algorithm.

Benefits of technology

It reduces the cost and operation difficulty of three-dimensional reconstruction of indoor environments, and realizes high-precision three-dimensional reconstruction of complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918259A_ABST
    Figure CN118918259A_ABST
Patent Text Reader

Abstract

The invention discloses a panoramic image-driven indoor environment three-dimensional reconstruction method, which relates to the technical field of general image data processing, and is characterized in that a panoramic multi-angle environment image of a target indoor area is acquired through a panoramic camera, functional area division is carried out, coding identification is carried out, and an indoor coding area set is obtained; and carrying out acquisition demand analysis and camera layout optimization on the indoor coding region set to obtain a region micro camera array set, and then carrying out local camera layout. Acquiring local images to obtain a regional local image set, and performing feature point extraction on the regional local image set to obtain a regional image feature point set; and performing panoramic feature point registration and three-dimensional model reconstruction on the panoramic multi-angle environment image and the regional image feature point set by using a three-dimensional reconstruction algorithm to generate a panoramic indoor environment three-dimensional model. The technical problems that in the prior art, an indoor environment three-dimensional reconstruction method is high in reconstruction cost and complex in operation are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to a general technical field related to image data processing, and in particular to a method for three-dimensional reconstruction of an indoor environment driven by a panoramic image. Background Art

[0002] With the rapid development of computer vision and image processing technology, 3D reconstruction technology has been widely used in many fields, such as interior design, construction, etc. Traditional indoor environment 3D reconstruction methods usually rely on laser scanners or structured light cameras. Although these methods can generate high-precision 3D models, they are costly and complex to operate.

[0003] Therefore, the existing indoor environment three-dimensional reconstruction method has the technical problems of high reconstruction cost and complex operation. Summary of the invention

[0004] This application provides a method for 3D reconstruction of indoor environments driven by panoramic images, which solves the technical problems of high reconstruction cost and complex operation in the existing indoor environment 3D reconstruction methods. It achieves high-precision 3D reconstruction of complex indoor environments and reduces the cost and operation difficulty of indoor environment 3D.

[0005] The present application provides a method for three-dimensional reconstruction of an indoor environment driven by a panoramic image, the method comprising: acquiring a panoramic multi-angle environmental image of a target indoor area through a panoramic camera, dividing the panoramic multi-angle environmental image into functional areas, acquiring a set of indoor functional areas, and encoding and identifying the set of indoor functional areas to obtain a set of indoor coded areas. Performing acquisition requirement analysis and camera layout optimization on the set of indoor coded areas to obtain a set of regional micro-camera arrays, and performing local camera layout on the target indoor area according to the set of regional micro-camera arrays. Based on the set of regional micro-camera arrays, local images of the set of indoor coded areas are respectively acquired to obtain a set of regional local images, and feature points are extracted from the set of regional local images to obtain a set of regional image feature points. Using a three-dimensional reconstruction algorithm, panoramic feature point registration and three-dimensional model reconstruction are performed on the panoramic multi-angle environmental image and the set of regional image feature points to generate a three-dimensional model of a panoramic indoor environment.

[0006] In the implementation, the indoor coding area set is obtained, including: gridding the panoramic multi-angle environmental image, obtaining panoramic distribution grid image information, grid numbering the panoramic distribution grid image information, and obtaining panoramic image grid number information. The indoor functional area set is matched and mapped with the panoramic image grid number information to determine the indoor area area information. The indoor functional area set is spatially resolved and functional attribute marked to obtain the regional spatial position sequence relationship and regional functional attribute information. The regional spatial position sequence relationship and regional functional attribute information as well as the indoor area area information are used as coding elements for coding design identification to determine the indoor coding area set.

[0007] In an implementation, obtaining the regional micro camera array set includes: obtaining camera requirement layout factor information, wherein the camera requirement layout factor information includes resolution requirements, viewing angle requirements, and spatial object distribution. Based on the camera requirement layout factor information, image acquisition requirements are analyzed for the indoor coding area set to obtain a regional image acquisition requirement parameter set. Camera selection is performed based on the regional image acquisition requirement parameter set to obtain a regional camera requirement model parameter set. Camera layout analysis is performed on the regional image acquisition requirement parameter set to obtain a camera layout solution parameter set, and the regional micro camera array set is determined based on the regional camera requirement model parameter set and the camera layout solution parameter set.

[0008] In an implementation, the obtaining of camera layout scheme parameters includes: performing coverage analysis based on the regional image acquisition requirement parameter set to obtain an initial camera layout parameter set, wherein the initial camera layout parameter set includes camera positions, angles, and quantities. Performing simulation on the initial camera layout parameter set to obtain camera layout simulation results, and evaluating and verifying the camera layout simulation results to obtain camera layout acquisition quality parameters. Performing optimization strategy analysis on the initial camera layout parameter set based on the camera layout acquisition quality parameters to determine a camera layout optimization improvement strategy. Performing iterative improvement and optimization on the initial camera layout parameter set based on the camera layout optimization improvement strategy to obtain the regional micro camera array set.

[0009] In an implementation, the obtaining of the regional local image set includes: performing local image acquisition on the indoor coding region set based on the regional micro camera array set to obtain a regional acquisition image set. Performing spectrum analysis and noise identification on the regional acquisition image set using Fourier transform to obtain image noise characteristic information. Performing filtering and denoising processing on the regional acquisition image set based on the image noise characteristic information to obtain an available regional image set. Performing sequence integration on the available regional image set according to the viewing angle number information of the regional micro camera array set to obtain the regional local image set.

[0010] In the implementation, the generation of the panoramic indoor environment three-dimensional stereo model includes: sequentially performing multi-dimensional feature point description on the regional image feature point set to determine a regional image feature point description subset. Performing similarity registration on the regional image feature point set based on the regional image feature point description subset to obtain a regional image registration feature point set. Performing feature point mapping fusion on the regional image feature point set through the regional image registration feature point set to obtain a regional panoramic fusion image. Using a three-dimensional reconstruction algorithm, the regional panoramic fusion image is rendered into the panoramic multi-angle environment image to perform three-dimensional model reconstruction to generate the panoramic indoor environment three-dimensional stereo model.

[0011] In the implementation, the generating of the panoramic indoor environment three-dimensional stereo model includes: using a three-dimensional reconstruction algorithm to perform surface reconstruction on the panoramic multi-angle environment image to obtain a panoramic environment three-dimensional grid model. Based on the indoor coding area set, the edge detection of the panoramic environment three-dimensional grid model is performed to extract the edge information of the model coding area. The regional panoramic fusion image is mapped and rendered onto the panoramic environment three-dimensional grid model according to the edge information of the model coding area to generate the panoramic indoor environment three-dimensional stereo model.

[0012] In the implementation, the obtaining of the panoramic environment three-dimensional grid model includes: extracting key feature points and performing stereo matching on the panoramic multi-angle environment image to obtain a multi-angle image matching feature point set. Calibrate the camera parameters of the panoramic multi-angle environment image using a camera calibration algorithm to determine the camera internal calibration parameters and the camera external calibration parameters. Reconstruct the point cloud surface of the multi-angle image matching feature point set and the camera internal calibration parameters and the camera external calibration parameters using a three-dimensional reconstruction algorithm to obtain the panoramic environment three-dimensional grid model.

[0013] The present application proposes a method for three-dimensional reconstruction of indoor environment driven by panoramic image, which acquires a panoramic multi-angle environmental image of the target indoor area through panoramic camera acquisition, divides the panoramic multi-angle environmental image into functional areas, acquires a set of indoor functional areas, and encodes and identifies the set of indoor functional areas to obtain a set of indoor coded areas. The indoor coded area set is analyzed for acquisition requirements and optimized for camera layout to obtain a set of regional micro-camera arrays, and local cameras are arranged for the target indoor area according to the set of regional micro-camera arrays. Based on the set of regional micro-camera arrays, local images of the indoor coded area set are respectively acquired to obtain a set of regional local images, and feature points are extracted from the set of regional local images to obtain a set of regional image feature points. The panoramic multi-angle environmental image and the set of regional image feature points are registered with panoramic feature points and reconstructed into a three-dimensional model by a three-dimensional reconstruction algorithm to generate a three-dimensional model of the panoramic indoor environment. The technical problem of high reconstruction cost and complex operation of the three-dimensional reconstruction method of indoor environment in the prior art is solved. High-precision three-dimensional reconstruction of complex indoor environment is achieved, and the cost and operation difficulty of three-dimensional indoor environment are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the accompanying drawings of the embodiment of the present disclosure will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0015] Figure 1 A schematic diagram of a flow chart of a method for three-dimensional reconstruction of an indoor environment driven by a panoramic image provided in an embodiment of the present application;

[0016] Figure 2 A schematic diagram of a flow chart of obtaining a set of indoor coding areas by a panoramic image-driven three-dimensional reconstruction method of an indoor environment provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0019] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0020] The present application embodiment provides a method for reconstructing a three-dimensional indoor environment driven by a panoramic image. Figure 1 As shown, the method includes:

[0021] A panoramic multi-angle environmental image of a target indoor area is acquired through a panoramic camera, the panoramic multi-angle environmental image is divided into functional areas, a set of indoor functional areas is acquired, and the set of indoor functional areas is coded and identified to obtain a set of indoor coded areas. The indoor coded area set is analyzed for acquisition requirements and optimized for camera layout to obtain a set of regional micro-camera arrays, and local cameras are deployed in the target indoor area according to the set of regional micro-camera arrays.

[0022] Use a panoramic camera to collect images from multiple angles of the target indoor area. The panoramic camera should be able to cover all angles of the room. The panoramic multi-angle environmental image includes panoramic environmental images of the front angle, the top angle, and the side angle of the indoor environment to ensure that there is no blind spot shooting. Subsequently, the panoramic multi-angle environmental image is divided into functional areas to obtain a set of indoor functional areas. When dividing the functional areas, the top-view angle image can be used to mark the functional attributes of each functional area. According to the actual usage, the attributes of each functional area are classified, such as living room, bedroom, kitchen, etc., to generate regional functional attribute information. The indoor functional area set is coded and identified to obtain an indoor coded area set. The indoor coded area set is analyzed for acquisition requirements and optimized for camera layout to obtain a regional micro-camera array set, which is a specific layout array of micro-cameras, including the layout position, layout number, and layout angle of micro-cameras, and the local camera layout of the target indoor area is performed according to the regional micro-camera array set.

[0023] like Figure 2 As shown, the method provided in the embodiment of the present application also includes:

[0024] The panoramic multi-angle environmental image is grid-divided to obtain panoramic distribution grid image information, and the panoramic distribution grid image information is grid-numbered to obtain panoramic image grid number information. The indoor functional area set is matched and mapped with the panoramic image grid number information to determine the indoor area area information. The indoor functional area set is spatially resolved and functional attribute marked to obtain the regional spatial position sequence relationship and regional functional attribute information. The regional spatial position sequence relationship and regional functional attribute information as well as the indoor area area information are used as coding elements for coding design identification to determine the indoor coding area set.

[0025] The panoramic image is divided according to a preset grid size to generate panoramic distribution grid image information. The divided panoramic distribution grid image information is numbered according to the grid position arrangement order. Each cell number is unique, which is convenient for subsequent processing to generate panoramic image grid number information. Further, the indoor functional area set is matched and mapped with the panoramic image grid number information to determine the indoor area area information corresponding to the indoor functional area. The indoor area area information is obtained based on the coverage number of the preset grid size contained in the overhead image grid in the panoramic image grid number information corresponding to the indoor functional area. The coverage number can be a decimal. For example, if there is a half coverage grid in the preset grid size, the coverage number is 0.5. The indoor functional area set is spatially resolved and functional attribute marked. The spatial position is the functional area and connection position connected to the indoor functional area. For example, the connection area of ​​the toilet includes the bedroom and is located in the due east direction of the toilet. The spatial position of the toilet includes the bedroom and the connection position is due east. The regional spatial position sequence relationship, that is, the position connection relationship and regional functional attribute information of each area, is obtained. The spatial position sequence relationship of the region, the regional functional attribute information, and the indoor area area information are used as coding elements for coding design identification. The coding elements are preset coding methods. For example, the east, south, west, and north of the connection position are set to 1, 2, 3, and 4 respectively. For example, if the spatial position of the toilet includes the bedroom and the connection position, that is, the east, the coding of the spatial position sequence relationship of the region in the toilet functional area is: bedroom: 1. Specifically, each functional area can also be set to a specific coding letter. For example, if the bedroom is set to A, the coding of the spatial position sequence relationship of the region in the toilet functional area is A1, and the indoor coding area set is determined.

[0026] The method provided in the embodiment of the present application also includes:

[0027] Obtain camera requirement layout factor information, wherein the camera requirement layout factor information includes resolution requirements, viewing angle requirements, and spatial object distribution. Perform image acquisition requirement analysis on the indoor coding area set based on the camera requirement layout factor information to obtain a regional image acquisition requirement parameter set. Perform camera selection based on the regional image acquisition requirement parameter set to obtain a regional camera requirement model parameter set. Perform camera layout analysis on the regional image acquisition requirement parameter set to obtain a camera layout solution parameter set, and determine the regional micro camera array set based on the regional camera requirement model parameter set and the camera layout solution parameter set.

[0028] The obtaining of the regional micro camera array set includes: obtaining camera layout factor information, wherein the camera layout factor information includes resolution requirements, viewing angle requirements, and spatial object distribution. The resolution requirement is the requirement for image resolution. High resolution can provide more detailed image details and is suitable for reconstruction of important areas. High-resolution cameras can be arranged at important area acquisition positions to better acquire images of important areas.

[0029] The viewing angle requirement is the size of the acquisition viewing angle. The larger the area, the smaller the corresponding acquisition viewing angle, and the smaller the area, the larger the acquisition viewing angle, so as to ensure that the camera can cover the required area. The spatial object distribution is the distribution of objects in the indoor environment, and determines the position of obstacles that need to be avoided when the camera is deployed. The spatial object distribution information helps to arrange the camera reasonably to avoid occlusion and reflection. Based on the camera demand layout factor information, the image acquisition requirements of the indoor coding area set are respectively analyzed. When performing the image acquisition requirement analysis, the coding information of each area is analyzed according to the indoor coding area set, and the function and area of ​​each area in the indoor coding area set are obtained. Based on the function and area of ​​each area, the corresponding resolution requirements and viewing angle requirements are matched, so as to obtain the image acquisition requirement parameter set of each area. The regional image acquisition requirement parameter set contains the resolution requirements for the regional image, the acquisition viewing angle requirements and the spatial object distribution. The spatial object distribution is the various types of objects contained in the area, and the spatial object distribution can be obtained by manual registration. The camera is selected based on the regional image acquisition requirement parameter set, that is, the camera is selected according to the resolution requirement in the regional image acquisition requirement parameter set, and a suitable camera model is matched to obtain a regional camera requirement model parameter set, wherein the regional camera requirement model parameter set contains the acquisition camera selection model corresponding to each region. The camera layout analysis is performed on the regional image acquisition requirement parameter set to obtain a camera layout scheme parameter set, and the regional micro camera array set is determined based on the regional camera requirement model parameter set and the camera layout scheme parameter set.

[0030] The method provided in the embodiment of the present application also includes:

[0031] Based on the regional image acquisition requirement parameter set, a coverage analysis is performed to obtain an initial camera layout parameter set, wherein the initial camera layout parameter set includes camera position, angle, and quantity. The initial camera layout parameter set is simulated to obtain a camera layout simulation result, and the camera layout simulation result is evaluated and verified to obtain a camera layout acquisition quality parameter. Based on the camera layout acquisition quality parameter, an optimization strategy analysis is performed on the initial camera layout parameter set to determine a camera layout optimization improvement strategy. Based on the camera layout optimization improvement strategy, the initial camera layout parameter set is iteratively improved and optimized to obtain the regional micro camera array set.

[0032] The acquisition of the camera layout scheme parameters includes: performing coverage analysis based on the regional image acquisition requirement parameter set to acquire an initial camera layout parameter set, wherein the initial camera layout parameter set includes camera positions, angles, and quantities. The camera position in the acquisition of the initial camera layout parameter set is selected as a position that avoids obstacles and is close to the center of the region. Since the center of the region can cover the largest area, the number of cameras can be reduced, and the camera's viewing angle should be cross-covered as much as possible to reduce blind spots and dead angles. The angle is acquired based on the viewing angle requirement. Taking a viewing angle of 60 degrees as an example, at least 7 cameras need to be set near the center to meet the viewing angle requirement and the viewing angle cross-coverage requirement as much as possible. Finally, based on the spatial object distribution, it is acquired whether there are objects in the space that need to be multi-faceted, such as three-dimensional objects such as tables, and at least two cameras are allocated to the corresponding objects for shooting. The shooting position is the position where the object covers the largest area, thereby obtaining an initial camera layout parameter set.

[0033] Further, the initial camera layout parameter set is simulated. When performing the simulation, the architectural plan and three-dimensional spatial data of the indoor environment are used to establish a simulated three-dimensional model of the indoor environment, and the initial camera layout parameter set is imported into the simulation software, including the position, angle and number of each camera. The field of view of each camera is projected into the three-dimensional environment model to generate a field of view coverage map. The field of view coverage map shows the coverage range of each camera. Based on the range covered by the field of view of each camera, the coverage range is marked in the three-dimensional environment model, and the coverage areas of all cameras are superimposed to generate a comprehensive coverage map. The comprehensive coverage map shows the area covered by all cameras together. The comprehensive coverage map is analyzed to detect uncovered blind spots. Blind spots refer to areas that are not covered by any camera field of view. The location and area of ​​the blind spots are recorded for subsequent optimization. Subsequently, the camera layout simulation results are evaluated and verified, and the camera layout acquisition quality parameters are obtained based on the ratio of the area of ​​the blind spot to the total area of ​​the three-dimensional environment model. Based on the camera layout acquisition quality parameter, an optimization strategy analysis is performed on the initial camera layout parameter set to determine a camera layout optimization improvement strategy, wherein the camera layout optimization improvement strategy is an increase in the number of cameras. There is a corresponding relationship between the camera layout optimization improvement strategy and the camera layout acquisition quality parameter. The larger the camera layout acquisition quality parameter, the greater the corresponding increase in the number of cameras.

[0034] Based on the camera layout optimization and improvement strategy, the initial camera layout parameter set is iteratively improved and optimized. When performing the iterative improvement and optimization, the first camera position is randomly arranged in the three-dimensional environment model based on the position and area of ​​the blind spot. When the camera's field of view projection intersects with the position of the blind spot, the camera position is fixed, and the camera scheduling adjustment is performed until the camera can cover the blind spot area to a maximum value to obtain the camera position and camera angle. If there are multiple additional cameras, after the first camera position is obtained, the second camera position is iterated for the remaining blind spots until all cameras are iterated, the camera position and camera angle are obtained, and the camera layout optimization and improvement strategy is determined.

[0035] Based on the regional micro-camera array set, local images of the indoor coding area set are respectively collected to obtain a regional local image set, and feature points are extracted from the regional local image set to obtain a regional image feature point set. A three-dimensional reconstruction algorithm is used to perform panoramic feature point registration and three-dimensional model reconstruction on the panoramic multi-angle environment image and the regional image feature point set to generate a panoramic indoor environment three-dimensional stereo model.

[0036] Based on the regional micro-camera array set, local images of the indoor coding area set are respectively collected to obtain a regional local image set. When the regional image feature point set is obtained, the regional local image set is geometrically corrected to eliminate camera distortion and perspective deformation, ensure that the geometric shape of the image is consistent with the actual environment, and use the cv2.undistort() function in OpenCV to perform geometric correction, thereby eliminating camera distortion. Further, the image is sharpened to enhance edge details, improve the detection accuracy of feature points, and adjust the image contrast to highlight important features in the image and improve the recognizability of feature points. The processed image is processed based on a feature point extraction algorithm, and the preferred algorithm is the SIFT algorithm. Feature points are extracted by a feature recognition algorithm to obtain a regional image feature point set. Finally, the panoramic multi-angle environment image and the regional image feature point set are panoramically registered and reconstructed by a three-dimensional model using a three-dimensional reconstruction algorithm to generate a panoramic indoor environment three-dimensional stereo model. The technical problem that the indoor environment three-dimensional reconstruction method in the prior art has a high reconstruction cost and complex operation is solved. High-precision three-dimensional stereo reconstruction of complex indoor environments is achieved, and the cost and difficulty of operation of indoor environments are reduced.

[0037] The method provided in the embodiment of the present application also includes:

[0038] Based on the regional micro-camera array set, local images are respectively collected for the indoor coding area set to obtain a regional collection image set. The regional collection image set is subjected to spectrum analysis and noise identification by Fourier transform to obtain image noise characteristic information. The regional collection image set is subjected to filtering and denoising based on the image noise characteristic information to obtain an available regional image set. The available regional image set is sequentially integrated according to the viewing angle number information of the regional micro-camera array set to obtain the regional local image set.

[0039] The method of obtaining a regional local image set includes: arranging micro cameras in each indoor coding area based on the regional micro camera array set, ensuring that the viewing angle and position of each camera conform to the initial layout plan. Performing local image acquisition on the indoor coding area set respectively to obtain a regional acquisition image set. Using Fourier transform to convert the regional acquisition image set from the spatial domain to the frequency domain, performing Fourier transform on each acquisition image, and generating a corresponding spectrum diagram. The spectrum diagram shows the various frequency components in the image, which is helpful for analyzing the noise characteristics in the image. Analyzing the noise characteristic information in the spectrum diagram, the noise is usually manifested as a high-frequency component, which can be clearly identified in the spectrum diagram. Recording the frequency range and intensity of the noise, obtaining the image noise characteristic information, and providing a basis for subsequent filtering and denoising processing. Selecting a suitable filter type according to the image noise characteristic information. Commonly used filters include low-pass filters, high-pass filters, band-pass filters, and band-stop filters. The parameters of the filter, such as cutoff frequency, bandwidth, etc., are set by the selected filter to filter the image and obtain the available regional image set. Finally, the available regional image set is sequentially integrated according to the viewing angle number information of the regional micro camera array set, where the viewing angle number information is the camera number information, that is, the micro cameras and the corresponding regional images are matched to obtain the regional local image set.

[0040] The method provided in the embodiment of the present application also includes:

[0041] The regional image feature point set is sequentially described by multi-dimensional feature points to determine a regional image feature point description subset. The regional image feature point set is similarly registered based on the regional image feature point description subset to obtain a regional image registration feature point set. The regional image feature point set is subjected to feature point mapping fusion through the regional image registration feature point set to obtain a regional panoramic fusion image. The regional panoramic fusion image is rendered into the panoramic multi-angle environment image using a three-dimensional reconstruction algorithm to perform three-dimensional model reconstruction to generate a three-dimensional stereo model of the panoramic indoor environment.

[0042] The method of generating a panoramic indoor environment three-dimensional stereo model includes: sequentially performing multi-dimensional feature point description on the regional image feature point set, and determining a regional image feature point descriptor set. That is, generating a descriptor for each detected feature point, and these descriptors are high-dimensional vectors, which contain local image information around the feature point. Through the descriptors, feature points can be matched between different images. Based on the regional image feature point descriptor set, the regional image feature point set is similarly registered, that is, the feature point descriptor sets of two or more images are compared to find the feature point pairs with the highest similarity. A matching algorithm based on distance metric, such as Euclidean distance or Hamming distance, can be used to calculate the similarity between descriptors, and the most reliable feature point matching pairs can be screened out through certain filtering rules, such as distance threshold and bidirectional matching, to ensure the accuracy and stability of the matching results, and obtain a regional image registration feature point set. Feature point mapping fusion is performed on the regional image feature point set through the regional image registration feature point set, that is, according to the feature point matching results, the transformation matrix between the images is calculated to map the feature points in one image to the coordinate system of another image. The calculated transformation matrix is ​​used to geometrically transform and stitch the images, and multiple images are fused into a panoramic image. During the fusion process, weighted averaging and smoothing of image edges can be used to reduce seam marks, thereby obtaining a regional panoramic fusion image. The regional panoramic fusion image is rendered into the panoramic multi-angle environment image using a three-dimensional reconstruction algorithm to perform three-dimensional model reconstruction, thereby generating a three-dimensional stereoscopic model of the panoramic indoor environment.

[0043] The method provided in the embodiment of the present application also includes:

[0044] The panoramic multi-angle environment image is surface reconstructed using a three-dimensional reconstruction algorithm to obtain a panoramic environment three-dimensional grid model. The panoramic environment three-dimensional grid model is edge detected based on the indoor coding area set to extract edge information of the model coding area. The regional panoramic fusion image is mapped and rendered onto the panoramic environment three-dimensional grid model according to the edge information of the model coding area to generate the panoramic indoor environment three-dimensional stereo model.

[0045] The panoramic multi-angle environment image is surface reconstructed using a three-dimensional reconstruction algorithm to obtain a panoramic environment three-dimensional grid model. The panoramic environment three-dimensional grid model is edge detected based on the indoor coding area set to extract the edge information of the model coding area. Based on the indoor coding area set, the corresponding area is extracted from the panoramic environment three-dimensional grid model, and the edge detection algorithm such as Canny edge detection and Sobel operator is used to perform edge detection on the extracted area to extract the edge information of the model coding area. The color information of the panoramic fusion image is mapped to the surface of the three-dimensional grid model to generate a three-dimensional stereo model with texture. During the texture mapping process, the coordinates of the image and the three-dimensional model need to be accurately aligned to ensure the correct projection of the texture, thereby generating the panoramic indoor environment three-dimensional stereo model.

[0046] The method provided in the embodiment of the present application also includes: extracting key feature points and performing stereo matching on the panoramic multi-angle environment image to obtain a multi-angle image matching feature point set. Calibrate the camera parameters of the panoramic multi-angle environment image using a camera calibration algorithm to determine the camera internal calibration parameters and the camera external calibration parameters. Reconstruct the point cloud surface of the multi-angle image matching feature point set and the camera internal calibration parameters and the camera external calibration parameters using a three-dimensional reconstruction algorithm to obtain the panoramic environment three-dimensional grid model.

[0047] The method of obtaining a panoramic environment three-dimensional grid model includes: extracting key feature points and performing stereo matching on the panoramic multi-angle environment image, obtaining a multi-angle image matching feature point set, that is, a matching relationship between feature points between images. The camera calibration algorithm is used to calibrate the camera parameters of the panoramic multi-angle environment image, and the camera internal calibration parameters and camera external calibration parameters are determined. The three-dimensional reconstruction algorithm is used to reconstruct the point cloud surface of the multi-angle image matching feature point set and the camera internal calibration parameters and camera external calibration parameters, that is, according to the camera calibration parameters and the position of the matching feature points, the triangulation method is used to calculate the coordinates of each feature point in the three-dimensional space, and the feature points are expanded to a dense point cloud through a multi-view stereo (MVS) algorithm to include more three-dimensional points, and the point cloud surface is reconstructed, and the Poisson surface reconstruction algorithm is used to convert the point cloud into a three-dimensional grid model. The grid model is composed of a series of connection points to form a continuous three-dimensional surface, and the panoramic environment three-dimensional grid model is obtained.

[0048] The technical solution provided by the embodiment of the present invention uses a panoramic camera to acquire a panoramic multi-angle environmental image of the target indoor area for functional area division, obtains a set of indoor functional areas and performs coding and identification, and obtains an indoor coding area set. The indoor coding area set is subjected to acquisition demand analysis and camera layout optimization to obtain a regional micro-camera array set, and then local camera layout is performed. Local images are acquired to obtain a regional local image set, and feature points are extracted from the regional local image set to obtain a regional image feature point set. The panoramic feature point registration and three-dimensional model reconstruction are performed on the panoramic multi-angle environmental image and the regional image feature point set using a three-dimensional reconstruction algorithm to generate a panoramic indoor environment three-dimensional stereo model. The technical problem of high reconstruction cost and complex operation of the indoor environment three-dimensional reconstruction method in the prior art is solved. High-precision three-dimensional reconstruction of complex indoor environments is achieved, and the cost and operation difficulty of three-dimensional indoor environments are reduced.

[0049] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A panoramic image driven 3D reconstruction method for indoor environments. It is characterized in that The method comprises: Acquire a panoramic multi-angle environmental image of a target indoor area through a panoramic camera, divide the panoramic multi-angle environmental image into functional areas, obtain an indoor functional area set, and encode and identify the indoor functional area set to obtain an indoor coded area set; Performing collection demand analysis and camera layout optimization on the indoor coding area set to obtain a regional micro camera array set, and performing local camera layout on the target indoor area according to the regional micro camera array set; Based on the regional micro-camera array set, local images of the indoor coding area set are respectively collected to obtain a regional local image set, and feature points are extracted from the regional local image set to obtain a regional image feature point set; A three-dimensional reconstruction algorithm is used to perform panoramic feature point registration and three-dimensional model reconstruction on the panoramic multi-angle environment image and the regional image feature point set to generate a panoramic indoor environment three-dimensional stereo model.

2. A method for three-dimensional reconstruction of an indoor environment driven by a panoramic image as claimed in claim 1, It is characterized in that The obtaining of the indoor coding area set includes: Gridding the panoramic multi-angle environmental image to obtain panoramic distribution grid image information, grid numbering the panoramic distribution grid image information to obtain panoramic image grid numbering information; Perform matching mapping analysis on the indoor functional area set and the panoramic image grid number information to determine the indoor area area information; Performing spatial position analysis and functional attribute marking on the indoor functional area set to obtain regional spatial position sequence relationship and regional functional attribute information; The spatial position sequence relationship of the region, the regional functional attribute information and the indoor area area information are used as coding elements for coding design identification to determine the indoor coding area set.

3. The method for three-dimensional reconstruction of indoor environment driven by panoramic image according to claim 1, It is characterized in that The method of obtaining a regional micro camera array set comprises: Obtaining camera layout factor information, wherein the camera layout factor information includes resolution requirements, viewing angle requirements, and spatial object distribution; Based on the camera requirement layout factor information, image acquisition requirements are analyzed for the indoor coding area set to obtain an area image acquisition requirement parameter set; Selecting a camera based on the regional image acquisition requirement parameter set to obtain a regional camera requirement model parameter set; Perform camera layout analysis on the regional image acquisition requirement parameter set to obtain a camera layout scheme parameter set, and determine the regional micro camera array set based on the regional camera requirement model parameter set and the camera layout scheme parameter set.

4. A method for three-dimensional reconstruction of an indoor environment driven by a panoramic image as claimed in claim 3, It is characterized in that The step of obtaining the camera layout scheme parameters includes: Perform coverage analysis based on the regional image acquisition requirement parameter set to obtain an initial camera layout parameter set, wherein the initial camera layout parameter set includes camera positions, angles, and quantities; Simulating the initial camera layout parameter set to obtain a camera layout simulation result, and evaluating and verifying the camera layout simulation result to obtain a camera layout acquisition quality parameter; Performing an optimization strategy analysis on the initial camera layout parameter set based on the camera layout acquisition quality parameter to determine a camera layout optimization improvement strategy; The initial camera layout parameter set is iteratively improved and optimized based on the camera layout optimization improvement strategy to obtain the regional micro camera array set.

5. The method for three-dimensional reconstruction of indoor environment driven by panoramic images as claimed in claim 1, It is characterized in that The obtaining of a regional local image set comprises: Based on the regional micro camera array set, local image acquisition is performed on the indoor coding area set respectively to obtain a regional acquisition image set; Using Fourier transform to perform spectrum analysis and noise identification on the image set collected in the region to obtain image noise characteristic information; Performing filtering and denoising processing on the regional collected image set based on the image noise characteristic information to obtain a usable regional image set; The available regional image set is sequentially integrated according to the viewing angle number information of the regional micro camera array set to obtain the regional local image set.

6. The method for three-dimensional reconstruction of indoor environment driven by panoramic image according to claim 1, It is characterized in that The generating of the panoramic indoor environment three-dimensional stereo model comprises: Performing multi-dimensional feature point description on the regional image feature point set in turn to determine a regional image feature point description subset; Performing similarity registration on the regional image feature point set based on the regional image feature point descriptor set to obtain a regional image registration feature point set; Performing feature point mapping fusion on the regional image feature point set through the regional image registration feature point set to obtain a regional panoramic fusion image; The regional panoramic fusion image is rendered into the panoramic multi-angle environment image by using a three-dimensional reconstruction algorithm to perform three-dimensional model reconstruction, thereby generating a three-dimensional stereoscopic model of the panoramic indoor environment.

7. A method for three-dimensional reconstruction of indoor environment driven by panoramic images as claimed in claim 6, It is characterized in that The generating of the panoramic indoor environment three-dimensional stereoscopic model comprises: Reconstructing the surface of the panoramic multi-angle environment image using a three-dimensional reconstruction algorithm to obtain a three-dimensional grid model of the panoramic environment; Performing edge detection on the panoramic environment three-dimensional grid model based on the indoor coding area set to extract edge information of the model coding area; The regional panoramic fusion image is mapped and rendered onto the panoramic environment three-dimensional grid model according to the model encoding area edge information to generate the panoramic indoor environment three-dimensional stereo model.

8. The method for three-dimensional reconstruction of indoor environment driven by panoramic images as claimed in claim 7, It is characterized in that The method of obtaining a panoramic environment three-dimensional grid model comprises: Extract key feature points and perform stereo matching on the panoramic multi-angle environmental image to obtain a multi-angle image matching feature point set; Performing camera parameter calibration on the panoramic multi-angle environment image by using a camera calibration algorithm to determine camera internal calibration parameters and camera external calibration parameters; A three-dimensional reconstruction algorithm is used to reconstruct a point cloud surface of the multi-angle image matching feature point set, the camera internal calibration parameters, and the camera external calibration parameters to obtain the panoramic environment three-dimensional grid model.

Citation Information

Patent Citations

  • Video monitoring facility layout method for indoor three-dimensional scene

    CN113489952A

  • Substation safety early warning method based on target three-dimensional distance measurement

    CN117875719A

  • Dynamic traffic information acquisition method and system based on binocular stereoscopic vision

    CN118262511A

  • Three-dimensional reconstruction method and system, and storage medium

    WO2023078335A1