Indoor three-dimensional effect acquisition method based on multi-view image

Abstract

The invention discloses an indoor three-dimensional effect acquisition method based on a multi-view image, and relates to the technical field of image data processing. The invention discloses an indoor three-dimensional effect acquisition method based on a multi-view image. The method comprises the steps of acquisition preparation, data acquisition, data preprocessing, three-dimensional reconstruction and data fusion, texture mapping and model optimization and output. According to the method, the dense point cloud model obtained by the photogrammetry method and the point cloud model obtained by laser scanning are registered, aligned and fused, so that the reconstruction precision and quality of the final three-dimensional model on reflective and transparent objects and smooth and texture-free surfaces can be improved, and noise and errors caused by a single method can be reduced; therefore, the robustness of the whole three-dimensional model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method for acquiring indoor three-dimensional effects based on multi-viewing angle images. Background Art

[0002] Indoor 3D model reconstruction refers to the process of restoring the 3D geometric structure of rooms, furniture and other objects from 2D images of indoor environments or other data sources. This technology has a wide range of applications in interior design, virtual reality, smart home, real estate display and other fields.

[0003] The indoor 3D effect acquisition method based on multi-view images, also known as photogrammetry, is a technology that reconstructs 3D models by taking 2D photos from multiple angles and using computer vision technology. This method is very common in indoor 3D modeling because it is relatively low-cost and highly flexible, but photogrammetry has difficulty extracting and matching enough feature points when photographing reflective and transparent objects such as glass and mirrors, as well as smooth and textureless surfaces such as white walls and solid-color floors, resulting in low coverage of feature points and poor results in indoor 3D model reconstruction.

[0004] Based on the above situation, the present invention proposes a method for acquiring indoor three-dimensional effects based on multi-view images with good reconstruction effect. Summary of the invention

[0005] In order to overcome the shortcomings of the existing photogrammetry method that it is difficult to extract and match enough feature points when shooting reflective and transparent objects such as glass and mirrors and smooth and textureless surfaces such as white walls and solid-color floors, resulting in low coverage of feature points and poor quality of indoor three-dimensional effect acquisition, and poor effect of three-dimensional model reconstruction, the present invention proposes an indoor three-dimensional effect acquisition method based on multi-view images with good reconstruction effect.

[0006] The method for acquiring indoor three-dimensional effects based on multi-view images includes the following steps: Step 1: Collection Preparation Select high-resolution cameras and high-precision laser scanners to obtain indoor two-dimensional floor plans, thereby obtaining the length of each room and corridor area. He Kuan , and then evenly place them in each room and corridor area according to the placement criteria Mark points and develop a shooting path; Step 2: Data Collection According to the shooting and scanning guidelines and along the shooting path, multiple photos covering all indoor areas are taken from different heights, angles and positions. During this period, a laser scanner is used to perform a full-scale scan of the indoor area along the scanning station on the shooting path; Step 3: Data preprocessing Preprocess the photos, extract feature points in the photos using the improved SIFT algorithm, and calibrate the camera using the landmarks to obtain the camera's internal and external parameters. Step 4: 3D reconstruction and data fusion Use the SFM algorithm to match the taken photos, and use the matched feature points and camera parameters to generate a sparse point cloud. Then use the MVS algorithm to generate a dense point cloud model. Import the laser scanning data of all scanning stations into the 3D modeling software, align and splice the scanning data of each station to form a complete point cloud model. Then, register and align the two point cloud models to ensure that the alignment error between the two point cloud models reaches the predetermined threshold. The two aligned point cloud models are merged into a final point cloud model. Finally, the final point cloud model is converted into a 3D mesh model and the model surface is generated. Step 5: Texture Mapping The high-resolution photos taken by the camera are used as texture maps and mapped onto the 3D mesh model to generate a 3D mesh model with texture, and then the texture on the 3D mesh model is color corrected; Step 6: Model optimization and output A filtering algorithm is used to remove noise points in the 3D mesh model, and then the surface of the 3D mesh model is smoothed by Laplacian smoothing. After the optimization is completed, the final 3D mesh model is exported to a commonly used format.

[0007] Preferably, the strategy for formulating the shooting path in step 1 is: Determine the width of each room and corridor area in the room Is it greater than ,like , a straight line path is used, that is, from the entrance to the farthest corner; if , using a ring path or a hub-and-spoke path; If the layout of the room or corridor is complex, a central radial path is used, that is, starting from the center and shooting radiates to the surrounding areas; if the layout of the room or corridor is simple, a circular path is used, that is, walking around the room or corridor area and shooting intermittently along the way.

[0008] Preferably, the placement standard of the marker points in step 1 is specifically: ,in represents the floor space of the room or corridor area, and , It represents the placement density of the landmark points. And when hour, ; when hour, ; when hour, .

[0009] Preferably, the shooting and scanning criteria in step 2 include: Evenly arrange shooting points and scanning stations on the shooting path to ensure that the photos taken by adjacent shooting points have an overlap of more than 60%, and that there is an overlap area of ​​10%-20% between adjacent scanning stations; Shoot from multiple angles at each corner and key object, including the front, side, and top, to fully capture the structure at the corner and obtain multi-view images of each key object; Use a wide-angle lens to shoot the ceiling and floor and ensure the integrity of the shot.

[0010] Preferably, the photo preprocessing step in step 3 includes: Correct the photos taken to eliminate lens distortion; Enhance the local contrast of the photo through local contrast enhancement technology, making the subtle structure of the surface of the object in the photo more obvious; Use the Canny edge detection algorithm to enhance edge information and improve the detection effect of feature points.

[0011] Preferably, the SIFT algorithm in step 3 is a feature detection and descriptor generation method used in the field of image processing and computer vision, which has scale invariance and rotation invariance, and can stably detect and match feature points under different viewing angles, lighting conditions and scales. The SIFT algorithm is mainly divided into four steps, namely, scale space extreme value detection, key point positioning, direction assignment and key point descriptor generation, and the specific steps of the improved SIFT algorithm are: Scale space extreme value detection: First, the image is processed with Gaussian blur multiple times to generate images of different scales, where each scale image is called an octave, and each octave contains multiple scale layers. Then, potential key points are detected by the difference between adjacent scale layers. The differential image can approximately represent the scale space of the image. At each pixel point in the differential image, it is compared with the 8 pixels in its neighborhood and the corresponding pixels of the previous scale layer and the next scale layer to find the local extreme value points. These extreme value points are candidate key points. Key point location: Accurately locate the position and scale of key points by fitting a three-dimensional quadratic function, remove noise key points with low response values, and eliminate edge response points with poor stability at different scales by calculating the principal curvature ratio of the Hessian matrix; Direction assignment: Calculate the gradient direction and gradient magnitude in the neighborhood of each key point, generate a gradient direction histogram, and determine the main direction of the key point based on the peak value of the histogram. If the gradient magnitude of a certain direction exceeds 80% of the main direction, then the direction is also considered to be an important direction. Then assign one or more directions to each key point so that the key point has rotation invariance. Keypoint descriptor fusion: At each keypoint In the neighborhood of the key point, the key point is taken as the center, rotated according to the direction of the key point, and then divided into 16 sub-regions, and in each sub-region the gradient amplitudes in 8 directions are calculated to obtain 16 8-dimensional SIFT descriptors. At the same time, an 8-dimensional LBP feature is extracted in each sub-region through the LBP algorithm. For each sub-region, the SIFT descriptor and the LBP feature are concatenated together to form a 16-dimensional feature vector. Then, the feature vectors of the 16 sub-regions are concatenated into a 256-dimensional complete feature vector. This feature vector is the final descriptor, which represents the local features at the key point.

[0012] Preferably, the step of extracting LBP features by using the LBP algorithm is: Will The neighborhood is divided into 16 sub-regions; For each pixel in the sub-area, calculate its LBP value; A histogram is generated based on the distribution of LBP values ​​in each sub-region, where the number of bins in the histogram is 8, and each bin represents the frequency of occurrence of a certain LBP value; The 8-bin LBP histogram is represented as a feature vector of length 8, which is the LBP feature of this sub-region.

[0013] Preferably, the steps for calculating the LBP value are: Establish the center pixel and obtain the grayscale value of the center pixel ; Select the pixel around the center Neighborhood pixels, this Neighborhood pixels are evenly distributed in a radius of On the circle of , and because the sub-region size is , so the corresponding and ; Get separately Gray value of neighboring pixels ,like , then assign 1, otherwise assign 0; Arrange the above assignment results in order as a The binary number of the bit is converted into a decimal number, which is the LBP value of the central pixel.

[0014] Preferably, the step of model registration in step 4 is: Initial registration: Roughly align the dense point cloud model generated by the MVS algorithm and the point cloud model obtained by laser scanning through the position information of the landmark points; Precise registration: The ICP algorithm is used to find the nearest corresponding point for each point in each pair of point clouds, and then the rotation and translation transformations are calculated based on the corresponding point pairs found. The calculated transformations are then applied and the point cloud positions are updated. This cycle is repeated until the alignment error of the two point cloud models reaches a predetermined threshold or the number of iterations reaches an upper limit.

[0015] Preferably, the step of smoothing the surface of the three-dimensional model by Laplacian smoothing in step six is: Import and read the 3D mesh model that needs to be optimized; Get the position of each vertex in the mesh , and establish its adjacent point structure set, that is, determine the neighbor vertex set of each vertex , calculate the Laplacian coordinates , the calculation formula is: ; Where N is the number of neighbor vertices; According to Laplacian coordinates Update the vertex position, the update formula is: ; in represents the updated vertex position, represents the smoothing factor and its value is 0.2; Repeat the smoothing steps as necessary until the desired degree of smoothness is achieved.

[0016] The present invention has the following advantages: 1. The present invention aligns and fuses the dense point cloud model obtained by photogrammetry and the point cloud model obtained by laser scanning, which can not only significantly improve the reconstruction accuracy and quality of the final three-dimensional model for reflective and transparent objects and smooth and textureless surfaces, but also reduce the noise and error caused by a single method, thereby improving the robustness of the overall three-dimensional model.

[0017] 2. The present invention optimizes the descriptor of the SIFT algorithm by combining the LBP feature, which not only increases the distinguishing ability of the descriptor, but also makes the descriptor more robust to illumination changes and noise, thereby significantly improving the improved SIFT algorithm's ability to extract feature points on reflective, smooth and transparent planes and objects, thereby effectively improving the reconstruction accuracy and quality of the final three-dimensional model for reflective and transparent objects and smooth and texture-free surfaces.

[0018] 3. The present invention obtains a two-dimensional indoor plan and formulates a reasonable shooting path according to the layout and size information of the room and corridor area, thereby shortening the shooting path while ensuring that the photographed photos can completely cover the indoor area, thereby facilitating the staff to quickly complete the shooting task in a short time, thereby improving the image acquisition efficiency of this indoor three-dimensional effect acquisition method. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The present invention is a flowchart of a method for acquiring indoor three-dimensional effects based on multi-view images used in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Embodiment 1, a method for acquiring indoor three-dimensional effects based on multi-view images, such as Figure 1 As shown, the following steps are included: Step 1: Collection Preparation Select high-resolution cameras and high-precision laser scanners to obtain indoor two-dimensional floor plans, thereby obtaining the length of each room and corridor area. He Kuan , and then evenly place them in each room and corridor area according to the placement criteria Mark points and develop a shooting path; Step 2: Data Collection According to the shooting and scanning guidelines and along the shooting path, multiple photos covering all indoor areas are taken from different heights, angles and positions. During this period, a laser scanner is used to perform a full-scale scan of the indoor area along the scanning station on the shooting path; Step 3: Data preprocessing Preprocess the photos, extract feature points in the photos using the improved SIFT algorithm, and calibrate the camera using the landmarks to obtain the camera's internal and external parameters. Step 4: 3D reconstruction and data fusion Use the SFM algorithm to match the taken photos, and use the matched feature points and camera parameters to generate a sparse point cloud. Then use the MVS algorithm to generate a dense point cloud model. Import the laser scanning data of all scanning stations into the 3D modeling software, align and splice the scanning data of each station to form a complete point cloud model. Then, register and align the two point cloud models to ensure that the alignment error between the two point cloud models reaches the predetermined threshold, and merge the two aligned point cloud models into a final point cloud model. Finally, convert the final point cloud model into a 3D mesh model and generate a model surface. In this way, the dense point cloud model obtained by photogrammetry and the point cloud model obtained by laser scanning are registered, aligned and fused. This can not only significantly improve the reconstruction accuracy and quality of the final 3D model for reflective and transparent objects as well as smooth and textureless surfaces, but also reduce the noise and errors caused by a single method, thereby improving the robustness of the overall 3D model. Step 5: Texture Mapping The high-resolution photos taken by the camera are used as texture maps and mapped onto the 3D mesh model to generate a 3D mesh model with texture, and then the texture on the 3D mesh model is color corrected; Step 6: Model optimization and output A filtering algorithm is used to remove noise points in the 3D mesh model, and then the surface of the 3D mesh model is smoothed by Laplacian smoothing. After the optimization is completed, the final 3D mesh model is exported to a commonly used format.

[0022] The strategy for formulating the shooting path in step 1 is: Determine the width of each room and corridor area in the room Is it greater than ,like , a straight line path is used, that is, from the entrance to the farthest corner; if , using a ring path or a hub-and-spoke path; If the layout of the room or corridor is complex, a central radial path is used, that is, starting from the center and shooting radiates in all directions; if the layout of the room or corridor is simple, a circular path is used, that is, walking around the room or corridor and shooting at intervals along the way. In this way, a two-dimensional indoor plan can be obtained, and a reasonable shooting path can be formulated according to the layout and size information of the room and corridor. While ensuring that the photographed photos can completely cover the indoor area, the shooting path is shortened, which makes it easier for staff to complete the shooting task quickly in a short time, thereby improving the image acquisition efficiency of this indoor three-dimensional effect acquisition method.

[0023] The placement standard of the marker points in step 1 is specifically: ,in represents the floor space of the room or corridor area, and , It represents the placement density of the landmark points. And when hour, ; when hour, ; when hour, .

[0024] The shooting and scanning criteria in step 2 include: Evenly arrange shooting points and scanning stations on the shooting path to ensure that the photos taken by adjacent shooting points have an overlap of more than 60%, and that there is an overlap area of ​​10%-20% between adjacent scanning stations; Shoot from multiple angles at each corner and key object, including the front, side, and top, to fully capture the structure at the corner and obtain multi-view images of each key object; Use a wide-angle lens to shoot the ceiling and floor and ensure the integrity of the shot.

[0025] For example: Suppose you are shooting in a living room with a relatively simple layout and a width of more than 5m, you can follow the steps below: Start from a corner of the living room, such as near the entrance; then move slowly along the wall, taking a photo every 1 meter or so, and ensuring that adjacent photos have more than 60% overlap; take a few more photos at each corner to ensure that the structure at the corner is fully captured; shoot from multiple angles at each key location (such as sofas, TV cabinets and windows), including the front, side and top; for large furniture (such as bookcases and wardrobes), open the door to shoot the internal structure; after shooting all areas, return to the starting point to check if there are any missed parts.

[0026] The steps of photo preprocessing in step 3 include: Correct the photos taken to eliminate lens distortion; Enhance the local contrast of the photo through local contrast enhancement technology, making the subtle structure of the surface of the object in the photo more obvious; Use the Canny edge detection algorithm to enhance edge information and improve the detection effect of feature points.

[0027] The SIFT algorithm in step 3 is a feature detection and descriptor generation method used in the field of image processing and computer vision. It has scale invariance and rotation invariance, and can stably detect and match feature points under different viewing angles, lighting conditions and scales. The SIFT algorithm is mainly divided into four steps, namely scale space extreme value detection, key point location, direction assignment and key point descriptor generation, and the specific steps of the improved SIFT algorithm are: Scale space extreme value detection: First, the image is processed with Gaussian blur multiple times to generate images of different scales, where each scale image is called an octave, and each octave contains multiple scale layers. Then, potential key points are detected by the difference between adjacent scale layers. The differential image can approximately represent the scale space of the image. At each pixel point in the differential image, it is compared with the 8 pixels in its neighborhood and the corresponding pixels of the previous scale layer and the next scale layer to find the local extreme value points. These extreme value points are candidate key points. Key point location: Accurately locate the position and scale of key points by fitting a three-dimensional quadratic function, remove noise key points with low response values, and eliminate edge response points with poor stability at different scales by calculating the principal curvature ratio of the Hessian matrix; Direction assignment: Calculate the gradient direction and gradient magnitude in the neighborhood of each key point, generate a gradient direction histogram, and determine the main direction of the key point based on the peak value of the histogram. If the gradient magnitude of a certain direction exceeds 80% of the main direction, then the direction is also considered to be an important direction. Then assign one or more directions to each key point so that the key point has rotation invariance. Keypoint descriptor fusion: At each keypoint In the neighborhood of the key point, the key point is taken as the center, rotated according to the direction of the key point, and then divided into 16 sub-regions, and in each sub-region the gradient amplitudes in 8 directions are calculated, thereby obtaining 16 8-dimensional SIFT descriptors, and at the same time an 8-dimensional LBP feature is extracted in each sub-region through the LBP algorithm. For each sub-region, the SIFT descriptor and the LBP feature are spliced ​​together to form a 16-dimensional feature vector, and then the feature vectors of the 16 sub-regions are spliced ​​into a 256-dimensional complete feature vector, which is the final descriptor. The final descriptor represents the local features at the key point. In this way, the descriptor of the SIFT algorithm is optimized by combining the LBP feature, which not only increases the distinguishing ability of the descriptor, but also makes the descriptor have better robustness to illumination changes and noise, thereby significantly improving the feature point extraction capability of the improved SIFT algorithm on reflective, smooth and transparent planes and objects, and effectively improving the reconstruction accuracy and quality of the final 3D model for reflective and transparent objects as well as smooth and textureless surfaces.

[0028] The steps of extracting LBP features through the LBP algorithm are: Will The neighborhood is divided into 16 sub-regions; For each pixel in the sub-area, calculate its LBP value; A histogram is generated based on the distribution of LBP values ​​in each sub-region, where the number of bins in the histogram is 8, and each bin represents the frequency of occurrence of a certain LBP value; The 8-bin LBP histogram is represented as a feature vector of length 8, which is the LBP feature of this sub-region.

[0029] The calculation steps of LBP value are as follows: Establish the center pixel and obtain the grayscale value of the center pixel ; Select the pixel around the center Neighborhood pixels, this Neighborhood pixels are evenly distributed in a radius of On the circle of , and because the sub-region size is , so the corresponding and ; Get separately Gray value of neighboring pixels ,like , then assign 1, otherwise assign 0; Arrange the above assignment results in order as a The binary number of the bit is converted into a decimal number, which is the LBP value of the central pixel.

[0030] The steps of model registration in step 4 are: Initial registration: Roughly align the dense point cloud model generated by the MVS algorithm and the point cloud model obtained by laser scanning through the position information of the landmark points; Precise registration: The ICP algorithm is used to find the nearest corresponding point for each point in each pair of point clouds, and then the rotation and translation transformations are calculated based on the corresponding point pairs found. The calculated transformations are then applied and the point cloud positions are updated. This cycle is repeated until the alignment error of the two point cloud models reaches a predetermined threshold or the number of iterations reaches an upper limit.

[0031] The step of smoothing the surface of the three-dimensional model by Laplacian smoothing in step six is: Import and read the 3D mesh model that needs to be optimized; Get the position of each vertex in the mesh , and establish its adjacent point structure set, that is, determine the neighbor vertex set of each vertex , calculate the Laplacian coordinates , the calculation formula is: ; Where N is the number of neighbor vertices; According to Laplacian coordinates Update the vertex position, the update formula is: ; in represents the updated vertex position, represents the smoothing factor and its value is 0.2; Repeat the smoothing steps as necessary until the desired degree of smoothness is achieved.

[0032] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for acquiring indoor three-dimensional effects based on multi-view images, characterized in that: The following steps are included: Step 1: Collection Preparation Select high-resolution cameras and high-precision laser scanners to obtain indoor two-dimensional floor plans, thereby obtaining the length of each room and corridor area. He Kuan , and then evenly place them in each room and corridor area according to the placement criteria Mark points and develop a shooting path; Step 2: Data Collection According to the shooting and scanning guidelines and along the shooting path, multiple photos covering all indoor areas are taken from different heights, angles and positions. During this period, a laser scanner is used to perform a full-scale scan of the indoor area along the scanning station on the shooting path; Step 3: Data preprocessing Preprocess the photos, extract feature points in the photos using the improved SIFT algorithm, and calibrate the camera using the landmarks to obtain the camera's internal and external parameters. Step 4: 3D reconstruction and data fusion Use the SFM algorithm to match the taken photos, and use the matched feature points and camera parameters to generate a sparse point cloud. Then use the MVS algorithm to generate a dense point cloud model. Import the laser scanning data of all scanning stations into the 3D modeling software, align and splice the scanning data of each station to form a complete point cloud model. Then, register and align the two point cloud models to ensure that the alignment error between the two point cloud models reaches the predetermined threshold. The two aligned point cloud models are merged into a final point cloud model. Finally, the final point cloud model is converted into a 3D mesh model and the model surface is generated. Step 5: Texture Mapping The high-resolution photos taken by the camera are used as texture maps and mapped onto the 3D mesh model to generate a 3D mesh model with texture, and then the texture on the 3D mesh model is color corrected; Step 6: Model optimization and output A filtering algorithm is used to remove noise points in the 3D mesh model, and then the surface of the 3D mesh model is smoothed by Laplacian smoothing. After the optimization is completed, the final 3D mesh model is exported to a commonly used format.

2. The method for acquiring indoor three-dimensional effects based on multi-view images according to claim 1, characterized in that: The strategy for formulating the shooting path in step 1 is: Determine the width of each room and corridor area in the room Is it greater than ,like , a straight line path is used, that is, from the entrance to the farthest corner; if , using a ring path or a hub-and-spoke path; If the layout of the room or corridor is complex, a central radial path is used, that is, starting from the center and shooting radiates to the surrounding areas; if the layout of the room or corridor is simple, a circular path is used, that is, walking around the room or corridor area and shooting intermittently along the way.

3. The method for acquiring indoor three-dimensional effects based on multi-view images according to claim 1, characterized in that: The placement standard of the marker points in step 1 is specifically: ,in represents the floor space of the room or corridor area, and , It represents the placement density of the landmark points. And when hour, ; when hour, ; when hour, .

4. The method for acquiring indoor three-dimensional effects based on multi-view images according to claim 1, characterized in that: The shooting and scanning criteria in step 2 include: Evenly arrange shooting points and scanning stations on the shooting path to ensure that the photos taken by adjacent shooting points have an overlap of more than 60%, and that there is an overlap area of ​​10%-20% between adjacent scanning stations; Shoot from multiple angles at each corner and key object, including the front, side, and top, to fully capture the structure at the corner and obtain multi-view images of each key object; Use a wide-angle lens to shoot the ceiling and floor and ensure the integrity of the shot.

5. The method for acquiring indoor three-dimensional effects based on multi-view images according to claim 1, characterized in that: The steps of photo preprocessing in step 3 include: Correct the photos taken to eliminate lens distortion; Enhance the local contrast of the photo through local contrast enhancement technology, making the subtle structure of the surface of the object in the photo more obvious; Use the Canny edge detection algorithm to enhance edge information and improve the detection effect of feature points.

6. The method for acquiring indoor three-dimensional effects based on multi-view images according to claim 1, characterized in that: The SIFT algorithm in step 3 is a feature detection and descriptor generation method used in the field of image processing and computer vision. It has scale invariance and rotation invariance, and can stably detect and match feature points under different viewing angles, lighting conditions and scales. The SIFT algorithm is mainly divided into four steps, namely scale space extreme value detection, key point location, direction assignment and key point descriptor generation, and the specific steps of the improved SIFT algorithm are: Scale space extreme value detection: First, the image is processed with Gaussian blur multiple times to generate images of different scales, where each scale image is called an octave, and each octave contains multiple scale layers. Then, potential key points are detected by the difference between adjacent scale layers. The differential image can approximately represent the scale space of the image. At each pixel point in the differential image, it is compared with the 8 pixels in its neighborhood and the corresponding pixels of the previous scale layer and the next scale layer to find the local extreme value points. These extreme value points are candidate key points. Key point location: Accurately locate the position and scale of key points by fitting a three-dimensional quadratic function, remove noise key points with low response values, and eliminate edge response points with poor stability at different scales by calculating the principal curvature ratio of the Hessian matrix; Direction assignment: Calculate the gradient direction and gradient magnitude in the neighborhood of each key point, generate a gradient direction histogram, and determine the main direction of the key point based on the peak value of the histogram. If the gradient magnitude of a certain direction exceeds 80% of the main direction, then the direction is also considered to be an important direction. Then assign one or more directions to each key point so that the key point has rotation invariance. Keypoint descriptor fusion: At each keypoint In the neighborhood of the key point, the key point is taken as the center, rotated according to the direction of the key point, and then divided into 16 sub-regions, and in each sub-region the gradient amplitudes in 8 directions are calculated to obtain 16 8-dimensional SIFT descriptors. At the same time, an 8-dimensional LBP feature is extracted in each sub-region through the LBP algorithm. For each sub-region, the SIFT descriptor and the LBP feature are concatenated together to form a 16-dimensional feature vector. Then, the feature vectors of the 16 sub-regions are concatenated into a 256-dimensional complete feature vector. This feature vector is the final descriptor, which represents the local features at the key point.

7. The method for acquiring indoor three-dimensional effects based on multi-view images according to claim 6, characterized in that: The steps of extracting LBP features through the LBP algorithm are: Will The neighborhood is divided into 16 sub-regions; For each pixel in the sub-area, calculate its LBP value; A histogram is generated based on the distribution of LBP values ​​in each sub-region, where the number of bins in the histogram is 8, and each bin represents the frequency of occurrence of a certain LBP value; The 8-bin LBP histogram is represented as a feature vector of length 8, which is the LBP feature of this sub-region.

8. The method for acquiring indoor three-dimensional effects based on multi-view images according to claim 7, characterized in that: The calculation steps of LBP value are as follows: Establish the center pixel and obtain the grayscale value of the center pixel ; Select the pixel around the center Neighborhood pixels, this Neighborhood pixels are evenly distributed in a radius of On the circle of , and because the sub-region size is , so the corresponding and ; Get separately Gray value of neighboring pixels ,like , then assign 1, otherwise assign 0; Arrange the above assignment results in order The binary number of the bit is converted into a decimal number, which is the LBP value of the central pixel.

9. The method for acquiring indoor three-dimensional effects based on multi-view images according to claim 1, characterized in that: The steps of model registration in step 4 are as follows: Initial registration: Roughly align the dense point cloud model generated by the MVS algorithm and the point cloud model obtained by laser scanning through the position information of the landmark points; Precise registration: The ICP algorithm is used to find the nearest corresponding point for each point in each pair of point clouds, and then the rotation and translation transformations are calculated based on the corresponding point pairs found. The calculated transformations are then applied and the point cloud positions are updated. This cycle is repeated until the alignment error of the two point cloud models reaches a predetermined threshold or the number of iterations reaches an upper limit.

10. The method for acquiring indoor three-dimensional effects based on multi-view images according to claim 1, characterized in that: The step of smoothing the surface of the three-dimensional model by Laplacian smoothing in step six is: Import and read the 3D mesh model that needs to be optimized; Get the position of each vertex in the mesh , and establish its adjacent point structure set, that is, determine the neighbor vertex set of each vertex , calculate the Laplacian coordinates , the calculation formula is: ； Where N is the number of neighbor vertices; According to Laplacian coordinates Update the vertex position, the update formula is: ； in represents the updated vertex position, represents the smoothing factor and its value is 0.2; Repeat the smoothing steps as necessary until the desired degree of smoothness is achieved.

Images