A 3DGS reconstruction method and system based on scene environment selection

Abstract

The application provides a 3DGS reconstruction method and system based on scene environment selection, and belongs to the technical field of three-dimensional reconstruction; the method comprises the following steps: acquiring multi-angle shooting videos of a scene environment to be reconstructed, dividing the obtained multi-angle shooting videos into multiple pictures and storing the pictures in a same folder; traversing the folder, and sequentially performing scene judgment under multiple shooting scenes on each picture in the traversing process to determine the belonging scene; if the current picture belongs to more than one of the multiple shooting scenes, the belonging scene is determined according to the scene priority; and the scene proportion of all the pictures in the folder is counted, and a 3DGS variant suitable for the scene with the highest proportion is selected to perform three-dimensional reconstruction. The application can realize high-quality three-dimensional reconstruction for images shot under any angle and any environment.

Description

Technical Field

[0001] This invention belongs to the field of 3D reconstruction technology, and in particular relates to a 3DGS reconstruction method and system based on scene environment selection. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the advancement of technology, the field of vision, especially 3D technology, has received increasing attention, and reconstruction methods based on neural representations have rapidly become a research hotspot. 3D reconstruction is a crucial problem in computer vision and graphics, aiming to reconstruct the 3D geometric and appearance information of the real world using information from multiple perspectives. 3DGS (Gaussian Splatting), as an emerging 3D representation method, excels in static scenes due to its efficient rendering speed and excellent reconstruction quality. 3DGS uses a 3D Gaussian distribution to model scenes, achieving high-quality scene modeling by optimizing the shape, position, color, and transparency of the Gaussian sphere.

[0004] 3DGS introduces point cloud representation and differentiable rendering, making the reconstruction process more real-time, suitable for a wider range of scene modeling, and offering faster rendering speeds. However, in practical scene modeling, 3D modeling based on 3DGS faces intractable technical challenges, such as:

[0005] (1) Insufficient generalization ability. Existing 3DGS methods rely on standardized scenes with static and uniform lighting (such as laboratory environments). When faced with complex scenes such as low light, strong exposure, motion blur, high dynamic range (HDR) or human / moving objects that are common in real applications, the reconstruction quality drops sharply. This is mainly manifested in severe loss of details in low light scenes, artifacts in strong exposure areas, motion blur or structural misalignment in the reconstruction of dynamic objects, and inaccurate restoration of brightness levels in HDR scenes.

[0006] (2) High dependence on manual intervention. Existing solutions require manual selection of reconstruction algorithms suitable for the scene. As a result, during the reconstruction process, users need to analyze scene features in advance and then manually match algorithms. This process is time-consuming and has a high professional threshold. In addition, manual judgment is easily limited by experience and there is a risk of misjudgment. For example, it is difficult to quantitatively evaluate image blur, cannot accurately detect dynamic displacement, and easily ignores the situation of multiple scene intersections.

[0007] (3) Limited shooting freedom. In order to match the fixed algorithm, users often need to constrain shooting conditions (such as specific time period, fixed lighting environment), which makes it impossible to support the flexible reconstruction needs of "any angle, any environment"; and the practicality of outdoor exploration, emergency scene and other scenarios is limited. Summary of the Invention

[0008] To overcome the shortcomings of the prior art, the present invention provides a 3DGS reconstruction method and system based on scene environment selection, which can achieve high-quality 3D reconstruction for images taken from any angle and in any environment.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0010] The first aspect of this invention provides a 3DGS reconstruction method based on scene environment selection.

[0011] A 3DGS reconstruction method based on scene environment selection includes:

[0012] Obtain multi-angle video footage of the scene environment to be reconstructed, split the obtained multi-angle video footage into multiple images and save them into the same folder;

[0013] The system iterates through multiple images in the folder, performing scene judgments under various shooting scenarios on each image during the iteration process to determine the scene to which the current image belongs. If the current image is judged to belong to more than one of the multiple shooting scenarios, the scene to which the current image belongs is determined according to the scene priority.

[0014] The system calculates the percentage of each scene represented by all images in the folder and selects the 3DGS variant that matches the scene with the highest percentage for 3D reconstruction.

[0015] Furthermore, the scene to which the current image belongs is determined according to scene priority, including: determining whether the current image meets the proportion conditions of human body, total number of blurs, low light image, HDR image, dynamic image, and high light image in sequence. In the process of determining in sequence, the proportion condition that the current image meets first corresponds to the scene to which the current image belongs.

[0016] Furthermore, for each image in the traversal process, scene judgments are performed under various shooting scenarios, including: low brightness scene, high brightness scene, HDR scene, blurred scene, human body scene, and dynamic scene.

[0017] Furthermore, the current image is subjected to high-brightness scene judgment, including: calculating the number of highlighted pixel values ​​in the current image based on the np.sum function, comparing the obtained number of highlighted pixel values ​​with the total number of pixels in the current image, and using the ratio as the basis for judging whether the current image belongs to a high-brightness scene.

[0018] Furthermore, the current image is subjected to blurry scene determination, including: performing grayscale conversion on the current image, and using the Laplacian operator to perform edge detection on the grayscale-converted current image to obtain the second derivative of the current image; calculating the variance of the Laplacian response based on the obtained second derivative to determine whether the current image is a blurry scene.

[0019] Furthermore, the current image is subjected to human scene detection, including: first, using the MediaPiple function to convert the current image to RGB format compatible with the MediaPiple function; then, performing human detection on the converted image, and returning True if key points of the human body can be detected in the current image.

[0020] Furthermore, dynamic scene determination is performed on the current image, including: First, defining a feature matching function to convert the current image and any image adjacent to the current image into grayscale images, and extracting key points and corresponding descriptors from the two images; then, using a brute-force matcher based on Hamming distance to perform k-nearest neighbor matching to filter high-quality matching pairs and determine whether the two images belong to the same scene; finally, determining whether the current image is a dynamic scene based on average displacement and matching point estimation.

[0021] A second aspect of the present invention provides a 3DGS reconstruction system based on scene environment selection.

[0022] A 3DGS reconstruction system based on scene environment selection includes:

[0023] The image segmentation module is configured to: acquire multi-angle video of the scene environment to be reconstructed, segment the acquired multi-angle video into multiple images and save them into the same folder;

[0024] The scene determination module is configured to: traverse multiple images in the folder, perform scene determination under multiple shooting scenarios for each image during the traversal process to determine the scene to which the current image belongs; if the current image is determined to belong to more than one of the multiple shooting scenarios, then determine the scene to which the current image belongs according to the scene priority.

[0025] The 3D reconstruction module is configured to: count the percentage of each scene to which all images in the folder belong, and select the 3DGS variant that matches the scene with the highest percentage for 3D reconstruction.

[0026] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a 3DGS reconstruction method based on scene environment selection as described in the first aspect of the present invention.

[0027] The fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a 3DGS reconstruction method based on scene environment selection as described in the first aspect of the present invention.

[0028] The above one or more technical solutions have the following beneficial effects:

[0029] (1) This invention sequentially performs scene judgment under multiple shooting scenarios for each image it traverses. That is, through quantitative detection algorithms for six types of scenes (such as low light intensity threshold judgment, dual threshold verification for HDR scenes, and displacement vector analysis for dynamic scenes), it can automatically match the optimal 3DGS variant for 3D modeling. For example, for low light scenes, ordinary 3DGS can be used in combination with image enhancement to restore dark details; for HDR scenes, HDRsplatting can be enabled to accurately restore high contrast levels; for dynamic scenes, DynGS can be called to eliminate motion blur. Therefore, compared with the prior art, this invention can break through the limitations of standardized scenes and achieve accurate adaptation and reconstruction of complex scenes.

[0030] (2) When a current image is determined to belong to one or more shooting scenarios simultaneously, this invention specifies that the scene to which the current image belongs is determined according to scene priority. That is, when a single image triggers multiple conditions simultaneously (such as a dynamic human body at night), the dominant scene is selected according to priority, without the need for manual intervention, and end-to-end automation from image input to algorithm output can be achieved. Compared with the prior art, this invention, based on this fully automated decision-making process, can eliminate subjective misjudgments.

[0031] (3) This invention breaks through the mandatory requirements of the shooting environment, and achieves "shoot and build as you go" under unconstrained conditions in terms of time, environment and operation. For example, in terms of time freedom, it can directly reconstruct whether it is low light at night or strong exposure at noon; in terms of environmental freedom, it supports non-ideal conditions such as hand-held shaking (blurred scene) and fast-moving target (dynamic scene); in terms of operational freedom, the user only needs to provide the video stream shot at any angle, without the need to pre-adjust parameters.

[0032] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0033] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0034] Figure 1This is a flowchart of a 3DGS reconstruction method based on scene environment selection in Embodiment 1 of the present invention.

[0035] Figure 2 A flowchart for 3DGS reconstruction using existing technology.

[0036] Figure 3 This is a flowchart illustrating how the scene to which the current image belongs is determined according to scene priority in Embodiment 1 of the present invention. Detailed Implementation

[0037] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0038] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0039] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0040] To facilitate understanding of the technical solution of this invention, the following terms are explained:

[0041] 1) Strategy_map: Parameters used to describe different environments.

[0042] 2) np: Numpy is a Python library for efficient matrix calculations.

[0043] 3) Ssim: Used to calculate the structural similarity between two images to determine if there have been changes in the scene.

[0044] 4) Lower's ratio test: A ratio test used in feature matching to eliminate incorrect matches.

[0045] 5) Orb: A computer vision algorithm used to extract and describe key points in an image.

[0046] 6) BFMatcher: A brute-force matcher in OpenCV, used to compare the distance between two feature descriptors one by one to find the nearest neighbor match.

[0047] 7) Laplancian Variance: A commonly used metric for image blur detection.

[0048] 8) HDR: High Dynamic Range, where brightness and contrast vary over a wider range than in a normal image.

[0049] 9) ffmpeg: A multimedia processing tool that can be used to record, convert, and stream audio and video.

[0050] 10) MediaPiple: A data stream processing and machine learning application development framework developed and open-sourced by Google.

[0051] Example 1

[0052] This embodiment discloses a 3DGS reconstruction method based on scene environment selection.

[0053] like Figure 1 As shown, a 3DGS reconstruction method based on scene environment selection includes:

[0054] Step S1: Obtain multi-angle video of the scene environment to be reconstructed, divide the obtained multi-angle video into multiple images and save them in the same folder;

[0055] Step S2: Traverse the multiple images in the folder, and perform scene judgment under multiple shooting scenarios for each image during the traversal process to determine the scene to which the current image belongs; if the current image is judged to belong to more than one of the multiple shooting scenarios at the same time, then determine the scene to which the current image belongs according to the scene priority.

[0056] Step S3: Calculate the scene proportion of all images in the folder, and select the 3DGS variant that matches the scene with the highest proportion for 3D reconstruction.

[0057] like Figure 2 As shown, existing 3DGS reconstruction methods often rely on human experience to select appropriate shooting environments to adapt to reconstruction strategies when handling different types of scenes. However, due to the significant differences in scene complexity, such as texture complexity, lighting conditions, and blur levels, directly adopting a uniform 3DGS modeling scheme often leads to problems such as insufficient reconstruction accuracy, artifact generation, or low rendering efficiency. Furthermore, relying solely on human judgment of photographs also has limitations. To address this, this invention provides a 3DGS reconstruction method based on scene environment selection. This method eliminates the need for manual judgment and reduces the tedious steps of users searching for suitable 3DGS, simplifying the process and allowing users to shoot scenes in different environments. It also increases user freedom of shooting; that is, this invention can achieve high-quality 3D reconstruction for images shot from any angle and in any environment. To facilitate understanding of the technical solution of this invention, the specific implementation methods of this invention will be further explained and described below.

[0058] In step S1, multi-angle video of the scene environment to be reconstructed is obtained, and the obtained multi-angle video is divided into multiple images and stored in the same folder.

[0059] The camera is used to capture images of the scene, objects, or buildings that the user needs to reconstruct from multiple angles (ideally, higher pixel quality is better) to cover the entire scene. The captured video is then processed using ffmpeg to segment it into images. These images are then placed in a folder as input for subsequent scene analysis. ffmpeg is a multimedia processing tool used to record, convert, and stream audio and video.

[0060] In step S2, multiple images in the folder are traversed, and scene judgments under multiple shooting scenarios are performed on each image during the traversal process to determine the scene to which the current image belongs. If the current image is judged to belong to more than one of multiple shooting scenarios, the scene to which the current image belongs is determined according to the scene priority.

[0061] For each image during the traversal process, scene judgments are performed under various shooting scenarios, including: low-brightness scenes, high-brightness scenes, HDR scenes, blurred scenes, human figures scenes, and dynamic scenes. Among them, low-brightness scenes are images taken in low light (nighttime), high-brightness scenes are images taken in high light (complex lighting), HDR scenes are images taken in high dynamic range (high color difference), blurred scenes are images showing different degrees of blur in different areas, human figures scenes are images of people, and dynamic scenes are images showing changes in objects, lighting, movement, and other elements in the scene over a certain period of time.

[0062] For each image during the traversal process, scene determination is performed under six shooting scenarios, specifically:

[0063] The first step is to perform low-light scene analysis on the current image, including:

[0064] The input color image (the current image) is converted to grayscale. Then, `np.mean(gray)` is used to calculate the average brightness of the image, with a threshold set to 90 (in common 8-bit grayscale images, pixel values ​​range from [0,255]; in industrial inspection, 80-100 is typically used as the low-light threshold). Values ​​below this threshold are considered low brightness. The calculated result is then returned and recorded for subsequent processing. `np.mean(gray)` is a Python library function that calls the `mean` function to calculate the average, where `gray` represents the parameter to be averaged.

[0065] The second step is to determine the high-brightness scene in the current image, including:

[0066] Use pixel_ratio_threshold as the threshold for the proportion of high brightness pixels, and brightness_threshold=240 (with 255 as the highest, and 240 is very close to the maximum value, meaning that the pixel is in the high brightness area) as the threshold for high brightness pixels.

[0067] Based on this, firstly, the current image is converted to a grayscale image using the function body; the function body is `cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)`, a function in OpenCV that converts an image from one color space to another. Then, the `np.sum` function is used to calculate the number of highlighted pixels in the converted image. This number is compared to the total number of pixels in the image. The ratio is used to determine whether the image belongs to a high-brightness scene; if the ratio is greater than a defined threshold, it is considered a high-brightness scene. The result is then returned and recorded for subsequent processing. The `np.sum` function is a Python library function used to perform summation calculations.

[0068] The third step is to determine the HDR scene of the current image, including:

[0069] Define std_threshold=50 (within the grayscale pixel range, the maximum standard deviation is about 76, so setting a relatively high value indicates that the brightness distribution difference is extremely large. If set too low, normal brightness or dark but low contrast will be misjudged) to indicate whether the overall brightness standard deviation is large enough. define local_contrast_threshold=30 (too low will easily misjudge noisy or dimensionally rich images, too high will miss real HDR scenes) to indicate whether the local contrast is large enough.

[0070] Based on this, firstly, the image is converted to grayscale, and the standard deviation of pixel brightness for the entire current image is calculated using `np.std`. Then, the size of the image selection box is set (which can be fine-tuned later based on the size of the input image), and a list `block` storing the standard deviations of each local brightness is initialized. Next, the image is iterated through, and the results are added to the list. The average of the local brightness standard deviations is calculated to obtain the overall local level. Finally, a judgment is made based on a defined threshold, and the result is returned and recorded for subsequent processing. The `np.std` function is a Python library function used to call the standard deviation function to calculate the standard deviation.

[0071] HDR images typically contain multiple exposure levels, resulting in uneven brightness distribution. The standard deviation is used as a means of judgment when the number of images increases. At the same time, local contrast is also an important piece of information in the image, used to judge the changes in brightness in local areas.

[0072] The fourth step is to perform scene detection on the current image, including:

[0073] A threshold is set as the blur determination threshold. The current image is converted to grayscale, and the Laplacian operator is used to perform edge detection on the converted image, obtaining the second derivative of the current image. The response capability of the Laplacian operator represents the sharpness of the edge image. By calculating the variance of the Laplacian response, a large variance indicates rich edge information and a sharp image; conversely, a small variance indicates a sharper image. The results are returned and recorded for subsequent processing.

[0074] The Laplacian operator is a commonly used second-order edge detection operator that is highly sensitive to changes in the edges and edges of an image. The variance of the Laplacian operator is often used as a quantitative indicator of image sharpness.

[0075] The fifth step is to determine the human scene in the current image, including:

[0076] First, the MediaPipe function is used to convert the current image to RGB format, which is compatible with MediaPipe. MediaPipe is an open-source machine learning framework from Google for multimedia tasks that identifies the human body by detecting the presence of human keypoints in the image. Then, a defined human pose detector, `pose_detector`, is used to detect the human body in the converted image. If human keypoints are detected, a `True` result is returned, and the result is saved and returned for subsequent processing. Furthermore, the human pose detector can be represented as:

[0077] pose_detector=

[0078] mp_pose.Pose(static_image_mode=True, min_detection_confidence=0.5);

[0079] Here, `Mp_pose` is a class provided by MediaPipe used to detect key points on the human body; `min_detection_confidence` represents the lowest confidence threshold at which the pose detection model considers the detection result valid; and `static_image_mode` indicates whether the input is treated as a static image. Thus, a human pose detector, `pose_detector`, is ultimately created.

[0080] Step 6: Perform dynamic scene determination on the current image, including:

[0081] 1) Define a feature matching function, namely an image feature matching function based on ORB feature extraction and BFMatcher, which converts the current image and any image adjacent to the current image into grayscale images; then, use the ORB algorithm (based on the average Euclidean distance of the matching point displacement, which can well reflect the intensity of the relative motion between images) to extract key points and corresponding descriptors of the two images; determine whether to perform subsequent operations based on the descriptors, and exit directly if there are too few key points.

[0082] The definition process of the feature matching function can be summarized as follows:

[0083] First, the input image is converted to grayscale. Next, the ORB is initialized, and keypoints and descriptors are extracted. Keypoints represent locations with significant features in the image, and descriptors represent the feature vectors corresponding to each keypoint. Then, anomaly handling is performed, creating a BFMatcher and performing KNN matching. Finally, poor matches are filtered based on Lowe's Ratio Test, and the results are returned.

[0084] Extracting key points and corresponding descriptors from two images using the ORB algorithm can be summarized as follows:

[0085] First, an instance of ORB needs to be created (orb = cv2.ORB_create(nfeatures=1000)); then, the detectAndCompute() function is called, i.e.:

[0086] kp1, des1 = orb.detectAndCompute(gray1, None);

[0087] Where gray1 represents the input grayscale image, None represents the mask parameter; Kp1 and des1 are the return values, representing the list of key points and the descriptor matrix, respectively.

[0088] 2) A brute-force matcher is used to perform k-nearest neighbor (KNN) matching based on Hamming distance to filter high-quality matching pairs and determine whether two images belong to the same scene. Specifically: a brute-force matcher is used to perform KNN matching based on Hamming distance, returning the two nearest neighbor matches for each descriptor. High-quality matching pairs are then filtered using Lower's Ratio Test; Lower's Ratio Test is a strategy for filtering high-quality matching pairs, widely used in feature matching to eliminate incorrect matches by comparing the nearest neighbor and its distance. Finally, it is determined whether two images belong to the same scene, i.e., the number of high-quality matches obtained from feature matching is greater than or equal to a default threshold, indicating that they belong to the same scene.

[0089] Finally, based on the average displacement and matching point estimation, it is determined whether the current image is a dynamic scene. Specifically, by estimating the displacement of feature matching points, the intensity of motion between the two images is estimated. After obtaining matching points using feature matching methods, the Euclidean distance is calculated, and the average displacement of each matching point is calculated as the motion amplitude. The proportion of high-quality matching points to the total number of keypoints in the first image is calculated. The existence of significant motion that is not within the same scene is estimated using the average displacement and matching point estimation, thus determining whether it is dynamic motion.

[0090] SSIM calculates the structural similarity between two images, which can be used to determine whether a scene has changed, thus better reflecting image changes. Combining feature matching quantity and motion displacement to determine whether a scene is dynamic is a multimodal fusion decision method with wide applications in action recognition and other fields.

[0091] After the above six steps, if the current image is determined to belong to more than one of multiple shooting scenarios, the scene to which the current image belongs is determined according to scene priority. Determining the scene to which the current image belongs includes: sequentially judging whether the current image meets the proportion conditions of human body, total blur, low light image, HDR image, moving image, and high light image. In the process of sequential judgment, the proportion condition that the current image first meets corresponds to the scene to which the current image belongs.

[0092] like Figure 3 As shown, the scene to which the current image belongs is determined according to scene priority, specifically as follows:

[0093] 1) Determine if a human figure exists within the scene of the current image. If so, calculate the proportion of the human figure. If not, proceed directly to the next step without calculating the proportion of the human figure. During the calculation, if the proportion of the human figure is greater than 0.5, the current image is considered to belong to a scene with a human figure, and the process ends.

[0094] 2) Determine if a blurred scene exists within the current image's context. If it does, calculate the percentage of blurred scenes. If not, proceed directly to the next step without calculating the percentage of blurred scenes. During the percentage calculation, if the percentage of blurred scenes is greater than 0.3, the current image is considered to belong to a blurred scene, and the process ends.

[0095] 3) Determine if a low-brightness (low-light) scene exists within the current image's context. If it does, calculate the proportion of low-light images. If not, proceed directly to the next step without calculating the proportion of low-light images. During the calculation of the proportion of low-light images, if the proportion is greater than 0.3, the current image is directly identified as belonging to a low-brightness scene, and the process exits.

[0096] 4) Determine if an HDR scene exists within the current image's context. If it does, calculate the HDR image percentage. If not, proceed directly to the next step without calculating the HDR image percentage. During the HDR image percentage calculation, if the HDR image percentage is greater than 0.3, the current image is automatically considered to belong to an HDR scene, and the process ends.

[0097] 5) Determine if a dynamic scene exists within the current image's context. If it does, calculate the proportion of dynamic images. If not, proceed directly to the next step without calculating the proportion of dynamic images. During the dynamic image proportion calculation, if the proportion of dynamic images is greater than 0.3, the current image is considered to belong to a dynamic scene, and the process ends.

[0098] 6) Determine if a high-brightness (high-light) scene exists in the current image's scene. If it exists, calculate the proportion of the high-light image. If it does not exist, proceed directly to the next step without calculating the proportion of the high-light image. When calculating the proportion of the high-light image, if the proportion of the high-light image is greater than 0.3, the current image is directly identified as a high-brightness scene, and the judgment process ends.

[0099] In step S3, the scene proportion of all images in the folder is counted, and the 3DGS variant adapted to the scene with the highest proportion is selected for 3D reconstruction.

[0100] Based on the above process, after determining the scene, the corresponding 3DGS variant is selected for 3D reconstruction. Specifically: image enhancement processing and ordinary 3DGS are used in low-light scenes, MS-GS is used in complex lighting scenes, SRGS is used in blurred scenes, HDRsplatting is used in high dynamic range scenes, GaussianAvatars is used in human body shooting scenes, and DynGS is used in dynamic scenes.

[0101] Based on the 3DGS reconstruction method based on scene environment selection provided by this invention, it is no longer necessary to manually identify or force scene shooting, and can achieve "shoot and rebuild", which not only speeds up the reconstruction speed, but also further improves the reconstruction quality; at the same time, it avoids the problem of human inability to judge in time or judgment errors, and also reduces the tedious steps for users to find the corresponding 3DGS algorithm themselves.

[0102] Example 2

[0103] This embodiment discloses a 3DGS reconstruction system based on scene environment selection.

[0104] A 3DGS reconstruction system based on scene environment selection includes:

[0105] The image segmentation module is configured to: capture the scene environment to be reconstructed from multiple angles, segment the captured video into multiple images and save them into the same folder;

[0106] The scene determination module is configured to: traverse multiple images in the folder, perform scene determination under multiple shooting scenarios for each image during the traversal process to determine the scene to which the current image belongs; if the current image is determined to belong to more than one of the multiple shooting scenarios, then determine the scene to which the current image belongs according to the scene priority.

[0107] The 3D reconstruction module is configured to: count the percentage of each scene to which all images in the folder belong, and select the 3DGS variant that matches the scene with the highest percentage for 3D reconstruction.

[0108] Example 3

[0109] The purpose of this embodiment is to provide a computer-readable storage medium.

[0110] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a 3DGS reconstruction method based on scene environment selection as described in Embodiment 1 of this disclosure.

[0111] Example 4

[0112] The purpose of this embodiment is to provide an electronic device.

[0113] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in a 3DGS reconstruction method based on scene environment selection as described in Embodiment 1 of this disclosure.

[0114] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0115] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0116] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A 3DGS reconstruction method based on scene environment selection, characterized in that, include: Obtain multi-angle video footage of the scene environment to be reconstructed, split the obtained multi-angle video footage into multiple images and save them into the same folder; The system iterates through multiple images in the folder, performing scene judgments under various shooting scenarios on each image during the iteration process to determine the scene to which the current image belongs. If the current image is determined to belong to more than one of the aforementioned shooting scenarios, the scene to which the current image belongs is determined according to the scene priority. The various shooting scenarios include: low-brightness scenes, high-brightness scenes, HDR scenes, blurred scenes, human figures scenes, and dynamic scenes; The scene to which the current image belongs is determined according to scene priority, including: judging whether the current image meets the proportion conditions of human body, total number of blurs, low light image, HDR image, dynamic image, and high light image in turn. In the process of judging in turn, the proportion condition that the current image meets first corresponds to the scene to which the current image belongs. The system calculates the percentage of each scene represented by all images in the folder and selects the 3DGS variant that matches the scene with the highest percentage for 3D reconstruction.

2. The 3DGS reconstruction method based on scene environment selection as described in claim 1, characterized in that, The process involves determining whether the current image is a high-brightness scene, including: calculating the number of highlighted pixels in the current image using the np.sum function, comparing the number of highlighted pixels with the total number of pixels in the current image, and using the ratio as the criterion for determining whether the current image belongs to a high-brightness scene.

3. The 3DGS reconstruction method based on scene environment selection as described in claim 1, characterized in that, The process for determining whether the current image is a blurred scene includes: performing grayscale conversion on the current image, and using the Laplacian operator to perform edge detection on the grayscale-converted current image to obtain the second derivative of the current image; and calculating the variance of the Laplacian response based on the obtained second derivative to determine whether the current image is a blurred scene.

4. The 3DGS reconstruction method based on scene environment selection as described in claim 1, characterized in that, The process involves determining the presence of a human in the current image, including: first, using the MediaPiple function to convert the current image to an RGB format compatible with the MediaPiple function; then, performing human detection on the converted image, and returning True if key points of a human are detected in the current image.

5. The 3DGS reconstruction method based on scene environment selection as described in claim 1, characterized in that, The dynamic scene determination of the current image includes: First, defining a feature matching function to convert the current image and any adjacent image into grayscale images, and extracting key points and corresponding descriptors from the two images; then, using a brute-force matcher based on Hamming distance to perform k-nearest neighbor (KNN) matching to filter high-quality matching pairs and determine whether the two images belong to the same scene; finally, determining whether the current image is a dynamic scene based on average displacement and matching point estimation.

6. A 3DGS reconstruction system based on scene environment selection, characterized in that, include: The image segmentation module is configured to: acquire multi-angle video of the scene environment to be reconstructed, segment the acquired multi-angle video into multiple images and save them into the same folder; The scene judgment module is configured to: traverse multiple images in the folder, and perform scene judgment under multiple shooting scenarios for each image during the traversal process to determine the scene to which the current image belongs; If the current image is determined to belong to more than one of the aforementioned shooting scenarios, the scene to which the current image belongs is determined according to the scene priority. The various shooting scenarios include: low-brightness scenes, high-brightness scenes, HDR scenes, blurred scenes, human figures scenes, and dynamic scenes; The scene to which the current image belongs is determined according to scene priority, including: judging whether the current image meets the proportion conditions of human body, total number of blurs, low light image, HDR image, dynamic image, and high light image in turn. In the process of judging in turn, the proportion condition that the current image meets first corresponds to the scene to which the current image belongs. The 3D reconstruction module is configured to: count the percentage of each scene to which all images in the folder belong, and select the 3DGS variant that matches the scene with the highest percentage for 3D reconstruction.

7. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the 3DGS reconstruction method based on scene environment selection as described in any one of claims 1-5.

8. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the 3DGS reconstruction method based on scene environment selection as described in any one of claims 1-5.

Images