A three-dimensional reconstruction method, device, equipment and storage medium

Abstract

This application discloses a 3D reconstruction method, apparatus, device, and storage medium, relating to the field of 3D reconstruction technology. The method involves acquiring raw micro-topographic images of a target area using a multi-view camera; filtering the raw micro-topographic images to obtain a target micro-topographic image; extracting feature points from the target micro-topographic image using an ORB-like algorithm and generating binary descriptors for the feature points to obtain multiple initial feature point sets; constructing an initial spatial grid based on light source-camera joint calibration information; performing feature point matching on the multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; filtering the initial feature point pairs based on optical flow field constraints to obtain intermediate feature point pairs; and filtering the intermediate feature point pairs based on Lambert cosine law ray constraints to obtain target feature point pairs, thereby obtaining a 3D point cloud spatial model of the micro-topography. This improves the accuracy of 3D reconstruction.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology, and in particular to a three-dimensional reconstruction method, apparatus, device and storage medium. Background Technology

[0002] The study of micro-geomorphic erosion evolution in loess landforms is a core topic in fields such as geological disaster prevention and soil and water conservation. Accurately acquiring dynamic data on micro-geomorphic changes during rainfall is crucial for revealing erosion mechanisms and establishing predictive models. Existing three-dimensional micro-geomorphic observation technologies mainly include lidar scanning, traditional structured light three-dimensional reconstruction, and photogrammetry, but they have significant limitations in practical applications. While lidar technology offers high precision, it suffers from high equipment costs, low data acquisition efficiency, and in rainy environments, the laser beam is susceptible to scattering interference from rain and fog, leading to decreased ranging accuracy and hindering continuous and stable observation. Traditional structured light 3D reconstruction technology often employs regular grid structured light, which has limited enhancement effects on surface texture and lacks specific design for rainy environments. Rain and fog can cause blurred structured light stripes and increased image noise, resulting in distorted 3D reconstruction. Traditional photogrammetry relies on natural textures, but the loess landform has a uniform color and sparse texture, making it difficult to acquire sufficient feature points. Furthermore, the asynchronous acquisition by multiple cameras and the lack of optimization of matching logic based on light source constraints and optical flow field constraints result in insufficient matching accuracy, failing to accurately reflect the dynamic changes of the erosion process. Summary of the Invention

[0003] In view of this, this application provides a three-dimensional reconstruction method, apparatus, device, and storage medium to improve the accuracy of three-dimensional reconstruction.

[0004] The objective of this application can be achieved through the following technical solutions: The first aspect of this application is to provide a three-dimensional reconstruction method, including: Acquire raw micro-topographic images of the target area using multi-view cameras; The original micro-topography image is filtered to obtain the target micro-topography image; The ORB-like algorithm is used to extract feature points from the target micro-topography image and generate binary descriptors for the feature points to obtain multiple initial feature point sets. An initial spatial grid is constructed based on the joint calibration information of the light source and camera. Feature point matching is performed on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; Based on the optical flow field constraint, the initial feature point pairs are iteratively calculated, and abnormal feature point pairs are eliminated to obtain intermediate feature point pairs; Based on the Lambert cosine law of light constraint, the physical law of intermediate feature point pairs is verified to obtain target feature point pairs; Based on camera calibration parameters and light source calibration parameters, the three-dimensional coordinates of target feature point pairs are calculated using triangulation to obtain a three-dimensional point cloud spatial model of micro-topography.

[0005] In an optional embodiment, before acquiring the raw micro-topographical images collected by the multi-view camera in the target area, the method further includes: Place the calibration target in the target area; Random light spots are projected onto the calibration target, and images of the calibration target are acquired using a multi-view camera. Based on the camera calibration parameters and the object space coordinates of feature points on the calibration target, calculate the position coordinates of the light source in each camera coordinate system; Based on the position coordinates of the light source in each camera coordinate system, the direction vector from the light source to the feature points on the calibration target surface is determined, and the joint calibration information of the light source and camera is obtained.

[0006] In one alternative embodiment, projecting random light spots onto a calibration target includes: A laser beam is emitted from a laser generator to a diffusion optical element; A random light spot is obtained by modulating the laser beam using a diffusion optical element; A random light spot is projected onto the calibrated target.

[0007] In one optional embodiment, filtering the original micro-topography image to obtain the target micro-topography image includes: For each pixel location in the original micro-topography image, obtain the grayscale value of the pixel location in multiple consecutive frames of the original micro-topography image; Statistical values ​​of grayscale values ​​are obtained based on grayscale values; The target micro-topographic image is obtained by removing pixels from the original micro-topographic image based on grayscale statistical values.

[0008] In one optional embodiment, feature point matching is performed on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs, including: Calculate the Hamming distance between the binary descriptors of feature points in each initial feature point set within the initial spatial grid; Feature point pairs with a Hamming distance less than a preset distance threshold are used as initial feature point pairs.

[0009] In one optional embodiment, the initial feature point pairs are iteratively calculated based on optical flow field constraints to remove abnormal feature point pairs, resulting in intermediate feature point pairs, including: Construct an optical flow field constraint equation, in which the pixel grayscale values ​​of adjacent frame target micro-topography images are constant; Within the neighborhood of the feature point, the sum of squared residuals of the optical flow field constraint equations is used as the least squares objective function to solve for the target optical flow vector; The initial feature point pairs corresponding to the target optical flow vectors that are greater than the second preset threshold are removed to obtain intermediate feature point pairs.

[0010] In one optional embodiment, the intermediate feature point pairs are physically verified based on the Lambert cosine law ray constraint to obtain the target feature point pairs, including: Obtain the direction vector from the light source to the micro-terrain surface; Based on the direction vector and micro-topographical reflection coefficient, and using the Lambert cosine law for ray constraint, the physical laws of intermediate feature point pairs are verified to obtain target feature point pairs.

[0011] A second aspect of this application is to provide a three-dimensional reconstruction apparatus, comprising: The acquisition module is used to acquire raw micro-topographic images of the target area collected by the multi-view camera; The filtering module is used to filter the original micro-topography image to obtain the target micro-topography image; The extraction module is used to extract feature points from the target micro-topography image using an ORB-like algorithm and generate binary descriptors for the feature points to obtain multiple initial feature point sets. The building module is used to construct the initial spatial mesh based on the joint calibration information of the light source and camera; The matching module is used to perform feature point matching on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; The calculation module is used to perform iterative calculations on the initial feature point pairs based on optical flow field constraints, remove abnormal feature point pairs, and obtain intermediate feature point pairs; The verification module is used to verify the physical laws of intermediate feature point pairs based on the Lambert cosine law ray constraint to obtain target feature point pairs; The generation module is used to calculate the three-dimensional coordinates of target feature point pairs based on camera calibration parameters and light source calibration parameters using triangulation, thereby obtaining a three-dimensional point cloud spatial model of micro-topography.

[0012] A third aspect of this application is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the method as described in the first aspect.

[0013] A fourth aspect of this application is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the method as described in the first aspect.

[0014] Compared with existing technologies, the 3D reconstruction method provided in this application acquires original micro-topographic images of the target area using multiple cameras; filters the original micro-topographic images to obtain target micro-topographic images; extracts feature points from the target micro-topographic images using an ORB-like algorithm and generates binary descriptors for the feature points, resulting in multiple initial feature point sets; constructs an initial spatial grid based on the joint calibration information of the light source and camera; performs feature point matching on the multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; filters the initial feature point pairs based on optical flow field constraints to obtain intermediate feature point pairs; and filters the intermediate feature point pairs based on Lambert cosine law ray constraints to obtain target feature point pairs, thereby obtaining a 3D point cloud spatial model of the micro-topography. This improves the accuracy of 3D reconstruction. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic flowchart of a three-dimensional reconstruction method provided in an embodiment of this application; Figure 2 A structural block diagram of the three-dimensional reconstruction device provided in the embodiments of this application; Figure 3 This is a structural block diagram of an electronic device for implementing a three-dimensional reconstruction method, provided in an embodiment of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0018] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship. "Contains A, B and / or C" means containing any one, two, or three of A, B, and C.

[0020] It should be understood that in the embodiments of this application, "B corresponding to A", "B corresponding to A", "A corresponds to B" or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0021] To address the technical problems existing in related technologies, embodiments of this application provide a three-dimensional reconstruction method, apparatus, device, and storage medium.

[0022] The 3D reconstruction method provided in this application can be executed by an electronic device, such as a terminal or a server. The terminal can be a smartphone, tablet, laptop, or other similar device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. It is understood that this application does not limit the specific entity executing the 3D reconstruction method.

[0023] The technical solution of this application will be described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments described below are used to explain the technical solution of this application and are not intended to limit actual use.

[0024] To address the technical problems existing in related technologies, embodiments of this application provide a three-dimensional reconstruction method, such as... Figure 1 As shown, Figure 1 This is an example flowchart of a three-dimensional reconstruction method provided in an embodiment of this application. It should be noted that the steps shown may be executed in a different logical order than that shown in the flowchart. The method may include the following steps S101 to S108.

[0025] Step S101: Acquire the original micro-topographic images of the target area captured by the multi-view camera.

[0026] In one alternative embodiment, a multi-view camera refers to a camera system that uses multiple lenses and sensors to simultaneously capture images or videos from different angles, thereby achieving more comprehensive visual information acquisition and processing. A multi-view camera may include at least two industrial cameras, with camera lenses designed to be waterproof and dustproof, and the camera's field of view covering the target area.

[0027] In one specific embodiment, the multi-view camera includes three industrial cameras with a resolution of 2048×1536 and a lens focal length of 25mm. They are encapsulated in a waterproof and dustproof shell and deployed at a 120° angle at a distance of 1.5m above the target micro-topographic area (1m×1m range). The camera angle is adjusted so that the field of view completely covers the observation area without blind spots.

[0028] In another optional embodiment, before acquiring the original micro-terrain images using multiple cameras, a trigger signal can be sent to the multiple cameras via a synchronization controller. A high-speed oscilloscope is used to detect the shutter trigger time of the multiple cameras, and synchronization parameters are adjusted to ensure that multiple cameras start shooting at the same time, with the trigger delay error less than a preset time threshold, thus guaranteeing the spatiotemporal synchronization of the multi-view images. The synchronization controller is connected to the multiple cameras via trigger signal lines.

[0029] In one specific embodiment, the preset time threshold can be 0.3ms, 0.5ms or 1ms, and this application does not limit it.

[0030] In another specific embodiment, the frame rate of the multi-view camera can be set to 50 frames / second, 60 frames / second, or 80 frames / second, which can realize continuous image acquisition of multiple frames per second and ensure data integrity during continuous observation.

[0031] In another alternative embodiment, the target area is a target micro-topographic region.

[0032] In one optional embodiment, artificial rainmaking equipment is used, with a rainfall intensity set at 50 mm / h to cover the target micro-topographical area. During the rainfall, multiple cameras simultaneously and continuously capture images.

[0033] In another optional embodiment, before acquiring the original micro-topographical image of the target area using a multi-view camera, the method further includes: placing a calibration target in the target area; projecting random light spots onto the calibration target and acquiring the calibration target image using a multi-view camera; calculating the position coordinates of the light source in each camera coordinate system based on the camera calibration parameters and the object coordinates of the feature points on the calibration target; determining the direction vector from the light source to the feature points on the surface of the calibration target based on the position coordinates of the light source in each camera coordinate system, thereby obtaining the light source-camera joint calibration information.

[0034] In another alternative embodiment, projecting a random light spot onto a calibration target includes: emitting a laser beam through a laser generator to a diffusion optical element; modulating the laser beam through the diffusion optical element to obtain a random light spot; and projecting the random light spot onto the calibration target.

[0035] In one specific embodiment, the density of random light spots is 100-500 spots / m², and the spot diameter is 2-5mm, ensuring that clear and identifiable feature points are formed on the micro-terrain surface, thereby improving the accuracy of stereo matching.

[0036] In another specific embodiment, the laser generator can be a 532nm green laser generator with an output power of 100mW. The diffusion optical element can be a random phase plate, a diffractive optical element, or a microlens array.

[0037] In a more specific embodiment, the diffusion optical element can be a speckle-generating optical diffuser, which can scatter and homogenize the incident collimated laser beam to form a laser speckle field with random intensity distribution. The diffusion optical element is coaxially disposed in the output optical path of the green laser and located in front of the laser emitting end. The laser beam emitted by the green laser is perpendicularly incident on the surface of the diffusion optical element and output outward in the form of speckle laser after modulation.

[0038] In another specific embodiment, the laser generator is positioned at the center of the multi-view camera to ensure that the light spot uniformly covers the target area.

[0039] In another specific embodiment, the Zhang Zhengyou calibration method is adopted. A checkerboard calibration board (checkerboard size 500mm×500mm) is used to collect 20 sets of calibration images at different angles and positions. The camera's intrinsic parameters (focal length, principal point coordinates, distortion coefficient) and extrinsic parameters (relative position and attitude between cameras) are calculated by calibration software to obtain the camera calibration parameters.

[0040] Step S102: Filter the original micro-topography image to obtain the target micro-topography image.

[0041] The k-plane average statistical filtering algorithm processes the grayscale value changes of the same pixel at different time points. In one optional embodiment, filtering the original micro-topographic image to obtain the target micro-topographic image includes: for each pixel location in the original micro-topographic image, obtaining the grayscale value of that pixel location in multiple consecutive frames of the original micro-topographic image; calculating a grayscale value statistical value based on the grayscale value; and removing pixels from the original micro-topographic image based on the grayscale value statistical value to obtain the target micro-topographic image.

[0042] In one specific embodiment, the grayscale statistical values ​​can be the mean and standard deviation of the grayscale values. The target micro-topographic image is obtained by removing pixels from the original micro-topographic image based on these grayscale statistical values. This includes: calculating the mean and standard deviation of the grayscale values ​​of multiple consecutive frames of the original micro-topographic image; and removing pixels that do not belong to a preset range. For example, the preset range can be [mean value - 2 times the standard deviation, mean value + 2 times the standard deviation]. It should be noted that the preset range can be set according to actual needs, and this application does not limit it. Utilizing the temporal randomness of rain and fog noise, identification and removal are performed using statistical methods (mean and standard deviation), resulting in a very direct and efficient effect. Since the algorithm processes each pixel independently in the temporal dimension, it does not blur the image like spatial filtering. Therefore, all spatial details such as original edges and textures in the image are perfectly preserved.

[0043] In another optional embodiment, after the original micro-topographic image is filtered using the k-surface average statistical filtering algorithm, the filtered image (hereinafter referred to as the first micro-topographic image) can be filtered again. Specifically, this includes: traversing a single frame of the first micro-topographic image through a preset sliding window, calculating the grayscale statistical value of the pixels within the preset sliding window, replacing the grayscale value of the pixels within the preset sliding window with the grayscale statistical value, and obtaining the target micro-topographic image.

[0044] In one specific embodiment, the grayscale value statistics can be the average grayscale value or the mean grayscale value.

[0045] The image after being filtered by the k-surface average filtering algorithm is slightly smoothed in the local spatial domain to further improve image clarity and provide a better image foundation for subsequent feature point extraction.

[0046] The combined application of the two algorithms achieves the dual goals of "precise rain and fog suppression + slight image optimization". It not only solves the core technical problem of image acquisition distortion in rainy environments, but also provides high-fidelity and high-definition image data for subsequent feature point extraction and stereo matching, ensuring the accuracy and stability of the three-dimensional reconstruction module of this application.

[0047] In this step, effective texture information can be preserved by removing noise. For example, for images in rainy environments, the noise of bright spots (raindrop reflections) and dark spots (rain and fog shadows) can be precisely suppressed while preserving effective information of micro-topographic surfaces and loose light spots, resulting in high-definition images with a signal-to-noise ratio improvement of more than 40%.

[0048] Step S103: Use an ORB-like algorithm to extract feature points from the target micro-topography image and generate binary descriptors for the feature points to obtain multiple initial feature point sets.

[0049] It should be noted that the classic ORB (Oriented Features from Accelerated Segment Test and Rotated Binary Robust Independent Elementary Features) operator is a combination of FAST (Features from accelerated segment test) corner detection and the improved BRIEF (Binary Robust Independent Elementary Features) operator. The BRIEF descriptor (Features) is a combination of binary descriptors, with its core structure built around the geometric position and relative grayscale relationship of feature points. It lacks local grayscale statistics and does not incorporate lighting and surface normal vector constraints. Specifically, it uses FAST corner detection to quickly locate image corner points, combines image pyramids to achieve scale invariance, and determines the principal direction of feature points through the grayscale centroid method to achieve rotation invariance. Based on the principal direction of feature points, it constructs a rotation-invariant BRIEF descriptor, generates a 256-bit binary code by randomly sampling neighboring pixel pairs and using the relative magnitude of pixel grayscale values. This only reflects the relative differences in local grayscale, without quantifying the overall grayscale distribution characteristics or considering the influence of lighting changes and surface normal vectors on grayscale.

[0050] The ORB-like operator is fully compatible with the rotation and scale invariance core framework of the classic ORB operator, retaining the 256-bit binary encoding of the original ORB descriptor. The core improvement is the fusion of local gray-level statistical features and Lambertian surface normal vector information based on camera pose into the descriptor, enabling it to simultaneously possess gray-level relative relationships, gray-level distribution statistical features, and illumination and surface geometric constraints. The specific design is as follows: The FAST corner detection, scale pyramid, and principal orientation determination methods of the classic ORB are adopted to ensure the accuracy and geometric invariance of feature point localization. The gray-level matrix of the feature point's neighborhood is statistically analyzed to calculate the local gray-level information (reflecting the richness of gray levels in the neighborhood) and the local gray-level variance (reflecting the dispersion of gray levels in the neighborhood and texture contrast), and then normalized and encoded. Based on the current camera pose parameters and the light source-camera joint calibration results, the possible Lambertian surface normal vectors of the corresponding surface locations of the feature points are calculated. Combining the Lambertian cosine law, the constraint information of the surface normal vector on illumination reflection is quantized and incorporated into the feature description. The local gray-level information, variance statistical features, and illumination constraint information of the Lambertian surface normal vector are fused together with the original ORB binary descriptor to form a multi-dimensional composite descriptor.

[0051] The ORB-like operator addresses the issue of insufficient feature discrimination in low-texture regions by fusing local gray-level statistical features. It incorporates Lambertian surface normal vector information based on camera pose and combines it with the Lambertian cosine law to achieve illumination constraints and compensation, reducing the impact of illumination variations on feature matching. While maintaining the speed and geometric invariance of the ORB operator, it significantly improves the accuracy and stability of feature point matching, perfectly meeting the image matching requirements for rain-field micro-topographic erosion observation.

[0052] In one optional embodiment, the method of extracting feature points from the target micro-topography image using an ORB-like algorithm and generating binary descriptors for the feature points includes: extracting potential feature points using the FAST corner detection algorithm and generating binary descriptors for the feature points by combining them with the BRIEF descriptor.

[0053] In one optional embodiment, an ORB-like algorithm is used to extract feature points from the target micro-topography image and generate binary descriptors for the feature points, including: The following formula is used to extract feature points from the target micro-topography image to obtain the initial feature points: ; in, This represents the i-th feature point corresponding to the target micro-topography image. This represents the x-coordinate of the i-th feature point in the target micro-topography image. This represents the ordinate of the i-th feature point in the target micro-topography image. Representing feature points The binary descriptor, sgn() is the sign function, This represents the relative offset of the x-coordinate of the first point in the k-th pair of sampling points corresponding to the i-th feature point relative to the i-th feature point. This represents the relative offset of the ordinate of the first point in the k-th pair of sampling points corresponding to the i-th feature point with respect to the i-th feature point. This represents the relative offset of the second point in the k-th pair of sampling points corresponding to the i-th feature point with respect to the i-th feature point's x-coordinate. I represents the relative offset of the ordinate of the second point in the k-th pair of sampling points corresponding to the i-th feature point with respect to the i-th feature point, and I() represents the gray value.

[0054] It should be noted that each feature point includes its location coordinates. Grayscale value Main direction angle .

[0055] sgn() is a sign function that satisfies .

[0056] Step S104: Construct an initial spatial grid based on the joint calibration information of the light source and camera.

[0057] In one optional embodiment, an initial spatial mesh is constructed based on the light source-camera joint calibration information, including: Based on the joint calibration information of the light source and camera, the initial spatial mesh is constructed using the following formula: ; in, Represents the initial spatial grid. Represents two-dimensional coordinates on the camera's imaging plane. Represents the initial three-dimensional surface points. Indicates the camera's field of view. This represents the camera projection function.

[0058] It should be noted that, It may come from publicly available geographic data or be obtained through other means; this application does not limit this.

[0059] Step S105: Perform feature point matching on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs.

[0060] In one optional embodiment, feature point matching is performed on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs, including: calculating the Hamming distance between the binary descriptors of feature points in each initial feature point set within the initial spatial grid; and using feature point pairs with a Hamming distance less than a preset distance threshold as initial feature point pairs.

[0061] In one specific embodiment, feature point matching is performed on multiple initial feature point sets within the initial spatial grid using the following formula to obtain multiple initial feature point pairs: ; in, This represents the i-th feature point corresponding to the target micro-topography image. This represents the j-th feature point corresponding to the target micro-topography image. The descriptor for the i-th feature point, Let represent the descriptor of the j-th feature point, and Hamming() represent the Hamming distance. This indicates a preset distance threshold. Indicates the valid matching region. This represents the initial set of feature point pairs.

[0062] Step S106: Iteratively calculate the initial feature point pairs based on the optical flow field constraints, remove abnormal feature point pairs, and obtain intermediate feature point pairs.

[0063] In one optional embodiment, the initial feature point pairs are iteratively calculated based on optical flow field constraints to remove abnormal feature point pairs, resulting in intermediate feature point pairs, including: An optical flow field constraint equation is constructed, wherein the pixel grayscale values ​​of adjacent frame target micro-topography images are constant; within the neighborhood of feature points, the sum of squared residuals of the optical flow field constraint equation is used as the least squares objective function to solve for the target optical flow vector; the initial feature point pairs corresponding to the target optical flow vectors that are greater than a second preset threshold are removed to obtain intermediate feature point pairs.

[0064] In one specific embodiment, the expression for the optical flow field constraint equation is: ; in, This represents the target micro-topography image in frame t. Let represent the grayscale gradient of the target micro-topography image in the x-direction of space in frame t. Let represent the grayscale gradient of the target micro-topography image in the y-direction of the t-th frame, and u and v represent the optical flow vector components. This represents the temporal grayscale change rate of the target micro-topography image in frame t. u represents the optical flow velocity component in the x-direction, v represents the optical flow velocity component in the y-direction, and t represents the image number index.

[0065] In one specific embodiment, the expression for the sum of squared residuals of the optical flow field constraint equations is: ; in, Represents the horizontal coordinates of pixels within the neighborhood. W represents the vertical coordinate of a pixel within the neighborhood.

[0066] In one specific embodiment, the initial feature point pairs corresponding to target optical flow vectors whose target optical flow vectors are greater than a second preset threshold are removed using the following formula to obtain intermediate feature point pairs: ; in, Represents the set of intermediate feature point pairs. and Represents the target optical flow vector. Represents the initial set of feature point pairs. This indicates the second preset threshold.

[0067] In this step, optical flow field constraints are introduced based on continuous high frame rate image sequences to iteratively optimize the coarse matching results, limit the movement range of feature points, eliminate false matching points caused by rain and fog noise, and improve matching stability.

[0068] Step S107: Based on the Lambert cosine law, the intermediate feature point pairs are physically verified to obtain the target feature point pairs.

[0069] In one optional embodiment, the intermediate feature point pairs are physically verified by ray constraints based on the Lambert cosine law to obtain the target feature point pairs, including: obtaining the direction vector from the light source to the micro-terrain surface; and using the direction vector and the micro-terrain reflection coefficient, the intermediate feature point pairs are physically verified by ray constraints based on the Lambert cosine law to obtain the target feature point pairs.

[0070] In one specific embodiment, obtaining the direction vector from the light source to the micro-terrain surface includes: obtaining a point cloud of the micro-terrain surface, where each point can be represented as: Based on the point cloud of the micro-terrain surface and the position coordinates of the light source in each camera coordinate system, calculate the vector from the light source to the micro-terrain surface; normalize this vector to obtain the direction vector from the light source to the micro-terrain surface.

[0071] In another specific embodiment, based on the direction vector and the micro-topographical reflection coefficient, the physical law of the intermediate feature point pair is verified using the Lambert cosine law ray constraint to obtain the target feature point pair:

[0072] in, Represents the set of target feature point pairs. Let represent the gray level predicted by ray constraint based on the Lambert cosine law for the i-th point cloud. The coordinates of the target's location in the t-th frame of the micro-topography image are: The actual grayscale This indicates the grayscale error threshold.

[0073] Step S108: Based on the camera calibration parameters and light source calibration parameters, calculate the three-dimensional coordinates of the target feature point pairs using triangulation to obtain a three-dimensional point cloud spatial model of the micro-topography.

[0074] In one optional embodiment, based on camera calibration parameters and light source calibration parameters, the three-dimensional coordinates of the target feature point pair are calculated using triangulation with the following formula: ; in, Represents the three-dimensional coordinates of a pair of target feature points. This represents the pixel coordinates of the feature point in the image captured by the first camera. This represents the pixel coordinates of the feature point in the image acquired by the second camera, where K represents the camera intrinsic parameters. This represents the extrinsic parameters of the first camera. This indicates the extrinsic parameters of the second camera. This represents the photographic scale factor.

[0075] The three-dimensional reconstruction method provided in this application embodiment can work stably under a rainfall intensity of 50 mm / h, with a three-dimensional reconstruction error of ≤0.8 mm. Through ORB-like feature extraction, dual constraints, and a coarse-to-fine matching process, the feature point matching accuracy is improved to over 95%. It can clearly capture the dynamic changes such as micro-landform depressions and gully development caused by raindrop impact. The generated three-dimensional sequence data can intuitively reflect the erosion evolution process and meet the accuracy requirements for the study of micro-landform erosion in Loess landforms.

[0076] The 3D reconstruction method provided in this application proposes a specialized design for rainy environments. It achieves rain and fog interference suppression through statistical filtering algorithms and high-frame-rate synchronous acquisition. Loose, random light spots are used to enhance the surface texture of micro-topography, addressing the problem of insufficient natural texture in loess landforms. A joint calibration mechanism for multi-view cameras and light sources is designed, balancing flexibility and accuracy to provide reliable initial values ​​for reconstruction. A triple matching logic of "ORB-like feature extraction + optical flow field constraint + Lambertian ray constraint" is constructed, combined with a coarse-to-fine process, significantly improving the accuracy and stability of feature point matching. Synchronous triggering of multi-view cameras ensures data consistency, enabling continuous dynamic observation of the erosion process and filling a technological gap. The cost is lower than lidar equipment, it is easy to operate and deploy flexibly, adaptable to complex geological observation environments in the field, and facilitates large-scale application in the geological industry.

[0077] Corresponding to the three-dimensional reconstruction method provided in the embodiments of this application, the embodiments of this application also provide a three-dimensional reconstruction apparatus, such as... Figure 2 As shown, the three-dimensional reconstruction device includes: The acquisition module 201 is used to acquire the original micro-topographic images collected by the multi-view camera in the target area; Filtering module 202 is used to filter the original micro-topography image to obtain the target micro-topography image; The extraction module 203 is used to extract feature points of the target micro-topography image using an ORB-like algorithm, and generate binary descriptors of the feature points to obtain multiple initial feature point sets; Module 204 is used to construct an initial spatial mesh based on the joint calibration information of the light source and camera; Matching module 205 is used to perform feature point matching on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; Calculation module 206 is used to perform iterative calculations on the initial feature point pairs based on optical flow field constraints, remove abnormal feature point pairs, and obtain intermediate feature point pairs; Verification module 207 is used to perform physical law verification on the intermediate feature point pair based on the ray constraint of the Lambert cosine law to obtain the target feature point pair; The generation module 208 is used to calculate the three-dimensional coordinates of the target feature point pairs based on camera calibration parameters and light source calibration parameters using triangulation, thereby obtaining a three-dimensional point cloud spatial model of micro-terrain.

[0078] Corresponding to the three-dimensional reconstruction method provided in the embodiments of this application, the embodiments of this application also provide an electronic device for performing the three-dimensional reconstruction method, such as... Figure 3 As shown, the electronic device includes: a processor 301; and a memory 302 for storing a program for a 3D reconstruction method. After the device is powered on and the processor runs the program for the 3D reconstruction method, it performs the following steps: Acquire raw micro-topographic images of the target area using multi-view cameras; The original micro-topography image is filtered to obtain the target micro-topography image; The feature points of the target micro-topography image are extracted using an ORB-like algorithm, and binary descriptors of the feature points are generated to obtain multiple initial feature point sets; An initial spatial grid is constructed based on the joint calibration information of the light source and camera. Feature point matching is performed on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; Based on the optical flow field constraint, the initial feature point pairs are iteratively calculated, and abnormal feature point pairs are eliminated to obtain intermediate feature point pairs; The physical law of the intermediate feature point pair is verified based on the ray constraint of the Lambert cosine law to obtain the target feature point pair; Based on camera calibration parameters and light source calibration parameters, the three-dimensional coordinates of the target feature point pairs are calculated using triangulation to obtain a three-dimensional point cloud spatial model of the micro-topography.

[0079] Corresponding to the three-dimensional reconstruction method provided in the embodiments of this application, the embodiments of this application also provide a computer-readable storage medium storing a program for the three-dimensional reconstruction method, which is executed by a processor to perform the following steps: Acquire raw micro-topographic images of the target area using multi-view cameras; The original micro-topography image is filtered to obtain the target micro-topography image; The feature points of the target micro-topography image are extracted using an ORB-like algorithm, and binary descriptors of the feature points are generated to obtain multiple initial feature point sets; An initial spatial grid is constructed based on the joint calibration information of the light source and camera. Feature point matching is performed on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; Based on the optical flow field constraint, the initial feature point pairs are iteratively calculated, and abnormal feature point pairs are eliminated to obtain intermediate feature point pairs; The physical law of the intermediate feature point pair is verified based on the ray constraint of the Lambert cosine law to obtain the target feature point pair; Based on camera calibration parameters and light source calibration parameters, the three-dimensional coordinates of the target feature point pairs are calculated using triangulation to obtain a three-dimensional point cloud spatial model of the micro-topography.

[0080] Corresponding to the three-dimensional reconstruction method provided in the embodiments of this application, the embodiments of this application also provide a computer program containing instructions, which, when executed by a computer, cause the computer to perform the following steps: Acquire raw micro-topographic images of the target area using multi-view cameras; The original micro-topography image is filtered to obtain the target micro-topography image; The feature points of the target micro-topography image are extracted using an ORB-like algorithm, and binary descriptors of the feature points are generated to obtain multiple initial feature point sets; An initial spatial grid is constructed based on the joint calibration information of the light source and camera. Feature point matching is performed on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; Based on the optical flow field constraint, the initial feature point pairs are iteratively calculated, and abnormal feature point pairs are eliminated to obtain intermediate feature point pairs; The physical law of the intermediate feature point pair is verified based on the ray constraint of the Lambert cosine law to obtain the target feature point pair; Based on camera calibration parameters and light source calibration parameters, the three-dimensional coordinates of the target feature point pairs are calculated using triangulation to obtain a three-dimensional point cloud spatial model of the micro-topography.

[0081] It should be noted that for a detailed description of the three-dimensional reconstruction apparatus, electronic device, computer-readable storage medium and computer program product provided in the embodiments of this application, please refer to the relevant description of the three-dimensional reconstruction method embodiments provided in the embodiments of this application, which will not be repeated here.

[0082] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0083] In a typical configuration, an electronic device includes one or more processors (Central Processing Units), input / output interfaces, network interfaces, and memory.

[0084] Memory may include non-persistent storage in computer-readable media, such as random access memory and / or non-volatile memory, like read-only memory or flash memory. Memory is an example of computer-readable media.

[0085] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable operations, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, compact disc read-only memory, digital video disc or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0086] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, compact disc read-only memory, optical storage, etc.) containing computer-usable program code.

[0087] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

Claims

1. A three-dimensional reconstruction method, characterized in that, include: Acquire raw micro-topographic images of the target area using multi-view cameras; The original micro-topography image is filtered to obtain the target micro-topography image; The feature points of the target micro-topography image are extracted using an ORB-like algorithm, and binary descriptors of the feature points are generated to obtain multiple initial feature point sets; An initial spatial grid is constructed based on the joint calibration information of the light source and camera; Feature point matching is performed on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; Based on the optical flow field constraint, the initial feature point pairs are iteratively calculated, and abnormal feature point pairs are eliminated to obtain intermediate feature point pairs; The physical law of the intermediate feature point pair is verified based on the ray constraint of the Lambert cosine law to obtain the target feature point pair; Based on camera calibration parameters and light source calibration parameters, the three-dimensional coordinates of the target feature point pairs are calculated using triangulation to obtain a three-dimensional point cloud spatial model of the micro-topography.

2. The three-dimensional reconstruction method according to claim 1, characterized in that, Before acquiring the raw micro-topographic images of the target area using multi-view cameras, the method further includes: Place a calibration target in the target area; Random light spots are projected onto the calibration target, and images of the calibration target are acquired by the multi-view camera. Based on the camera calibration parameters and the object-space coordinates of the feature points on the calibration target, calculate the position coordinates of the light source in each camera coordinate system; Based on the position coordinates of the light source in each camera coordinate system, the direction vector from the light source to the feature points on the calibration target surface is determined, and the joint calibration information of the light source and camera is obtained.

3. The three-dimensional reconstruction method according to claim 2, characterized in that, The projection of random light spots onto the calibrated target includes: A laser beam is emitted from a laser generator to a diffusion optical element; The laser beam is modulated by the diffusion optical element to obtain a random light spot; The random light spot is projected onto the calibration target.

4. The three-dimensional reconstruction method according to claim 1, characterized in that, The step of filtering the original micro-topography image to obtain the target micro-topography image includes: For each pixel location in the original micro-topography image, obtain the grayscale value of the pixel location in multiple consecutive frames of the original micro-topography image; Based on the grayscale values, statistical values ​​of grayscale values ​​are calculated. The target micro-topography image is obtained by removing pixels from the original micro-topography image based on the grayscale statistical values.

5. The three-dimensional reconstruction method according to claim 1, characterized in that, The step of performing feature point matching on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs includes: Calculate the Hamming distance between the binary descriptors of feature points in each of the initial feature point sets within the initial spatial grid; Feature point pairs with a Hamming distance less than a preset distance threshold are used as initial feature point pairs.

6. The three-dimensional reconstruction method according to claim 1, characterized in that, The iterative calculation of the initial feature point pairs based on optical flow field constraints, eliminating abnormal feature point pairs, and obtaining intermediate feature point pairs includes: An optical flow field constraint equation is constructed, wherein the pixel grayscale values ​​of the target micro-topography images in adjacent frames are constant; Within the neighborhood of the feature point, the sum of squared residuals of the optical flow field constraint equations is used as the least squares objective function to solve for the target optical flow vector; The initial feature point pairs corresponding to the target optical flow vectors that are greater than the second preset threshold are removed to obtain intermediate feature point pairs.

7. The three-dimensional reconstruction method according to claim 1, characterized in that, The physical law verification of the intermediate feature point pairs based on the Lambert cosine law ray constraint is used to obtain the target feature point pairs, including: Obtain the direction vector from the light source to the micro-terrain surface; Based on the direction vector and micro-topographical reflection coefficient, and using the Lambert cosine law ray constraint, the intermediate feature point pair is physically verified to obtain the target feature point pair.

8. A three-dimensional reconstruction device, characterized in that, include: The acquisition module is used to acquire raw micro-topographic images of the target area collected by the multi-view camera; A filtering module is used to filter the original micro-topography image to obtain the target micro-topography image; The extraction module is used to extract feature points from the target micro-topography image using an ORB-like algorithm, and generate binary descriptors for the feature points to obtain multiple initial feature point sets. The building module is used to construct the initial spatial mesh based on the joint calibration information of the light source and camera; The matching module is used to perform feature point matching on multiple initial feature point sets within the initial spatial grid to obtain multiple initial feature point pairs; The calculation module is used to perform iterative calculations on the initial feature point pairs based on optical flow field constraints, remove abnormal feature point pairs, and obtain intermediate feature point pairs; The verification module is used to verify the physical laws of the intermediate feature point pairs based on the ray constraint of the Lambert cosine law to obtain the target feature point pairs; The generation module is used to calculate the three-dimensional coordinates of the target feature point pairs based on camera calibration parameters and light source calibration parameters using triangulation, thereby obtaining a three-dimensional point cloud spatial model of micro-terrain.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the three-dimensional reconstruction method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the three-dimensional reconstruction method according to any one of claims 1-7.

Images