Personnel positioning method and system based on 3D Gaussian splash model and video fusion

Abstract

The present invention discloses a personnel positioning method and system based on a 3D Gaussian splash model and video fusion. The method first obtains input data from at least two perspectives through a multi-perspective video stream acquisition module, and uses a sparse point cloud initialization module to construct an initial three-dimensional Gaussian model; uses a temporal dynamic tracking module combined with an optical flow algorithm to achieve cross-frame parameter updates of the Gaussian ellipsoid, and uses a temporal consistency optimization module to suppress parameter drift; uses a target detection module to identify the construction personnel boundary box, and uses a three-dimensional-two-dimensional space matching module to establish a correspondence between pixels and three-dimensional coordinates; calculates three-dimensional coordinates through a multi-perspective fusion positioning module, and improves positioning accuracy through a multi-sensor fusion optimization module combined with IMU data. Combining the 3D Gaussian splash model with multi-perspective geometry and temporal optimization, high-precision dynamic positioning of personnel without marking is achieved in construction scenarios, effectively solving the tracking drift and occlusion problems of traditional methods in complex environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and construction safety monitoring, and particularly relates to a personnel positioning method and system based on a 3D Gaussian splash model and video fusion. Background Art

[0002] With the rapid development of the construction industry, construction site safety management and personnel positioning technology are gaining increasing attention. Traditional methods for positioning construction personnel primarily rely on active sensor technologies such as the Global Positioning System (GPS), radio frequency identification (RFID), or ultra-wideband (UWB). However, these methods suffer from signal obstruction, strong equipment dependence, and high deployment costs in complex indoor or semi-open construction environments. Furthermore, vision-based markerless positioning methods have gradually become a research hotspot due to their advantages such as requiring no additional equipment and flexible deployment. However, existing visual positioning technologies still face challenges such as poor adaptability to dynamic scenes, multi-target occlusion, and high computational complexity.

[0003] Traditional construction worker positioning technologies are primarily categorized as sensor-based and vision-based. Sensor-based methods (such as GPS, UWB, and RFID) perform well in open environments, but are susceptible to signal attenuation and multipath effects indoors or in densely populated environments, resulting in reduced positioning accuracy. Furthermore, these methods require construction workers to wear specialized equipment, increasing management costs and personnel burdens. While vision-based markerless positioning methods (such as deep learning-based target detection and tracking) avoid device dependency, they are prone to target loss or mismatching in complex dynamic scenes due to factors such as lighting changes, occlusion, and changes in viewing angle.

[0004] In recent years, significant progress has been made in visual positioning technology based on 3D reconstruction. Structure from Motion (SFM) and Simultaneous Localization and Mapping (SLAM) techniques can recover the 3D structure of a scene from multi-view images, providing a spatial reference for locating people. However, traditional SFM / SLAM methods rely on sparse feature point matching, making it difficult to stably track moving targets in dynamic scenes. Furthermore, voxel- or grid-based 3D representations (such as NeRF) have high computational complexity and struggle to meet real-time requirements.

[0005] 3D Gaussian Splatting is an emerging 3D scene representation method that efficiently models scenes using parameterized Gaussian ellipsoids, combining high rendering quality with real-time computational capabilities. Compared to traditional point cloud or voxel representations, the 3D Gaussian splatting model is more flexible in adapting to dynamic scene changes and supports efficient optical flow tracking and parameter optimization. However, existing methods still have shortcomings in maintaining temporal consistency, dynamic target tracking, and multi-sensor fusion. This is particularly true in construction scenarios, where complex human motion and frequent occlusions lead to accumulated positioning errors and drift.

[0006] Multi-view video fusion is an effective means of improving positioning accuracy, but existing methods lack systematic design in terms of viewpoint selection, feature matching, and triangulation optimization. For example, insufficient geometric constraints on back-projected rays can lead to biased depth estimation, while simple linear interpolation struggles to handle motion blur or occlusion. Furthermore, incomplete fusion strategies for auxiliary sensor data, such as inertial measurement units (IMUs), fail to fully utilize motion prior information to suppress positioning jitter.

[0007] Therefore, there is an urgent need to develop a new method for dynamic video fusion and construction worker positioning to solve the problems of target loss or mismatching caused by factors such as equipment dependence, lighting changes, occlusion, and perspective changes. Summary of the Invention

[0008] In light of this, it is necessary to provide a personnel localization method and system based on a 3D Gaussian splatter model and video fusion. This method achieves high-precision and robust dynamic personnel localization through techniques such as sparse initialization, optical flow tracking, timing optimization, and multi-view triangulation. This method not only avoids the use of active sensors but also significantly improves positioning stability and accuracy in complex environments through dynamic Gaussian model updates and IMU data fusion.

[0009] In a first aspect, an embodiment of the present application provides a method for positioning a person based on a 3D Gaussian splash model and video fusion, the method comprising:

[0010] S1, collects video streams from at least two perspectives as input data;

[0011] S2, generates an initial Gaussian distribution for the first frame of video through sparse initialization based on the SFM point cloud, constructs the pose and density parameters of the three-dimensional Gaussian ellipsoid. Each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color and opacity parameters, and constructs the initial three-dimensional Gaussian model;

[0012] S3, for subsequent frames, tracks the displacement changes of the three-dimensional Gaussian ellipsoid through the optical flow algorithm, and dynamically updates its position, covariance matrix and color feature parameters;

[0013] S4, maintains the parameter consistency of the Gaussian model in the time dimension through the L2 regularized temporal optimization algorithm that minimizes parameter drift;

[0014] S5, using the Mask R-CNN object detection model to extract the bounding box of the construction workers in the video frame;

[0015] S6, spatially matching the detected personnel area with the three-dimensional Gaussian ellipsoid area with the highest density in the Gaussian model to establish a spatial correspondence between pixel coordinates and three-dimensional coordinates;

[0016] S7, calculates the three-dimensional coordinates of the person through perspective geometry back-projection, and performs triangulation to improve positioning accuracy if multi-view input is used;

[0017] S8, integrates the inertial measurement unit (IMU) data and the motion prediction model to perform temporal smoothing and error suppression on the positioning results.

[0018] Optionally, in an implementation of the first aspect of the present invention, the sparse initialization in S2 specifically includes:

[0019] Use COLMAP to perform motion structure recovery (SFM) reconstruction on the first frame of multi-view video and extract sparse point cloud;

[0020] Initialize the Gaussian ellipsoid with the point cloud center as the mean and the point cloud distribution as the covariance;

[0021] Spherical harmonics are used to encode the viewing angle-dependent color features, and the initial opacity is set to a value in the range of 0.8-1.0.

[0022] Optionally, in an implementation of the first aspect of the present invention, generating an initial Gaussian distribution includes:

[0023] Combining a three-dimensional Gaussian distribution with a temporal component, temporal opacity is encoded via temporal radial basis functions to model the emergence or disappearance of scene content;

[0024] The position and rotation parameters of the three-dimensional Gaussian distribution are represented by a time-conditioned parameter function to model the motion and deformation in the scene.

[0025] For any point in space and time , whose opacity is calculated as a temporal opacity function The product of the spatial Gaussian distribution, where the spatial location and covariance of the spatial Gaussian distribution are 、 is the time-dependent function, where the opacity of the Gaussian distribution is:

[0026] ,

[0027] in, Indicates time opacity, 、 are the time-dependent spatial locations and covariances, Representative A Gaussian distribution.

[0028] The time radial basis function is implemented using a learnable neural network or a preset basis function group to control the visible and invisible state changes of the Gaussian distribution on the time axis, which is expressed as:

[0029] ,

[0030] in, is the time center, indicating the most prominent timestamp of the Gaussian distribution, Represents the time scaling factor, which is used to determine its effective duration. Indicates time-independent spatial opacity.

[0031] For each Gaussian distribution, a time condition function is used to model the changes in the position and rotation parameters of the three-dimensional Gaussian ellipsoid in the time dimension, which is expressed as:

[0032] ,

[0033] in, Represents a Gaussian distribution over time spatial location, , is the corresponding coefficient, ;

[0034] Compute the pixel's color based on the final spatial opacity , expressed as:

[0035] ,

[0036] in, It is Gaussian colors.

[0037] Optionally, in an implementation of the first aspect of the present invention, the optical flow algorithm tracking in S3 adopts a hybrid tracking strategy combining Farneback dense optical flow and LK sparse optical flow, including:

[0038] The Farneback algorithm is used to estimate global dense optical flow. The optical flow information of each pixel in the entire image is calculated using the information of the previous and next two frames of images. The image frame is approximated by polynomial expansion, and the least squares method is used to estimate the displacement of the pixel points.

[0039] The LK algorithm is used to track local details of sparse optical flow, select feature points for local matching, and only track the movement of some key points in the image.

[0040] During the tracking process, if the displacement of the Gaussian ellipsoid is detected to exceed the preset threshold, the system will start the reinitialization process, re-detect the target feature points, and update the covariance matrix and color feature parameters.

[0041] In each frame of the video sequence, a color histogram similarity measure between a target region of the current frame and a target region of at least one adjacent frame is calculated.

[0042] When the similarity measure is lower than a preset threshold, it is determined that a significant change or occlusion event occurs in the target;

[0043] In response to the determination, a reinitialization operation of the tracker is triggered.

[0044] Optionally, in an implementation of the first aspect of the present invention, the timing optimization of S4 includes:

[0045] Establish Markov constraints on the change of Gaussian parameters between adjacent frames:

[0046] Applying positional parameters The second-order difference constraint is used to prevent the parameters from changing drastically in a short period of time. The constraint is expressed as:

[0047] ,

[0048] in, is the constraint threshold, which is used to control the amplitude of parameter changes. Respectively indicate that at adjacent times The positional parameters.

[0049] Exponential decay weighting is used to adjust the influence weight of historical frame parameters, giving small weights to the parameters of distant frames and large weights to the parameters of close frames. Defined as:

[0050] ,

[0051] in is the decay factor, which is used to control the decay rate of the weight, and is used to pay more attention to recent data in the time series while avoiding forgetting historical information too early.

[0052] In order to minimize parameter drift, the L2 regularization term is introduced to penalize the change of parameters. The optimization objective function is expressed as:

[0053] ,

[0054] in, is the weight of the first-order difference constraint, is the L2 regularization weight, is the average parameter value of the current frame.

[0055] Optionally, in an implementation of the first aspect of the present invention, the spatial matching in S6 includes:

[0056] Compute the density-weighted average of all Gaussian ellipsoids within the person's bounding box.

[0057] Gaussian clusters with opacity > 0.9 and density ranking in the top 5% are selected as matching targets.

[0058] Abnormal Gaussian distribution is eliminated by RANSAC algorithm.

[0059] Optionally, in an implementation of the first aspect of the present invention, during the multi-view triangulation in S7.

[0060] Select a perspective combination with a baseline distance greater than 2 meters. A large baseline distance can significantly increase the angle of the back-projected ray, thereby enhancing the geometric constraints of triangulation and reducing calculation errors.

[0061] A depth consistency check is performed on matching points with a back-projection ray angle less than 15°. By comparing the depth relationship between the back-projection ray and the target point under multiple perspectives, it is determined whether it conforms to physical laws. If not, the matching point is eliminated to avoid misleading the estimation of the final three-dimensional coordinates.

[0062] The LM algorithm is used to perform nonlinear optimization of the three-dimensional coordinates. By minimizing the reprojection error, the matching points under multiple perspectives are jointly optimized to make the final three-dimensional coordinates close to the true value.

[0063] Optionally, in an implementation of the first aspect of the present invention, the fusing of the inertial measurement unit (IMU) data in S8 is specifically:

[0064] Establish a loosely coupled fusion framework based on error state Kalman filtering;

[0065] When the Gaussian model positioning confidence is lower than the threshold, it switches to IMU dead reckoning;

[0066] Predicting the periodic positioning deviation caused by the swing of human lower limbs through kinematic model.

[0067] In a second aspect, an embodiment of the present application provides a personnel positioning system based on a 3D Gaussian splash model and video fusion, which is applied to the personnel positioning method based on a 3D Gaussian splash model and video fusion as described in the first aspect, including:

[0068] The multi-view video stream acquisition module is used to acquire video streams from at least two perspectives as input data.

[0069] The sparse point cloud initialization module is used to generate an initial Gaussian distribution for the first frame of video through sparse initialization based on the SFM point cloud, and construct the pose and density parameters of the three-dimensional Gaussian ellipsoid. Each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color and opacity parameters to construct an initial three-dimensional Gaussian model.

[0070] The temporal dynamic tracking module is used to track the displacement changes of the three-dimensional Gaussian ellipsoid through the optical flow algorithm in subsequent frames, and dynamically update its position, covariance matrix and color feature parameters.

[0071] The temporal consistency optimization module is used to maintain the parameter consistency of the Gaussian model in the time dimension through an L2 regularized temporal optimization algorithm that minimizes parameter drift.

[0072] The object detection module is used to extract the bounding boxes of construction workers in video frames using the Mask R-CNN object detection model.

[0073] The 3D-2D spatial matching module is used to spatially match the detected personnel area with the 3D Gaussian ellipsoid area with the highest density in the Gaussian model, and establish a spatial correspondence between pixel coordinates and 3D coordinates.

[0074] The multi-view fusion positioning module is used to calculate the three-dimensional coordinates of the person through perspective geometry back projection. If multi-view input is used, triangulation is performed to improve positioning accuracy.

[0075] The multi-sensor fusion optimization module is used to fuse the inertial measurement unit (IMU) data and the motion prediction model to perform temporal smoothing and error suppression on the positioning results.

[0076] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0077] processor;

[0078] a memory for storing processor-executable instructions;

[0079] Wherein, the processor is configured to implement the personnel positioning method based on 3D Gaussian splash model and video fusion as described in the first aspect when executing the instructions.

[0080] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program, and the program instructs a device to execute the personnel positioning method based on the 3D Gaussian splash model and video fusion as described in the first aspect.

[0081] The present invention discloses a personnel positioning method and system based on a 3D Gaussian splash model and video fusion. The method first obtains input data from at least two perspectives through a multi-perspective video stream acquisition module, and uses a sparse point cloud initialization module to construct an initial three-dimensional Gaussian model; uses a temporal dynamic tracking module combined with an optical flow algorithm to implement cross-frame parameter updates of the Gaussian ellipsoid, and uses a temporal consistency optimization module to suppress parameter drift; uses a target detection module to identify the construction personnel boundary box, and uses a three-dimensional-two-dimensional space matching module to establish a correspondence between pixels and three-dimensional coordinates; finally, the three-dimensional coordinates are calculated through a multi-perspective fusion positioning module, and the positioning accuracy is improved through a multi-sensor fusion optimization module combined with IMU data. The present invention innovatively combines the 3D Gaussian splash model with multi-perspective geometry and temporal optimization to achieve high-precision dynamic positioning of personnel without marking in construction scenarios, effectively solving the tracking drift and occlusion problems of traditional methods in complex environments.

[0082] Beneficial effects:

[0083] (1) High-precision dynamic positioning: By integrating multi-view video streams with a 3D Gaussian splash model, combined with optical flow tracking and timing optimization algorithms, sub-meter dynamic positioning accuracy for construction workers is achieved, which is more than 40% higher than traditional visual positioning methods.

[0084] (2) Strong anti-interference capability: The Gaussian model parameter update mechanism with spatiotemporal joint optimization is adopted to effectively resist interference such as illumination changes and partial occlusion, and the tracking success rate in complex construction scenes is increased to 92%.

[0085] (3) Superior real-time performance: Through the layered optical flow tracking strategy (dense + sparse optical flow combination) and L2 regularization timing optimization, the system processing speed reaches 25fps, meeting the real-time monitoring needs of the construction site.

[0086] (4) Advantages of multi-source data fusion: The innovatively designed loosely coupled IMU fusion framework can automatically switch to inertial navigation when visual positioning fails, and the positioning continuity is improved by 3 times compared with the traditional pure vision method.

[0087] (5) Low deployment cost: No pre-installed marking equipment or special sensors are required. Only 2-4 ordinary surveillance cameras are needed to achieve full area coverage, reducing system deployment costs by more than 60%. The accuracy matching mechanism automatically adapts to the increase and decrease of personnel and equipment movement in the construction scene, and the model update delay is less than 0.5 seconds. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 A flowchart of a personnel positioning method based on a 3D Gaussian splash model and video fusion is provided in one embodiment of the present application.

[0089] Figure 2A diagram of a layered system architecture provided for one embodiment of the present application.

[0090] Figure 3 A data flow diagram provided for an embodiment of the present application.

[0091] Figure 4 A schematic diagram of a personnel positioning system module based on a 3D Gaussian splash model and video fusion provided in one embodiment of the present application.

[0092] Figure 5 A schematic diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0093] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0094] It should be noted that, in the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art in the art to which this application relates. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0095] It should be noted that, in the embodiments of the present application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order. Features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0096] Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0097] Example 1

[0098] Figure 1This is a flowchart of a personnel localization method based on a 3D Gaussian splatter model and video fusion, provided in one embodiment of this application. The entire process combines techniques such as SFM, optical flow, Mask R-CNN, and Gaussian splattering to extract the 3D coordinates of construction workers from multi-view video streams. This process then smooths and mitigates errors through timing optimization and IMU data. Figure 2 This is a diagram of a layered system architecture provided in one embodiment of the present application. The system includes an input layer, a core processing layer, and an output layer. Figure 3 A data flow diagram provided for an embodiment of the present application.

[0099] like Figure 1-3 As shown, a personnel positioning method based on a 3D Gaussian splash model and video fusion includes:

[0100] S1, collect video streams from at least two perspectives as input data.

[0101] In 3D Gaussian Splatting technology, multi-view input is crucial. Rotating drone footage provides a better perspective. In vision-based markerless person localization, single-view video is limited by the inherent flaws of two-dimensional imaging (such as occlusion, perspective distortion, and missing depth information), making it difficult to stably and accurately estimate the target's 3D position. Multi-view video streams, however, provide richer spatial information, enhancing the robustness of 3D reconstruction through disparity calculation and triangulation, thereby improving positioning accuracy. Therefore, multi-view input is crucial for 3D reconstruction and pose estimation.

[0102] S2, generates an initial Gaussian distribution for the first frame of video through sparse initialization based on SFM point cloud, constructs the pose and density parameters of a three-dimensional Gaussian ellipsoid. Each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color and opacity parameters to construct an initial three-dimensional Gaussian model.

[0103] SFM (Structure from Motion) technology is used to estimate sparse point clouds and camera poses from multi-view images. Each 3D Gaussian is defined by its mean, covariance, opacity, and view-dependent color (SH). These parameters together constitute the pose and density parameters of the 3D Gaussian ellipsoid.

[0104] The core goal of this step is to reconstruct the sparse 3D scene structure from the first frame of multi-view video and initialize the 3D Gaussian Splatting model based on this structure to provide initial parameters for subsequent dynamic tracking and optimization.

[0105] Specifically, the sparse initialization in S2 involves performing structure-from-motion (SFM) reconstruction on the first frame of the multi-view video using COLMAP to extract a sparse point cloud. Using the COLMAP tool, SFM reconstruction is performed on the first frame of the multi-view video to recover the sparse point cloud of the scene. COLMAP gradually constructs the sparse point cloud of the scene and the camera pose through steps such as feature extraction, matching, geometric verification, and optimization. This process typically includes feature extraction (e.g., SIFT), feature matching, geometric constraints (e.g., RANSAC optimization), and sparse reconstruction (using SBA optimization).

[0106] A Gaussian ellipsoid is initialized using the point cloud center as its mean and the point cloud distribution as its covariance. Based on a sparse point cloud, each point is initialized as a 3D Gaussian function. The center position of this Gaussian function is determined by the point in the point cloud, while the covariance matrix describes the distribution of the point cloud. Specifically, each Gaussian ellipsoid is defined by its spatial position (mean) and covariance matrix; these parameters together determine the shape and density of the Gaussian in three-dimensional space.

[0107] Spherical harmonics are used to encode view-dependent color features, and the initial opacity is set to a value in the range of 0.8-1.0. To represent view-dependent color features, spherical harmonics (SH) are often used to encode colors. Spherical harmonics can effectively represent color changes under changes in lighting and view angle, thereby maintaining color consistency during the rendering process. The color parameters of each Gaussian ellipsoid are represented by SH coefficients, which can be trained and optimized using the color information in the point cloud. The initial opacity is usually set between 0.8 and 1.0 to ensure good visibility of the Gaussian model during rendering while avoiding loss of details due to excessive transparency. This parameter can be adjusted experimentally to achieve the best visual effect.

[0108] Generating an initial Gaussian distribution includes combining a three-dimensional Gaussian distribution with a temporal component and encoding temporal opacity via a temporal radial basis function to model the appearance or disappearance of scene content. Combining the three-dimensional Gaussian distribution with the temporal component, expanding the 3D Gaussian to a 4D spatiotemporal domain, and constructing a spatiotemporal Gaussian distribution to model dynamic scenes. Temporal opacity and parameterized motion / rotation enhance expressiveness, thereby capturing static, dynamic, and transient content in the scene.

[0109] The position and rotation parameters of the three-dimensional Gaussian distribution are represented by a time-conditioned parameter function to model the motion and deformation in the scene.

[0110] For any point in space and time , whose opacity is calculated as a temporal opacity function The product of the spatial Gaussian distribution, where the spatial location and covariance of the spatial Gaussian distribution are 、 is the time-dependent function, where the opacity of the Gaussian distribution is:

[0111] ,

[0112] in, Indicates time opacity, 、 are the time-dependent spatial locations and covariances, Representative A Gaussian distribution.

[0113] The time radial basis function is implemented using a learnable neural network or a preset basis function group to control the visible and invisible state changes of the Gaussian distribution on the time axis, which is expressed as:

[0114] ,

[0115] in, is the time center, indicating the most prominent timestamp of the Gaussian distribution, Represents the time scaling factor, which is used to determine its effective duration. Represents time-independent spatial opacity. A temporal radial basis function is used to represent temporal opacity at any point in time using a Gaussian distribution. This function effectively models scene content that appears or disappears over the duration of a video.

[0116] In order to model the motion and deformation in the scene, the position and rotation parameters of each Gaussian distribution are parameterized by a time condition function. For each Gaussian distribution, a time condition function is used to model the changes in the position and rotation parameters of the three-dimensional Gaussian ellipsoid in the time dimension, which is expressed as:

[0117] ,

[0118] in, Represents a Gaussian distribution over time spatial location, , is the corresponding coefficient, ; Calculate the pixel color based on the final spatial opacity , expressed as:

[0119] ,

[0120] in, It is Through the above steps, the generated initial Gaussian distribution can effectively model the spatiotemporal changes in dynamic scenes, thereby achieving high-quality real-time rendering and compact model representation.

[0121] S3, for subsequent frames, tracks the displacement changes of the three-dimensional Gaussian ellipsoid through the optical flow algorithm, and dynamically updates its position, covariance matrix and color feature parameters.

[0122] Specifically, the optical flow algorithm tracking in S3 adopts a hybrid tracking strategy that combines Farneback dense optical flow and LK sparse optical flow, including: using the Farneback algorithm to perform global dense optical flow estimation, calculating the optical flow information of each pixel in the entire image through the image information of the previous and next two frames, approximating the image frame through polynomial expansion, and estimating the displacement of the pixel points using the least squares method.

[0123] The Gunnar Farneback algorithm is used for global dense optical flow estimation, calculating the optical flow information for each pixel in the entire image using information from two consecutive image frames. This algorithm constructs an image pyramid structure, approximates the image frames using polynomial expansion, and estimates pixel displacement using the least squares method. This method provides a highly accurate global motion field and is suitable for capturing large-scale displacement changes.

[0124] The Lucas-Kanade (LK) algorithm is used for local detail tracking using sparse optical flow. Feature points are selected for local matching, tracking the motion of only a few key points in the image. Within the sparse optical flow portion, the Lucas-Kanade (LK) algorithm is employed for local detail tracking. This algorithm detects corner points in the image and calculates the motion vectors of local feature points based on the assumptions of constant brightness and minimal motion. The LK algorithm is computationally efficient and suitable for tracking local details, such as the edges of a Gaussian ellipsoid or key regions.

[0125] By combining Farneback dense optical flow with LK sparse optical flow, the system can simultaneously obtain global motion information and local detail information. This hybrid strategy is more robust when processing complex scenes, especially when the target undergoes significant changes or occlusions, and can more accurately capture the displacement changes of the Gaussian ellipsoid.

[0126] During tracking, if the displacement of the Gaussian ellipsoid exceeds a preset threshold, the system initiates a reinitialization process, redetecting the target feature points and updating the covariance matrix and color feature parameters. During tracking, if the displacement of the Gaussian ellipsoid exceeds a preset threshold, the system initiates a reinitialization process. This process involves redetecting the target feature points and updating the covariance matrix and color feature parameters. The purpose of reinitialization is to ensure parameter consistency of the Gaussian model in the temporal dimension.

[0127] In each frame of a video sequence, a color histogram similarity metric is calculated between the target region of the current frame and the target region of at least one adjacent frame. When the similarity metric falls below a preset threshold, a significant change or occlusion event is determined to have occurred in the target. In response to this determination, a reinitialization operation of the tracker is triggered.

[0128] In each frame of a video sequence, a color histogram similarity metric is calculated between the target region of the current frame and the target region of at least one adjacent frame. When the similarity metric falls below a preset threshold, a significant target change or occlusion event is determined. This mechanism helps promptly detect abnormal target behavior, thereby triggering a tracker reinitialization operation. In response to this determination, a tracker reinitialization operation is triggered. This operation includes redetecting target feature points and updating the covariance matrix and color feature parameters to ensure parameter consistency of the Gaussian model in the temporal dimension.

[0129] Through the above hybrid optical flow tracking strategy, the system can effectively track the displacement changes of the three-dimensional Gaussian ellipsoid and reinitialize it when necessary to maintain the parameter consistency of the Gaussian model.

[0130] S4 maintains the parameter consistency of the Gaussian model in the time dimension by minimizing the parameter drift through the L2 regularized temporal optimization algorithm.

[0131] Specifically, the timing optimization in S4 includes establishing Markov constraints on the variation of Gaussian parameters between adjacent frames. By introducing a Markov model, the variation of Gaussian parameters between adjacent frames is constrained. For example, a second-order difference constraint is imposed on the position parameters to prevent drastic parameter changes in a short period of time.

[0132] Applying positional parameters The second-order difference constraint is used to prevent the parameters from changing drastically in a short period of time. The constraint is expressed as:

[0133] ,

[0134] in, is the constraint threshold, which is used to control the amplitude of parameter changes. Respectively indicate that at adjacent times The positional parameters.

[0135] In order to pay more attention to recent data and avoid forgetting historical information too early, an exponential decay weighting strategy is adopted. Exponential decay weighting is used to adjust the weight of historical frame parameters, giving small weights to the parameters of distant frames and large weights to the parameters of close frames. Defined as:

[0136] ,

[0137] in is the decay factor, which is used to control the decay rate of the weight, and is used to pay more attention to recent data in the time series while avoiding forgetting historical information too early.

[0138] In order to minimize parameter drift, the L2 regularization term is introduced to penalize the change of parameters. The optimization objective function is expressed as:

[0139] ,

[0140] in, is the weight of the first-order difference constraint, is the L2 regularization weight, is the average parameter value of the current frame.

[0141] The objective function is solved using gradient descent or other optimization algorithms. During the optimization process, gradients must be calculated and parameters updated. L2 regularization increases the diagonal terms of the parameters, making the model more likely to choose small parameter values ​​during training, thereby reducing overfitting and parameter drift.

[0142] In 3D Gaussian splatter technology, this temporal optimization method is used to model dynamic scenes. By constraining the temporal variation of Gaussian parameters, the model's temporal stability is ensured. By introducing spatiotemporal consistency constraints, the variations in Gaussian parameters in dynamic scenes can be effectively modeled. Through this approach, the temporal optimization algorithm in S4 effectively maintains the temporal parameter consistency of the Gaussian model, thereby improving the accuracy and robustness of 3D reconstruction.

[0143] S5, uses the Mask R-CNN object detection model to extract the bounding box of the construction workers in the video frame.

[0144] Specifically, in S5, the Mask R-CNN object detection model is used to extract the bounding boxes of construction workers in video frames. The specific steps are as follows: 1. Model loading and initialization. A pretrained Mask R-CNN model is used, typically based on a backbone network such as ResNet-50 or ResNet-152. After loading the model, the category label file (such as mscoco_labels.names) and color map file (such as colors.txt) need to be loaded for subsequent bounding box and mask visualization. 2. Video frame processing. Each frame is read from the video stream and preprocessed (such as normalization and resizing) to meet the model input requirements. The input image size is typically 400×400 pixels. 3. Object detection and instance segmentation. The preprocessed image is input to the Mask R-CNN model. The model first generates candidate regions (ROIs) using a region proposal network (RPN). Then, each candidate region is classified and subjected to bounding box regression, while simultaneously generating a mask for instance segmentation. The mask output is a four-dimensional tensor representing the pixel-level segmentation results of each detected object. 4. Bounding box and mask extraction. Extract the bounding box and mask of each detected object from the model output. The bounding box is usually determined by the confidence score and regression offset, while the mask achieves pixel-level accurate segmentation through the ROIAlign operation. If the confidence of a bounding box is lower than the preset threshold, it is ignored. 5. Result visualization. The extracted bounding box and mask are drawn on the original image, which usually includes the following steps: Drawing the bounding box: Use OpenCV's rectangle function to draw the bounding box and annotate the category label and confidence. Drawing the mask: Convert the mask to a binary image and use the contour function to draw the contour for enhanced visualization. Color mapping: Use a color map file to assign different colors to each detected object based on the category label to distinguish targets of different categories. 6. Output and save. Save the processed image as a new video file or display the detection results in a live video stream. For construction worker detection, other modules (such as steps 6 and 7) can be combined to perform 3D coordinate calculation and spatial matching. 7. Performance Optimization. To increase detection speed, a lightweight backbone network (such as MobileNet) can be used or optimization techniques such as model pruning and quantization can be employed. Furthermore, data augmentation (such as rotation, scaling, and flipping) can be used to improve the model's generalization capabilities.

[0145] Through the above steps, Mask R-CNN can efficiently extract the bounding boxes of construction workers from video frames and generate accurate instance segmentation masks, providing key information for subsequent 3D reconstruction and spatial matching.

[0146] S6, spatially matching the detected personnel area with the three-dimensional Gaussian ellipsoid area with the highest density in the Gaussian model, and establishing a spatial correspondence between pixel coordinates and three-dimensional coordinates.

[0147] Specifically, the spatial matching in S6 includes calculating the density-weighted average of all Gaussian ellipsoids within the person's bounding box. Within the detected person's bounding box, the density-weighted average of all Gaussian ellipsoids is calculated. This average reflects the concentration of Gaussian distribution in the area and is used to identify the Gaussian ellipsoid most likely to contain the target.

[0148] Filter Gaussian clusters with opacity greater than 0.9 and density ranking in the top 5% as matching targets. Filter Gaussian clusters with opacity greater than 0.9 and density ranking in the top 5% from all Gaussian ellipsoids as matching targets. These Gaussian clusters have higher visibility and density and are more likely to correspond to detected human regions.

[0149] The RANSAC algorithm is used to remove anomalous Gaussian distributions. The RANSAC algorithm processes the selected Gaussian clusters to remove anomalous distributions. The RANSAC algorithm randomly selects matching point pairs, calculates the homography matrix, and verifies the spatial consistency of all matching point pairs, thereby removing incorrectly matched Gaussian distributions. The improved RANSAC algorithm can also improve computational accuracy and robustness by increasing the set of matching point pairs.

[0150] Establish a spatial correspondence between pixel coordinates and 3D coordinates. After removing abnormal Gaussian distributions, the remaining Gaussian clusters form a spatial correspondence with the detected person regions. By matching the coordinate information of point pairs, a mapping relationship between pixel coordinates and 3D coordinates can be established, thereby achieving 3D reconstruction of the person region.

[0151] S7, calculates the three-dimensional coordinates of the person through perspective geometry back-projection. If multi-view input is used, triangulation is performed to improve positioning accuracy.

[0152] Specifically, during multi-view triangulation in S7, a view combination with a baseline distance greater than 2 meters is selected. A larger baseline distance significantly increases the angle of the back-projected rays, thereby strengthening the geometric constraints of the triangulation and reducing computational errors. In the case of multi-view input, a view combination with a baseline distance greater than 2 meters is selected. This strategy helps improve the accuracy of 3D coordinate estimation.

[0153] A depth consistency check is performed on matching points with back-projection ray angles less than 15°. By comparing the depth relationship between the back-projection ray and the target point from multiple viewpoints, the accuracy of the check is determined. If not, the matching point is discarded to avoid misleading estimates of the final 3D coordinates. A depth consistency check is performed on matching points with back-projection ray angles less than 15°. By comparing the depth relationship between the back-projection ray and the target point from multiple viewpoints, the accuracy of the check is determined. If not, the matching point is discarded to avoid misleading estimates of the final 3D coordinates.

[0154] The Levenberg-Marquardt (LM) algorithm is used for nonlinear optimization of 3D coordinates. By minimizing the reprojection error, matching points from multiple perspectives are jointly optimized, bringing the final 3D coordinates close to the true values. The Levenberg-Marquardt (LM) algorithm is used for nonlinear optimization of 3D coordinates. By minimizing the reprojection error, matching points from multiple perspectives are jointly optimized, bringing the final 3D coordinates close to the true values. The LM algorithm combines the steepest descent method with the Gauss-Newton method, making it an efficient optimization algorithm that effectively handles nonlinear least squares problems.

[0155] During the optimization process, a reprojection error objective function is constructed and solved iteratively to obtain the optimal three-dimensional coordinate estimation. This objective function is usually expressed as: ,in, is the estimated 3D point, is the projection of the point on the image, is the observed image point. By minimizing this error, a more accurate 3D coordinate estimate can be obtained. When used for 3D positioning of people in multi-view video streams, this method significantly improves the accuracy of 3D coordinate estimation by selecting appropriate view combinations, performing depth consistency checks, and employing the LM algorithm for nonlinear optimization.

[0156] S8, integrates the inertial measurement unit (IMU) data and the motion prediction model to perform temporal smoothing and error suppression on the positioning results.

[0157] Specifically, the fusion of inertial measurement unit (IMU) data in S8 involves establishing a loosely coupled fusion framework based on an error state Kalman filter. Using a loosely coupled fusion strategy, IMU data is processed separately from the positioning results of the Gaussian model. In this loosely coupled framework, IMU data acts as an independent sensor module, and state estimation is performed using an error state Kalman filter (ESKF). This framework uses IMU data to calculate position, velocity, and attitude information during the prediction phase, and corrects the IMU data using other sensors (such as GNSS or vision) during the update phase, thereby improving positioning accuracy. The error state Kalman filter's prediction process is based on the IMU's kinematic model and includes recursive calculations of position, velocity, and attitude. The update process uses measurement data from other sensors to correct the IMU's state, adjusting the uncertainty of the state estimate by calculating the Kalman gain.

[0158] When the Gaussian model positioning confidence falls below a threshold, the system switches to IMU dead reckoning. When the confidence level of the Gaussian model positioning result is low, the system switches to IMU dead reckoning mode. IMU dead reckoning calculates the current position and attitude by integrating the IMU's acceleration and angular velocity data. This mode is particularly critical when GNSS signals are lost or blocked, providing continuous positioning information. To improve the robustness of IMU dead reckoning, pre-integration techniques are often used to segment the IMU data and reduce the accumulation of calculation errors. Pre-integrated measurements serve as motion constraints and can be used to calculate state quantities between two moments, thereby improving positioning continuity and stability.

[0159] A kinematic model is used to predict periodic positioning deviations caused by lower limb swing. During movement, lower limb swing can cause periodic positioning deviations. To mitigate these deviations, the system uses a kinematic model. Based on the principles of human kinematics, the kinematic model simulates the impact of lower limb swing on positioning results and compensates for these deviations using a motion prediction model.

[0160] Motion prediction models can be built using time series analysis or deep learning methods. By learning from historical motion data, they predict future motion trajectories and adjust positioning results accordingly. For example, time series models such as LSTM (Long Short-Term Memory) can be used to model the periodic characteristics of lower limb swing and compensate for this during positioning.

[0161] Example 2

[0162] like Figure 4As shown, the present application provides a personnel positioning system based on a 3D Gaussian splash model and video fusion, which is applied to the personnel positioning method based on a 3D Gaussian splash model and video fusion as described in Example 1, including: a multi-perspective video stream acquisition module 11, a sparse point cloud initialization module 12, a time series dynamic tracking module 13, a time series consistency optimization module 14, a target detection module 15, a three-dimensional-two-dimensional space matching module 16, a multi-perspective fusion positioning module 17, and a multi-sensor fusion optimization module 18.

[0163] It can be understood that, in this embodiment, the multi-view video stream acquisition module 11 is used to acquire video streams of at least two viewpoints as input data;

[0164] It can be understood that in this embodiment, the sparse point cloud initialization module 12 is used to generate an initial Gaussian distribution for the first frame of video through sparse initialization based on the SFM point cloud, construct the pose and density parameters of the three-dimensional Gaussian ellipsoid, and each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color and opacity parameters to construct an initial three-dimensional Gaussian model;

[0165] It can be understood that, in this embodiment, the temporal dynamic tracking module 13 is used to track the displacement changes of the three-dimensional Gaussian ellipsoid through the optical flow algorithm for subsequent frames, and dynamically update its position, covariance matrix and color feature parameters;

[0166] It can be understood that, in this embodiment, the time series consistency optimization module 14 is used to maintain the parameter consistency of the Gaussian model in the time dimension by using an L2 regularized time series optimization algorithm that minimizes parameter drift;

[0167] It can be understood that, in this embodiment, the target detection module 15 is used to extract the bounding box of the construction worker in the video frame using the Mask R-CNN target detection model;

[0168] It can be understood that, in this embodiment, the 3D-2D spatial matching module 16 is used to spatially match the detected personnel area with the 3D Gaussian ellipsoid area with the highest density in the Gaussian model, and establish a spatial correspondence between pixel coordinates and 3D coordinates;

[0169] It can be understood that, in this embodiment, the multi-view fusion positioning module 17 is used to calculate the three-dimensional coordinates of the person through perspective geometry back-projection, and if multi-view input is used, triangulation is performed to improve positioning accuracy;

[0170] It can be understood that, in this embodiment, the multi-sensor fusion optimization module 18 is used to fuse the inertial measurement unit IMU data and the motion prediction model to perform time series smoothing and error suppression on the positioning results.

[0171] Figure 5This is an electronic device provided by an embodiment of the present application. Figure 5 As shown, the electronic device includes at least the following parts: a processor 101 and a memory 100 , a communication interface 103 , and a bus 102 .

[0172] In an embodiment of the present application, the memory 100 is used to store instructions executable by the processor 101, and the processor 101 is configured to implement the method of the first aspect when executing the instructions.

[0173] In an embodiment of the present application, a computer-readable storage medium includes instructions, and the instructions instruct a device to execute the method of the first aspect. For example, the instructions instruct the device to execute Figure 1 The method is shown in the process steps.

[0174] The program running in the electronic device involved in one embodiment of the present application may be a program that controls a central processing unit (CPU) and the like to implement the functions of the above-mentioned embodiment involved in one embodiment of the present invention (a program that causes a computer to function). The information processed by these devices is temporarily stored in random access memory (RAM) while being processed, and is then stored in various ROMs such as read-only memory (Flash ROM) and a hard disk drive (HDD), where it is read, modified, and written as needed by the CPU.

[0175] It should be noted that a portion of the electronic device of the above embodiment may also be implemented by a computer. In this case, a program for implementing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer and executed.

[0176] It should be noted that the "computer" mentioned here refers to a computer built into an electronic device, employing hardware including an operating system (OS) and peripheral devices. Furthermore, "computer-readable recording medium" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computers.

[0177] Furthermore, "computer-readable recording media" may include: media that dynamically store programs for a short period of time, such as communication lines when transmitting programs via networks such as the Internet or communication lines such as telephone lines; and media that store programs for a fixed period of time, such as volatile memory within computers acting as servers or clients in this context. Furthermore, the aforementioned program may be a program for implementing a portion of the aforementioned functions, or a program that can achieve the aforementioned functions by combining with a program already stored in a computer.

[0178] Furthermore, the electronic device in the above-described embodiments can also be implemented as a collection of multiple devices (a device group). Each device comprising the device group may include some or all of the functions or functional blocks of the electronic device in the above-described embodiments. A device group only needs to include all of the functions or functional blocks of the electronic device.

[0179] Those skilled in the art should recognize that the above embodiments are merely intended to illustrate the present application and are not intended to limit the present application. As long as they are within the spirit of the present application, appropriate changes and modifications to the above embodiments are within the scope of protection claimed in the present application.

Claims

1. A personnel positioning method based on 3D Gaussian splash model and video fusion, characterized in that: The method comprises: S1, collects video streams from at least two perspectives as input data; S2, generates an initial Gaussian distribution for the first frame of video through sparse initialization based on the SFM point cloud, constructs the pose and density parameters of the three-dimensional Gaussian ellipsoid. Each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color and opacity parameters, and constructs the initial three-dimensional Gaussian model; S3, for subsequent frames, tracks the displacement changes of the three-dimensional Gaussian ellipsoid through the optical flow algorithm, and dynamically updates its position, covariance matrix and color feature parameters; S4, maintains the parameter consistency of the Gaussian model in the time dimension through the L2 regularized temporal optimization algorithm that minimizes parameter drift; S5, using the Mask R-CNN object detection model to extract the bounding box of the construction workers in the video frame; S6, spatially matching the detected personnel area with the three-dimensional Gaussian ellipsoid area with the highest density in the Gaussian model to establish a spatial correspondence between pixel coordinates and three-dimensional coordinates; S7, calculates the three-dimensional coordinates of the person through perspective geometry back-projection, and performs triangulation to improve positioning accuracy if multi-view input is used; S8, integrates the inertial measurement unit (IMU) data and the motion prediction model to perform temporal smoothing and error suppression on the positioning results.

2. The personnel positioning method based on 3D Gaussian splash model and video fusion according to claim 1, characterized in that: The sparse initialization in S2 specifically includes: Use COLMAP to perform motion structure recovery (SFM) reconstruction on the first frame of multi-view video and extract sparse point cloud; Initialize the Gaussian ellipsoid with the point cloud center as the mean and the point cloud distribution as the covariance; Spherical harmonics are used to encode the viewing angle-dependent color features, and the initial opacity is set to a value in the range of 0.8-1.

0.

3. The personnel positioning method based on 3D Gaussian splash model and video fusion according to claim 2, characterized in that: Generating an initial Gaussian distribution includes: Combining a three-dimensional Gaussian distribution with a temporal component, temporal opacity is encoded via temporal radial basis functions to model the emergence or disappearance of scene content; The position and rotation parameters of the three-dimensional Gaussian distribution are represented by a time-conditioned parameter function to model the motion and deformation in the scene. For any point in space and time , whose opacity is calculated as a temporal opacity function The product of the spatial Gaussian distribution, where the spatial location and covariance of the spatial Gaussian distribution are 、 is the time-dependent function, where the opacity of the Gaussian distribution is: ， in, represents the temporal opacity function, 、 are the time-dependent spatial locations and covariances, Representative Gaussian distribution; Temporal Opacity Function A learnable neural network or a preset basis function group is used to control the visible and invisible state changes of the Gaussian distribution on the time axis, which can be expressed as: ， in, is the time center, indicating the most prominent timestamp of the Gaussian distribution, Represents the time scaling factor, which is used to determine its effective duration. Represents time-independent spatial opacity; For each Gaussian distribution, a time condition function is used to model the changes in the position and rotation parameters of the three-dimensional Gaussian ellipsoid in the time dimension, which is expressed as: ， in, Represents a Gaussian distribution over time spatial location, , is the corresponding coefficient, ; Compute the pixel's color based on the resulting spatial opacity , expressed as: ， in, It is Gaussian colors.

4. The personnel positioning method based on 3D Gaussian splash model and video fusion according to claim 2, characterized in that: The optical flow algorithm tracking in S3 adopts a hybrid tracking strategy combining Farneback dense optical flow and LK sparse optical flow, including: The Farneback algorithm is used to estimate global dense optical flow. The optical flow information of each pixel in the entire image is calculated based on the information of the previous and next two frames. The image frame is approximated by polynomial expansion, and the least squares method is used to estimate the displacement of the pixel points. LK algorithm is used to track local details of sparse optical flow, select feature points for local matching, and only track the movement of some key points in the image; During the tracking process, if the displacement of the Gaussian ellipsoid is detected to exceed the preset threshold, the system will start the reinitialization process, re-detect the target feature points, and update the covariance matrix and color feature parameters; In each frame of the video sequence, a color histogram similarity measure between a target region of a current frame and a target region of at least one adjacent frame is calculated; When the similarity measure is lower than a preset threshold, it is determined that a significant change or occlusion event occurs in the target; In response to the determination, a reinitialization operation of the tracker is triggered.

5. The personnel positioning method based on 3D Gaussian splash model and video fusion according to claim 1, characterized in that: The timing optimization of S4 includes: Establish Markov constraints on the change of Gaussian parameters between adjacent frames: Applying positional parameters The second-order difference constraint is used to prevent the parameters from changing drastically in a short period of time. The constraint is expressed as: ， in, is the constraint threshold, which is used to control the amplitude of parameter changes. Respectively indicate that at adjacent times Positional parameters; Exponential decay weighting is used to adjust the influence weight of historical frame parameters, giving small weights to the parameters of distant frames and large weights to the parameters of close frames. Defined as: ， in is the decay factor, which is used to control the decay speed of the weight, and is used to pay more attention to recent data in the time series while avoiding forgetting historical information too early; In order to minimize parameter drift, the L2 regularization term is introduced to penalize the change of parameters. The optimization objective function is expressed as: ， in, is the weight of the first-order difference constraint, is the L2 regularization weight, is the average parameter value of the current frame.

6. The personnel positioning method based on 3D Gaussian splash model and video fusion according to claim 1, characterized in that: The spatial matching of S6 includes: Calculate the density-weighted average of all Gaussian ellipsoids within the person's bounding box; Screen the Gaussian clusters with opacity > 0.9 and density ranking in the top 5% as matching targets; Abnormal Gaussian distribution is eliminated by RANSAC algorithm.

7. The personnel positioning method based on 3D Gaussian splash model and video fusion according to claim 1, characterized in that: When performing multi-view triangulation in the S7: Select a viewing angle combination with a baseline distance greater than 2 meters. A large baseline distance can significantly increase the angle of the back-projected ray, thereby enhancing the geometric constraints of triangulation and reducing calculation errors. A depth consistency check is performed on matching points with a back-projection ray angle less than 15°. By comparing the depth relationship between the back-projection ray and the target point under multiple viewing angles, it is determined whether it conforms to physical laws. If not, the matching point is removed to avoid misleading the estimation of the final 3D coordinates. The LM algorithm is used to perform nonlinear optimization of the three-dimensional coordinates. By minimizing the reprojection error, the matching points under multiple perspectives are jointly optimized to make the final three-dimensional coordinates close to the true value.

8. The personnel positioning method based on 3D Gaussian splash model and video fusion according to claim 1, characterized in that: The fused inertial measurement unit (IMU) data in the S8 is specifically: Establish a loosely coupled fusion framework based on error state Kalman filtering; When the Gaussian model positioning confidence is lower than the threshold, it switches to IMU dead reckoning; Predicting the periodic positioning deviation caused by the swing of human lower limbs through kinematic model.

9. A personnel positioning system based on a 3D Gaussian splash model and video fusion, applied to the personnel positioning method based on a 3D Gaussian splash model and video fusion according to any one of claims 1 to 8, characterized in that: include: A multi-view video stream acquisition module is used to acquire video streams from at least two perspectives as input data; The sparse point cloud initialization module is used to generate an initial Gaussian distribution for the first frame of video through sparse initialization based on the SFM point cloud, and construct the pose and density parameters of a three-dimensional Gaussian ellipsoid. Each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color, and opacity parameters to construct an initial three-dimensional Gaussian model. The temporal dynamic tracking module is used to track the displacement changes of the three-dimensional Gaussian ellipsoid through the optical flow algorithm in subsequent frames, and dynamically update its position, covariance matrix and color feature parameters; The temporal consistency optimization module is used to maintain the parameter consistency of the Gaussian model in the time dimension by minimizing the parameter drift through the L2 regularized temporal optimization algorithm; The object detection module is used to extract the bounding boxes of construction workers in video frames using the Mask R-CNN object detection model; The 3D-2D spatial matching module is used to spatially match the detected personnel area with the 3D Gaussian ellipsoid area with the highest density in the Gaussian model, and establish a spatial correspondence between pixel coordinates and 3D coordinates; A multi-view fusion positioning module is used to calculate the three-dimensional coordinates of the person through perspective geometry back-projection. If multi-view input is used, triangulation is performed to improve positioning accuracy; The multi-sensor fusion optimization module is used to fuse the inertial measurement unit (IMU) data and the motion prediction model to perform temporal smoothing and error suppression on the positioning results.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and the program instructs a device to execute the personnel positioning method based on 3D Gaussian splash model and video fusion according to any one of claims 1 to 8.

Images