Personnel positioning method and system based on 3D Gaussian splash model and video fusion

Abstract

The invention discloses a personnel positioning method and system based on a 3D Gaussian splash model and video fusion. The method comprises the following steps: firstly, acquiring input data of at least two visual angles through a multi-visual angle video stream acquisition module, and constructing an initial three-dimensional Gaussian model by utilizing a sparse point cloud initialization module; a time sequence dynamic tracking module is combined with an optical flow algorithm to realize cross-frame parameter updating of a Gaussian ellipsoid, and a time sequence consistency optimization module is adopted to suppress parameter drift; identifying a constructor bounding box by using a target detection module, and establishing a corresponding relation between pixels and three-dimensional coordinates through a three-dimensional-two-dimensional space matching module; three-dimensional coordinates are calculated through a multi-view fusion positioning module, and the positioning precision is improved through a multi-sensor fusion optimization module in combination with IMU data. A 3D Gaussian splash model is combined with multi-view geometry and time sequence optimization, high-precision personnel dynamic positioning without marking in a construction scene is realized, and the problems of tracking drift and shielding in a complex environment in a traditional method are effectively solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and construction safety monitoring, and particularly relates to a personnel positioning method and system based on a 3D Gaussian splash model and video fusion. Background Art

[0002] With the rapid development of the construction industry, safety management and personnel positioning technologies at construction sites have received increasing attention. Traditional construction personnel positioning methods mainly rely on active sensor technologies such as Global Positioning System (GPS), Radio Frequency Identification (RFID), or Ultra-Wideband (UWB). However, these methods have problems such as signal occlusion, strong device dependence, and high deployment costs in complex indoor or semi-open construction environments. In addition, markerless positioning methods based on vision have gradually become a research hotspot due to advantages such as no need to wear additional devices and flexible deployment. However, existing vision positioning technologies still face challenges such as poor adaptability to dynamic scenes, multi-target occlusion, and high computational complexity.

[0003] Traditional construction personnel positioning technologies are mainly divided into two categories: sensor-based and vision-based. Sensor-based methods (such as GPS, UWB, RFID, etc.) perform well in open environments, but are vulnerable to signal attenuation and multipath effects in indoor or dense structure environments, resulting in a decrease in positioning accuracy. In addition, these methods require construction personnel to wear special devices, increasing management costs and personnel burdens. Markerless vision-based positioning methods (such as object detection and tracking based on deep learning) avoid the problem of device dependence, but in complex dynamic scenes, due to factors such as lighting changes, occlusion, and perspective changes, problems such as target loss or mis-matching are likely to occur.

[0004] In recent years, significant progress has been made in vision positioning technologies based on three-dimensional reconstruction. Structure from Motion (SFM) and Simultaneous Localization and Mapping (SLAM) technologies can recover the three-dimensional structure of a scene from multi-view images, providing a spatial reference for personnel positioning. However, traditional SFM / SLAM methods rely on sparse feature point matching and are difficult to stably track moving targets in dynamic scenes. In addition, three-dimensional representation methods based on voxels or grids (such as NeRF) have high computational complexity and are difficult to meet real-time requirements.

[0005] 3D Gaussian Splatting is an emerging method for three-dimensional scene representation. It efficiently models the scene through parameterized Gaussian ellipsoids, combining high rendering quality and real-time computing capabilities. Compared with traditional point cloud or voxel representations, the 3D Gaussian Splatting model can more flexibly adapt to dynamic scene changes and supports efficient optical flow tracking and parameter optimization. However, existing methods still have deficiencies in maintaining temporal consistency, dynamic target tracking, and multi-sensor fusion. Especially in construction scenes, where personnel movement is complex and occlusions are frequent, it leads to cumulative positioning errors and drift problems.

[0006] Multi-view video fusion is an effective means to improve positioning accuracy, but existing methods lack systematic design in view selection, feature matching, and triangulation optimization. For example, insufficient geometric constraints of back-projected rays may lead to depth estimation errors, and simple linear interpolation is difficult to handle motion blur or occlusion situations. In addition, the fusion strategy of auxiliary sensor data such as inertial measurement units (IMUs) is not yet perfect, and the motion prior information is not fully utilized to suppress positioning jitter.

[0007] Therefore, there is an urgent need to develop a new method for video dynamic fusion and construction personnel positioning to solve problems such as target loss or mis-matching caused by factors such as equipment dependence, light changes, occlusions, and view changes. Summary of the Invention

[0008] In view of this, it is necessary to provide a personnel positioning method and system based on the 3D Gaussian Splatting model and video fusion. Through technologies such as sparse initialization, optical flow tracking, temporal optimization, and multi-view triangulation, high-precision and robust dynamic personnel positioning can be achieved. This method not only avoids the use of active sensors but also significantly improves the positioning stability and accuracy in complex environments through dynamic Gaussian model updates and IMU data fusion.

[0009] In a first aspect, an embodiment of the present application provides a personnel positioning method based on the 3D Gaussian Splatting model and video fusion, the method comprising: S1, collecting video streams from at least two viewpoints as input data; S2, generating an initial Gaussian distribution for the first frame of video through sparse initialization based on the SFM point cloud, constructing the pose and density parameters of the three-dimensional Gaussian ellipsoid, where each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color, and opacity parameters, and constructing an initial three-dimensional Gaussian model; S3, tracking the displacement changes of the three-dimensional Gaussian ellipsoid for subsequent frames through the optical flow algorithm, and dynamically updating its position, covariance matrix, and color feature parameters; S4, maintaining the parameter consistency of the Gaussian model in the time dimension through the L2 regularization temporal optimization algorithm that minimizes parameter drift; S5. Use the Mask R-CNN object detection model to extract the bounding boxes of construction workers in the video frames; S6. Perform spatial matching between the detected person area and the three-dimensional Gaussian ellipsoid area with the highest density in the Gaussian model, and establish the spatial correspondence relationship between pixel coordinates and three-dimensional coordinates; S7. Calculate the three-dimensional coordinates of the person through perspective geometric back-projection. If multi-view input is adopted, triangulation is performed to improve the positioning accuracy; S8. Integrate the inertial measurement unit (IMU) data and the motion prediction model to perform temporal smoothing and error suppression on the positioning results.

[0010] Optionally, in an implementation manner of the first aspect of the present invention, the sparse initialization in S2 specifically includes: Perform Structure from Motion (SFM) reconstruction on the first frame of the multi-view video through COLMAP to extract the sparse point cloud; Initialize the Gaussian ellipsoid with the center of the point cloud as the mean and the point cloud distribution as the covariance; Adopt spherical harmonic functions to encode the view-related color features, and set the initial opacity to a value in the range of 0.8 - 1.0.

[0011] Optionally, in an implementation manner of the first aspect of the present invention, the generation of the initial Gaussian distribution includes: Combine the three-dimensional Gaussian distribution with the time component, and encode the time opacity through the time radial basis function to model the emergence or disappearance of the scene content; Adopt the time-conditioned parameter function to represent the position and rotation parameters of the three-dimensional Gaussian distribution to model the motion and deformation in the scene.

[0012] For any spatio-temporal point , its opacity is calculated as the product of the time opacity function and the spatial Gaussian distribution, where the spatial position and covariance , are time-related functions. Among them, the opacity of the Gaussian distribution is: , where, represents the time opacity, , are the time-related spatial position and covariance, represents the th Gaussian distribution.

[0013] The time radial basis function is implemented by a learnable neural network or a preset basis function group, and is used to control the appearance and disappearance state change of the Gaussian distribution on the time axis, and is expressed as: , Among them, is the time center, representing the most prominent timestamp of the Gaussian distribution, represents the time scaling factor, which is used to determine its effective duration, represents the time-independent spatial opacity.

[0014] For each Gaussian distribution, a time conditional function is used to model the changes in the position and rotation parameters of the three-dimensional Gaussian ellipsoid in the time dimension, expressed as: , Among them, represents the spatial position of the Gaussian distribution at time , , is the corresponding coefficient, ; Calculate the color of the pixel based on the final spatial opacity , expressed as: , Among them, is the color of the th Gaussian.

[0015] Optionally, in an implementation manner of the first aspect of the present invention, the optical flow algorithm tracking in S3 adopts a hybrid tracking strategy combining Farneback dense optical flow and LK sparse optical flow, including: Use the Farneback algorithm to perform global dense optical flow estimation, calculate the optical flow information of each pixel point of the entire image through the information of the previous and subsequent two frames of images, approximate the image frames through polynomial expansion, and use the least squares method to estimate the displacement of the pixel points.

[0016] Use the LK algorithm to perform sparse optical flow local detail tracking, select feature points for local matching, and only track the movement of some key points in the image.

[0017] During the tracking process, if it is detected that the displacement of the Gaussian ellipsoid exceeds the preset threshold, the system will start the re-initialization process, re-detect the target feature points, and update the covariance matrix and color feature parameters.

[0018] In each frame of the video sequence, calculate the color histogram similarity measure between the target area of the current frame and the target areas of at least one adjacent frame.

[0019] When the similarity measure is lower than the preset threshold, it is determined that a significant change or occlusion event has occurred to the target; In response to the determination, trigger the re-initialization operation of the tracker.

[0020] Optionally, in an implementation of the first aspect of the present invention, the timing optimization of S4 includes: Establish a Markov constraint on the change of Gaussian parameters between adjacent frames: Apply a second-order difference constraint to the position parameters to prevent the parameters from changing drastically in a short time. The constraint is expressed as: , where is the constraint threshold for controlling the amplitude of parameter change, respectively represent the position parameters at adjacent times .

[0021] Adopt exponential decay weighted adjustment to the influence weights of historical frame parameters, assign small weights to the parameters of distant frames, and assign large weights to the parameters of close frames. The weight is defined as: , where is the decay factor for controlling the decay speed of the weight, which is used to pay more attention to recent data in the time series while avoiding premature forgetting of historical information.

[0022] To minimize parameter drift, an L2 regularization term is introduced to penalize the change of parameters. The optimization objective function is expressed as: , where is the weight of the first-order difference constraint, is the L2 regularization weight, is the average parameter value of the current frame.

[0023] Optionally, in an implementation of the first aspect of the present invention, the spatial matching of S6 includes: Calculate the density-weighted average value of all Gaussian ellipsoids within the person's bounding box.

[0024] Screen Gaussian clusters with opacity > 0.9 and density ranking in the top 5% as matching targets.

[0025] Remove abnormal Gaussian distributions through the RANSAC algorithm.

[0026] Optionally, in an implementation of the first aspect of the present invention, during multi-view triangulation in S7.

[0027] Select view combinations with a baseline distance greater than 2 meters. A large baseline distance can significantly increase the angle between back-projected rays, thereby enhancing the geometric constraint of triangulation and reducing calculation errors.

[0028] Perform a depth consistency check on the matching points where the angle between the back-projected rays is less than 15°. By comparing the depth relationship between the back-projected rays and the target points from multiple perspectives, determine whether it conforms to physical laws. If not, eliminate the matching points to avoid misleading the estimation of the final three-dimensional coordinates.

[0029] Use the LM algorithm to perform non-linear optimization on the three-dimensional coordinates. By minimizing the reprojection error, jointly optimize the matching points from multiple perspectives to make the final three-dimensional coordinates close to the true values.

[0030] Optionally, in an implementation manner of the first aspect of the present invention, the fusion of IMU data in S8 is specifically as follows: Establish a loose-coupling fusion framework based on error-state Kalman filtering; Switch to IMU dead reckoning when the Gaussian model positioning confidence is lower than the threshold; Predict the periodic positioning deviation caused by the swinging of the lower limbs of the person through a kinematic model.

[0031] In a second aspect, an embodiment of the present application provides a personnel positioning system based on 3D Gaussian splash model and video fusion, which is applied to the personnel positioning method based on 3D Gaussian splash model and video fusion as described in the first aspect, including: A multi-view video stream acquisition module, which is used to acquire video streams from at least two perspectives as input data.

[0032] A sparse point cloud initialization module, which is used to generate an initial Gaussian distribution through sparse initialization based on SFM point cloud for the first frame of video, construct the pose and density parameters of a three-dimensional Gaussian ellipsoid, and each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color, and opacity parameters to construct an initial three-dimensional Gaussian model.

[0033] A temporal dynamic tracking module, which is used to track the displacement change of the three-dimensional Gaussian ellipsoid through an optical flow algorithm for subsequent frames, and dynamically update its position, covariance matrix, and color feature parameters.

[0034] A temporal consistency optimization module, which is used to maintain the parameter consistency of the Gaussian model in the time dimension through an L2-regularized temporal optimization algorithm that minimizes parameter drift.

[0035] A target detection module, which is used to extract the bounding box of construction workers in the video frame using the Mask R-CNN target detection model.

[0036] A three-dimensional to two-dimensional space matching module, which is used to perform spatial matching between the detected person area and the area of the three-dimensional Gaussian ellipsoid with the highest density in the Gaussian model, and establish the spatial correspondence relationship between pixel coordinates and three-dimensional coordinates.

[0037] A multi - perspective fusion positioning module, which is used to calculate the three - dimensional coordinates of personnel through perspective geometric back - projection. If multi - perspective input is adopted, triangulation is performed to improve the positioning accuracy.

[0038] A multi - sensor fusion optimization module, which is used to fuse the data of the inertial measurement unit (IMU) and the motion prediction model to perform temporal smoothing and error suppression on the positioning result.

[0039] In a third aspect, an embodiment of the present application provides an electronic device, including: A processor; A memory for storing instructions executable by the processor; Wherein, when the processor is configured to execute the instructions, it implements the personnel positioning method based on the 3D Gaussian splash model and video fusion as described in the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a computer - readable storage medium. The computer - readable storage medium stores a program, and the program instructs the device to execute the personnel positioning method based on the 3D Gaussian splash model and video fusion as described in the first aspect.

[0041] The present invention discloses a personnel positioning method and system based on a 3D Gaussian splash model and video fusion. The method first obtains input data from at least two perspectives through a multi - perspective video stream acquisition module, and constructs an initial three - dimensional Gaussian model using a sparse point cloud initialization module; realizes cross - frame parameter update of the Gaussian ellipsoid through a temporal dynamic tracking module in combination with an optical flow algorithm, and suppresses parameter drift using a temporal consistency optimization module; uses an object detection module to identify the bounding box of construction personnel, and establishes the correspondence between pixels and three - dimensional coordinates through a three - dimensional to two - dimensional space matching module; finally calculates the three - dimensional coordinates through a multi - perspective fusion positioning module, and improves the positioning accuracy by combining IMU data through a multi - sensor fusion optimization module. The present invention innovatively combines the 3D Gaussian splash model with multi - perspective geometry and temporal optimization, realizes high - precision personnel dynamic positioning without markers in a construction scene, and effectively solves the problems of tracking drift and occlusion of traditional methods in complex environments.

[0042] Beneficial effects:

[0043] (1) High - precision dynamic positioning: By fusing multi - perspective video streams and the 3D Gaussian splash model, combining optical flow tracking and temporal optimization algorithms, the dynamic positioning accuracy of construction personnel at the sub - meter level is achieved, which is more than 40% higher than that of traditional vision - based positioning methods.

[0044] (2) Strong anti - interference ability: Adopting a spatio - temporal joint optimization mechanism for updating Gaussian model parameters, effectively resists interference such as light changes and partial occlusions, and the tracking success rate in complex construction scenes is increased to 92%.

[0045] (3) Superior real-time performance: Through the hierarchical optical flow tracking strategy (combining dense and sparse optical flow) and L2-regularized temporal optimization, the system processing speed reaches 25fps, meeting the real-time monitoring requirements at the construction site.

[0046] (4) Advantages of multi-source data fusion: The innovative loose-coupling IMU fusion framework can automatically switch to inertial navigation when visual positioning fails, and the positioning continuity is increased by 3 times compared with traditional pure vision methods.

[0047] (5) Low deployment cost: There is no need to pre-install marking devices or dedicated sensors. Only 2-4 ordinary monitoring cameras are required to achieve full-area coverage, and the system deployment cost is reduced by more than 60%. The degree matching mechanism automatically adapts to the increase and decrease of personnel and the movement of equipment in the construction scenario, and the model update delay is less than 0.5 seconds. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic flowchart of a personnel positioning method based on a 3D Gaussian splash model and video fusion provided by an embodiment of the present application.

[0049] Figure 2 It is a hierarchical system architecture diagram provided by an embodiment of the present application.

[0050] Figure 3 It is a data flow diagram provided by an embodiment of the present application.

[0051] Figure 4 It is a schematic diagram of a personnel positioning system module based on a 3D Gaussian splash model and video fusion provided by an embodiment of the present application.

[0052] Figure 5 It is a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments.

[0054] It should be noted that "at least one" in the embodiments of the present application refers to one or more, and multiple refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application.

[0055] It should be noted that, in the embodiments of the present application, words such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order. Features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0056] Based on the implementations in this application, all other implementations obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] Embodiment 1

[0058] Figure 1 A flow chart of a personnel positioning method based on a 3D Gaussian splash model and video fusion provided in an embodiment of the present application. The entire process combines SFM, optical flow, Mask R-CNN, Gaussian splash and other technologies to extract the three-dimensional coordinates of construction workers from multi-view video streams, and smooth and suppress errors through timing optimization and IMU data. Figure 2 A layered system architecture diagram provided for an embodiment of the present application. The system includes an input layer, a core processing layer, and an output layer. Figure 3 A data flow diagram provided for an embodiment of the present application.

[0059] like Figures 1-3 As shown, a personnel positioning method based on a 3D Gaussian splash model and video fusion includes: S1, collect video streams from at least two perspectives as input data.

[0060] In 3D Gaussian Splatting technology, multi-view input is crucial, and rotating shooting through drones can provide a better perspective. In vision-based markerless personnel positioning, single-view video is limited by the inherent defects of two-dimensional imaging (such as occlusion, perspective distortion, and lack of depth information), making it difficult to stably and accurately estimate the three-dimensional position of the target. Multi-view video streams can provide richer spatial information, enhance the robustness of three-dimensional reconstruction through disparity calculation and triangulation, and thus improve positioning accuracy. Therefore, multi-view input is of great significance for 3D reconstruction and posture estimation.

[0061] S2. Generate an initial Gaussian distribution for the first-frame video through sparse initialization based on the SFM point cloud, and construct the pose and density parameters of a three-dimensional Gaussian ellipsoid. Each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color, and opacity parameters, and an initial three-dimensional Gaussian model is constructed.

[0062] The SFM (Structure from Motion) technique is used to estimate the sparse point cloud and camera pose from multi-view images. Each 3D Gaussian is defined by its mean, covariance, opacity, and view-dependent color represented by SH. These parameters together constitute the pose and density parameters of the three-dimensional Gaussian ellipsoid.

[0063] The core objective of this step is to reconstruct the sparse 3D scene structure from the multi-view first-frame video and initialize the 3D Gaussian Splatting model based on this structure to provide initial parameters for subsequent dynamic tracking and optimization.

[0064] Specifically, the sparse initialization in S2 specifically includes: performing Structure from Motion (SFM) reconstruction on the first frame of the multi-view video through COLMAP to extract the sparse point cloud. Using the COLMAP tool, perform SFM reconstruction based on the first-frame image of the multi-view video to recover the sparse point cloud of the scene. COLMAP gradually constructs the sparse point cloud and camera pose of the scene through steps such as feature extraction, matching, geometric verification, and optimization. This process usually includes feature extraction (such as SIFT), feature matching, geometric constraints (such as RANSAC optimization), and sparse reconstruction (through SBA optimization).

[0065] Initialize the Gaussian ellipsoid with the center of the point cloud as the mean and the point cloud distribution as the covariance. Based on the sparse point cloud, each point is initialized as a 3D Gaussian function. The center position of this Gaussian function is determined by the point in the point cloud, and the covariance matrix describes the distribution of the point cloud. Specifically, each Gaussian ellipsoid is defined by its spatial position (mean) and covariance matrix, and these parameters together determine the shape and density of the Gaussian in three-dimensional space.

[0066] The color features related to the viewing angle are encoded using spherical harmonic functions, and the initial opacity is set to a value in the range of 0.8 - 1.0. To represent the color features related to the viewing angle, spherical harmonics (SH) are typically used to encode colors. Spherical harmonics can effectively represent color changes under illumination and viewing angle variations, thus maintaining color consistency during the rendering process. The color parameters of each Gaussian ellipsoid are represented by SH coefficients, and these coefficients can be trained and optimized using the color information in the point cloud. The initial opacity is usually set between 0.8 and 1.0 to ensure good visibility of the Gaussian model during the rendering process while avoiding loss of details due to excessive transparency. This parameter can be adjusted through experiments to achieve the best visual effects.

[0067] The generation of the initial Gaussian distribution includes: combining the three-dimensional Gaussian distribution with a time component, and encoding the temporal opacity through a time radial basis function to model the emergence or disappearance of scene content. By combining the three-dimensional Gaussian distribution with a time component, and expanding the 3D Gaussian into a 4D spatio-temporal domain, a spatio-temporal based Gaussian distribution is constructed to model dynamic scenes. The expressive power will be enhanced through temporal opacity and parameterized motion / rotation, thereby capturing static, dynamic, and transient content in the scene.

[0068] A time-conditioned parametric function is used to represent the position and rotation parameters of the three-dimensional Gaussian distribution to model the motion and deformation in the scene.

[0069] For any spatio-temporal point , its opacity is calculated as the product of the temporal opacity function and the spatial Gaussian distribution, where the spatial position and covariance , are time-related functions. Among them, the opacity of the Gaussian distribution is: , where, represents the temporal opacity, , are the time-related spatial position and covariance, represents the -th Gaussian distribution.

[0070] The time radial basis function is implemented using a learnable neural network or a preset group of basis functions, and is used to control the appearance and disappearance state changes of the Gaussian distribution on the time axis, expressed as: , where, is the time center, representing the time stamp when the Gaussian distribution is most prominent, represents the time scaling factor, used to determine its effective duration, Represents the time - independent spatial opacity. Temporal radial basis functions are used to represent the temporal opacity of a Gaussian distribution at any point in time. This function can effectively simulate the scene content that appears or disappears during the video duration.

[0071] To model the motion and deformation in the scene, the position and rotation parameters of each Gaussian distribution are parameterized by a time - conditional function. For each Gaussian distribution, a time - conditional function is used to model the changes in the position and rotation parameters of a three - dimensional Gaussian ellipsoid over the time dimension, expressed as: , where, represents the spatial position of the Gaussian distribution at time , , is the corresponding coefficient, ; Calculate the color of the pixel based on the final spatial opacity , expressed as: , where, is the color of the th Gaussian. Through the above steps, the generated initial Gaussian distribution can effectively model the spatio - temporal changes in a dynamic scene, thus achieving high - quality real - time rendering and a compact model representation.

[0072] S3. For subsequent frames, track the displacement changes of the three - dimensional Gaussian ellipsoid through an optical flow algorithm, and dynamically update its position, covariance matrix, and color feature parameters.

[0073] Specifically, the optical flow algorithm tracking in S3 adopts a hybrid tracking strategy that combines the Farneback dense optical flow and the LK sparse optical flow, including: using the Farneback algorithm for global dense optical flow estimation, calculating the optical flow information of each pixel point in the entire image through the information of the front and back two frames of images, approximating the image frames through polynomial expansion, and estimating the displacement of the pixel points using the least - squares method.

[0074] Adopt the Gunnar Farneback algorithm for global dense optical flow estimation, and calculate the optical flow information of each pixel point in the entire image through the information of the front and back two frames of images. This algorithm approximates the image frames through polynomial expansion by constructing an image pyramid structure, and estimates the displacement of the pixel points through the least - squares method. This method can provide a high - precision global motion field and is suitable for capturing large - range displacement changes.

[0075] Use the LK algorithm for local detail tracking of sparse optical flow, select feature points for local matching, and only track the motion of some key points in the image. In the sparse optical flow part, the Lucas-Kanade (LK) algorithm is used for local detail tracking. This algorithm detects corner points in the image and calculates the motion vectors of local feature points based on the assumptions of brightness constancy and small motion. The LK algorithm has high computational efficiency and is suitable for tracking local details, such as the edges of Gaussian ellipsoids or key regions.

[0076] By combining the hybrid strategy of Farneback dense optical flow and LK sparse optical flow, the system can simultaneously obtain global motion information and local detail information. This hybrid strategy has higher robustness in dealing with complex scenarios, especially when the target undergoes significant changes or occlusion, and can more accurately capture the displacement changes of the Gaussian ellipsoid.

[0077] During the tracking process, if the displacement of the Gaussian ellipsoid is detected to exceed the preset threshold, the system will initiate a re-initialization process, re-detect the target feature points, and update the covariance matrix and color feature parameters. During the tracking process, if the displacement of the Gaussian ellipsoid is detected to exceed the preset threshold, the system will initiate a re-initialization process. This process includes re-detecting the target feature points and updating the covariance matrix and color feature parameters. The purpose of re-initialization is to ensure the parameter consistency of the Gaussian model in the time dimension.

[0078] In each frame of the video sequence, calculate the similarity measure of the color histograms of the target region in the current frame and at least one adjacent frame target region. When the similarity measure is lower than the preset threshold, it is determined that a significant change or occlusion event of the target occurs; in response to the determination, trigger the re-initialization operation of the tracker.

[0079] In each frame of the video sequence, calculate the similarity measure of the color histograms of the target region in the current frame and at least one adjacent frame target region. When the similarity measure is lower than the preset threshold, it is determined that a significant change or occlusion event of the target occurs. This mechanism helps to timely detect abnormal behaviors of the target, thereby triggering the re-initialization operation of the tracker. In response to the determination, trigger the re-initialization operation of the tracker. This operation includes re-detecting the target feature points and updating the covariance matrix and color feature parameters to ensure the parameter consistency of the Gaussian model in the time dimension.

[0080] Through the above hybrid optical flow tracking strategy, the system can effectively track the displacement changes of the three-dimensional Gaussian ellipsoid and re-initialize when necessary to maintain the parameter consistency of the Gaussian model.

[0081] S4, maintain the parameter consistency of the Gaussian model in the time dimension through the L2 regularization time series optimization algorithm that minimizes parameter drift.

[0082] Specifically, the timing optimization of S4 includes: establishing a Markov constraint on the change of Gaussian parameters between adjacent frames. By introducing a Markov model, the change of Gaussian parameters between adjacent frames is constrained. For example, a second-order difference constraint is imposed on the position parameters to prevent the parameters from changing drastically in a short period of time.

[0083] A second-order difference constraint is imposed on the position parameters to prevent the parameters from changing drastically in a short period of time. The constraint is expressed as: , where is the constraint threshold, used to control the amplitude of parameter change, respectively represent the position parameters at adjacent times .

[0084] To pay more attention to recent data while avoiding premature forgetting of historical information, an exponential decay weighting strategy is adopted. The exponential decay weighting is used to adjust the influence weight of historical frame parameters, giving small weights to the parameters of distant frames and large weights to the parameters of close frames. The weight is defined as: , where is the decay factor, used to control the decay rate of the weight, to pay more attention to recent data in the time series while avoiding premature forgetting of historical information.

[0085] To minimize parameter drift, an L2 regularization term is introduced to penalize the change of parameters. The optimization objective function is expressed as: , where is the weight of the first-order difference constraint, is the L2 regularization weight, is the average parameter value of the current frame.

[0086] The above objective function is solved by the gradient descent method or other optimization algorithms. During the optimization process, the gradient needs to be calculated and the parameters updated. The L2 regularization term makes the model more inclined to choose small parameter values during training by increasing the diagonal terms of the parameters, thereby reducing overfitting and parameter drift.

[0087] In the 3D Gaussian splashing technique, this timing optimization method is used for the modeling of dynamic scenes. By constraining the change of Gaussian parameters over time, the stability of the model in the time dimension is ensured. By introducing spatio-temporal consistency constraints, the change of Gaussian parameters in dynamic scenes can be effectively modeled. Through the above method, the timing optimization algorithm in S4 can effectively maintain the parameter consistency of the Gaussian model in the time dimension, thereby improving the accuracy and robustness of 3D reconstruction.​​

[0088] S5. Use the Mask R-CNN object detection model to extract the bounding boxes of construction workers in video frames.

[0089] Specifically, in S5, using the Mask R-CNN object detection model to extract the bounding boxes of construction workers in video frames, the specific steps are as follows: 1. Model loading and initialization. Use a pre-trained Mask R-CNN model, which is usually based on backbone networks such as ResNet-50 and ResNet-152. After the model is loaded, it is necessary to load the class label file (such as mscoco_labels.names) and the color mapping file (such as colors.txt) for subsequent visualization of bounding boxes and masks. 2. Video frame processing. Read each frame image from the video stream and perform preprocessing (such as normalization, resizing, etc.) to adapt to the model input requirements. Usually, the size of the input image is 400×400 pixels. 3. Object detection and instance segmentation. Input the preprocessed image into the Mask R-CNN model. The model first generates candidate regions (ROIs) through the Region Proposal Network (RPN), then classifies and regresses the bounding boxes for each candidate region, and simultaneously generates the masks for instance segmentation. The mask output is a four-dimensional tensor representing the pixel-level segmentation result of each detected object. 4. Bounding box and mask extraction. Extract the bounding boxes and masks of each detected object from the model output. The bounding box is usually determined by the confidence score and regression offsets, while the mask achieves pixel-level precise segmentation through the ROIAlign operation. If the confidence of a certain bounding box is lower than the preset threshold, the bounding box is ignored. 5. Result visualization. Draw the extracted bounding boxes and masks on the original image, usually including the following steps: Drawing the bounding box: Use the rectangle function of OpenCV to draw the bounding box and label the class label and confidence. Drawing the mask: Convert the mask to a binary image and use the contour function to draw the contour to enhance the visualization effect. Color mapping: According to the class label, use the color mapping file to assign different colors to each detected object to distinguish different classes of targets. 6. Output and saving. Save the processed image as a new video file or display the detection results in the real-time video stream. For the detection of construction workers, three-dimensional coordinate calculation and spatial matching can also be combined with other modules (such as steps 6 and 7). 7. Performance optimization. To improve the detection speed, a lightweight backbone network (such as MobileNet) can be adopted or techniques such as model pruning and quantization can be used for optimization. In addition, data augmentation (such as rotation, scaling, flipping) can be used to improve the generalization ability of the model.

[0090] Through the above steps, Mask R-CNN can efficiently extract the bounding boxes of construction workers from video frames and generate accurate instance segmentation masks, providing key information for subsequent 3D reconstruction and spatial matching.

[0091] S6. Perform spatial matching between the detected person region and the region of the 3D Gaussian ellipsoid with the highest density in the Gaussian model to establish the spatial correspondence between pixel coordinates and 3D coordinates.

[0092] Specifically, the spatial matching in S6 includes: calculating the density-weighted average of all Gaussian ellipsoids within the person bounding box. Calculate the density-weighted average of all Gaussian ellipsoids within the detected person bounding box. This average reflects the concentration degree of the Gaussian distribution in this region and is used to identify the Gaussian ellipsoid most likely to contain the target.

[0093] Select Gaussian clusters with opacity > 0.9 and density ranking in the top 5% as the matching targets. Select Gaussian clusters with opacity greater than 0.9 and density ranking in the top 5% from all Gaussian ellipsoids as the matching targets. These Gaussian clusters have high visibility and density and are more likely to correspond to the detected person region.

[0094] Eliminate abnormal Gaussian distributions through the RANSAC algorithm. Use the RANSAC algorithm to process the selected Gaussian clusters and eliminate abnormal distributions. The RANSAC algorithm randomly selects matching point pairs, calculates the homography matrix, and verifies the spatial consistency of all matching point pairs, thereby eliminating mis-matched Gaussian distributions. The improved RANSAC algorithm can also improve the calculation accuracy and robustness by increasing the set of matching point pairs.

[0095] Establish the spatial correspondence between pixel coordinates and 3D coordinates. After eliminating abnormal Gaussian distributions, the remaining Gaussian clusters form a spatial correspondence with the detected person region. Through the coordinate information of the matching point pairs, the mapping relationship between pixel coordinates and 3D coordinates can be established, thus realizing the 3D reconstruction of the person region.

[0096] S7. Calculate the 3D coordinates of the person through perspective geometric back-projection. If multi-view input is adopted, perform triangulation to improve the positioning accuracy.

[0097] Specifically, in the case of multi-view triangulation in S7: Select view combinations with a baseline distance greater than 2 meters. A large baseline distance can significantly increase the angle between the back-projection rays, thereby enhancing the geometric constraint of triangulation and reducing calculation errors. In the case of multi-view input, select view combinations with a baseline distance greater than 2 meters. This strategy helps to improve the estimation accuracy of 3D coordinates.

[0098] Perform a depth consistency check on the matching points where the back-projection ray angles are less than 15°. By comparing the depth relationships between the back-projection rays and the target points from multiple perspectives, determine whether they conform to physical laws. If not, eliminate the matching points to avoid misleading the estimation of the final three-dimensional coordinates. For the matching points where the back-projection ray angles are less than 15°, perform a depth consistency check. By comparing the depth relationships between the back-projection rays and the target points from multiple perspectives, determine whether they conform to physical laws. If not, eliminate the matching points to avoid misleading the estimation of the final three-dimensional coordinates.

[0099] Use the LM algorithm to perform non-linear optimization on the three-dimensional coordinates. By minimizing the reprojection error, jointly optimize the matching points from multiple perspectives to make the final three-dimensional coordinates close to the true values. Use the Levenberg-Marquardt (LM) algorithm to perform non-linear optimization on the three-dimensional coordinates. By minimizing the reprojection error, jointly optimize the matching points from multiple perspectives to make the final three-dimensional coordinates close to the true values. The LM algorithm combines the steepest descent method and the Gauss-Newton method and is an efficient optimization algorithm that can effectively handle non-linear least squares problems.

[0100] During the optimization process, construct a reprojection error objective function and iteratively solve this objective function to obtain the optimal three-dimensional coordinate estimation. This objective function is usually expressed as: , where, is the estimated three-dimensional point, is the projection of this point on the image, is the observed image point. By minimizing this error, a more accurate three-dimensional coordinate estimation can be obtained. When this method is used for the three-dimensional positioning of people in a multi-view video stream, by selecting an appropriate combination of perspectives, performing a depth consistency check, and using the LM algorithm for non-linear optimization, the estimation accuracy of the three-dimensional coordinates can be significantly improved.

[0101] S8, fuse the inertial measurement unit (IMU) data and the motion prediction model to perform temporal smoothing and error suppression on the positioning results.

[0102] Specifically, the fusion of IMU data in S8 is as follows: Establish a loose-coupling fusion framework based on error-state Kalman filtering. Adopt a loose-coupling fusion strategy to separate and process the IMU data from the positioning results of the Gaussian model. In the loose-coupling framework, the IMU data serves as an independent sensor module, and state estimation is performed through an Error State Kalman Filter (ESKF). This framework calculates position, velocity, and attitude information using IMU data during the prediction phase and corrects the IMU data using other sensors (such as GNSS or vision) during the update phase, thereby improving the positioning accuracy. The prediction process of the error-state Kalman filter is based on the kinematic model of the IMU, including recursive calculations of position, velocity, and attitude. The update process corrects the state of the IMU using the measurement data of other sensors and adjusts the uncertainty of the state estimation by calculating the Kalman gain.

[0103] When the positioning confidence of the Gaussian model is lower than the threshold, switch to IMU dead reckoning. When the positioning confidence of the Gaussian model is low, the system switches to the IMU dead reckoning mode. IMU dead reckoning calculates the current position and attitude by integrating the acceleration and angular velocity data of the IMU. This mode is particularly crucial when the GNSS signal is lost or blocked and can provide continuous positioning information. To improve the robustness of IMU dead reckoning, pre-integration techniques are usually adopted to process the IMU data in segments and reduce the accumulation of calculation errors. The pre-integrated measurement values can be used as motion constraints for calculations between the state quantities at two moments, thereby improving the continuity and stability of positioning.

[0104] Predict the periodic positioning deviation caused by the swinging of the lower limbs of the person through the kinematic model. During the movement of the person, the swinging of the lower limbs will cause periodic positioning deviations. To suppress this deviation, the system uses a kinematic model for prediction. The kinematic model is based on the principles of human kinematics, simulates the influence of the swinging of the lower limbs on the positioning results, and compensates for these deviations through a motion prediction model.

[0105] The motion prediction model can be modeled based on time series analysis or deep learning methods. By learning historical motion data, it predicts the future motion trajectory and adjusts the positioning results accordingly. For example, time series models such as LSTM (Long Short-Term Memory Network) can be used to model the periodic characteristics of the swinging of the lower limbs and perform compensation during the positioning process.

[0106] Embodiment 2 As Figure 4As shown in the figure, the present application provides a personnel positioning system based on a 3D Gaussian splatter model and video fusion, which is applied to the personnel positioning method based on the 3D Gaussian splatter model and video fusion as described in Embodiment 1, and includes: a multi-view video stream acquisition module 11, a sparse point cloud initialization module 12, a temporal dynamic tracking module 13, a temporal consistency optimization module 14, an object detection module 15, a three-dimensional to two-dimensional space matching module 16, a multi-view fusion positioning module 17, and a multi-sensor fusion optimization module 18.

[0107] It can be understood that in this embodiment, the multi-view video stream acquisition module 11 is used to acquire video streams of at least two views as input data; It can be understood that in this embodiment, the sparse point cloud initialization module 12 is used to generate an initial Gaussian distribution for the first frame of video through sparse initialization based on the SFM point cloud, construct the pose and density parameters of the three-dimensional Gaussian ellipsoid, and each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color, and opacity parameters to construct an initial three-dimensional Gaussian model; It can be understood that in this embodiment, the temporal dynamic tracking module 13 is used to track the displacement change of the three-dimensional Gaussian ellipsoid through the optical flow algorithm for subsequent frames, and dynamically update its position, covariance matrix, and color feature parameters; It can be understood that in this embodiment, the temporal consistency optimization module 14 is used to maintain the parameter consistency of the Gaussian model in the time dimension through the L2 regularization temporal optimization algorithm that minimizes parameter drift; It can be understood that in this embodiment, the object detection module 15 is used to extract the bounding box of the construction personnel in the video frame by using the Mask R-CNN object detection model; It can be understood that in this embodiment, the three-dimensional to two-dimensional space matching module 16 is used to perform spatial matching between the detected personnel area and the area of the three-dimensional Gaussian ellipsoid with the highest density in the Gaussian model, and establish the spatial correspondence between the pixel coordinates and the three-dimensional coordinates; It can be understood that in this embodiment, the multi-view fusion positioning module 17 is used to calculate the three-dimensional coordinates of the personnel through perspective geometric back-projection, and perform triangulation if multi-view input is adopted to improve the positioning accuracy; It can be understood that in this embodiment, the multi-sensor fusion optimization module 18 is used to fuse the inertial measurement unit IMU data and the motion prediction model to perform temporal smoothing and error suppression on the positioning result.

[0108] Figure 5 This is an electronic device provided by an embodiment of the present application. As Figure 5 shown, the electronic device at least includes the following parts: a processor 101, a memory 100, a communication interface 103, and a bus 102.

[0109] In an embodiment of the present application, the memory 100 is used to store executable instructions for the processor 101, and the processor 101 is configured to implement the method according to the first aspect when executing the instructions.

[0110] In an embodiment of the present application, a computer-readable storage medium includes instructions that direct a device to execute the method according to the first aspect. For example, the instructions direct the device to execute the method shown in the Figure 1 flow steps.

[0111] In an electronic device according to an embodiment of the present application, the program that operates may be a program that controls a central processing unit (CPU) or the like to implement the functions of the above-described embodiments related to a solution of the present invention (a program that causes a computer to function). Then, the information processed by these devices is temporarily stored in a random access memory (RAM) when being processed, and then stored in various ROMs such as a read-only memory (FlashROM), a hard disk drive (HDD), etc., and read out, corrected, and written by the CPU as needed.

[0112] It should be noted that a part of the electronic device according to the above-described embodiment can also be implemented by a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and implemented by reading the program recorded on the recording medium into the computer and executing it.

[0113] It should be noted that the "computer" mentioned here refers to a computer built into an electronic device, which is a computer including hardware such as an OS and peripheral devices. In addition, the "computer-readable recording medium" refers to a removable medium such as a floppy disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into a computer.

[0114] Moreover, the "computer-readable recording medium" may include: a medium that dynamically stores a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line; a medium that stores a program for a fixed time, such as a volatile memory inside a computer of a server or a client in this case. In addition, the above program may be a program for implementing a part of the above functions, and may also be a program that can implement the above functions by combining with a program already recorded in a computer.

[0115] In addition, the electronic device in the above-described embodiment can also be implemented as an aggregate (device group) composed of a plurality of devices. Each device constituting the device group may have some or all of the functions or functional blocks of the electronic device in the above-described embodiment. As the device group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0116] Those of ordinary skill in the art should recognize that the above embodiments are merely used to illustrate the present application and are not intended to limit the present application. As long as appropriate changes and variations made to the above embodiments fall within the scope of the spirit of the present application, they fall within the scope of protection required by the present application.

Claims

1. A personnel positioning method based on a 3D Gaussian splash model and video fusion, characterized in that The method includes: S1, collecting video streams from at least two perspectives as input data; S2, generating an initial Gaussian distribution for the first frame of the video through sparse initialization based on the SFM point cloud, constructing the pose and density parameters of a three-dimensional Gaussian ellipsoid, where each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color, and opacity parameters, and constructing an initial three-dimensional Gaussian model; S3, tracking the displacement changes of the three-dimensional Gaussian ellipsoids for subsequent frames through an optical flow algorithm, and dynamically updating their position, covariance matrix, and color feature parameters; S4, maintaining the parameter consistency of the Gaussian model in the time dimension through the L2-regularized temporal optimization algorithm that minimizes parameter drift; S5, using the Mask R-CNN object detection model to extract the bounding boxes of construction workers in the video frames; S6, spatially matching the detected person area with the area of the three-dimensional Gaussian ellipsoid with the highest density in the Gaussian model, and establishing the spatial correspondence relationship between pixel coordinates and three-dimensional coordinates; S7, calculating the three-dimensional coordinates of the person through perspective geometric back-projection, and performing triangulation if multi-perspective input is adopted to improve the positioning accuracy; S8, fusing the data of the inertial measurement unit IMU and the motion prediction model to perform temporal smoothing and error suppression on the positioning results.

2. The personnel positioning method based on the 3D Gaussian splash model and video fusion according to claim 1, characterized in that, The sparse initialization in S2 specifically includes: Performing structure from motion (SFM) reconstruction on the first frame of the multi-perspective video through COLMAP to extract the sparse point cloud; Initializing the Gaussian ellipsoid with the center of the point cloud as the mean and the point cloud distribution as the covariance; Encoding the view-related color features using spherical harmonics, and setting the initial opacity to a value in the range of 0.8 - 1.

0.

3. A personnel positioning method based on a 3D Gaussian splash model and video fusion according to claim 2, characterized in that, The generation of the initial Gaussian distribution includes: Combining the three-dimensional Gaussian distribution with the time component, and encoding the time opacity through a time radial basis function to model the emergence or disappearance of scene content; Using a time-conditioned parameter function to represent the position and rotation parameters of the three-dimensional Gaussian distribution to model the motion and deformation in the scene; For any spacetime point , its opacity is calculated as the product of a temporal opacity function and a spatial Gaussian distribution, where the spatial position and covariance , of the spatial Gaussian distribution are functions of time, and the opacity of the Gaussian distribution is as follows: ， Among them, represents the temporal opacity, and are the time-related spatial positions and covariances, represents the th Gaussian distribution; The time radial basis function is implemented using a learnable neural network or a preset basis function group, and is used to control the appearance and disappearance state changes of the Gaussian distribution on the time axis, expressed as: ， Among them, is the time center, representing the most prominent timestamp of the Gaussian distribution, represents the time scaling factor, which is used to determine its effective duration, represents the space opacity independent of time; For each Gaussian distribution, a time-conditioned function is used to model the changes in the position and rotation parameters of the three-dimensional Gaussian ellipsoid in the time dimension, expressed as: ， Among them, represents the spatial position of the Gaussian distribution at time , , is the corresponding coefficient, ; Calculate the color of the pixel based on the final spatial opacity , expressed as: ， Among them, is the color of the nth Gaussian.

4. A personnel positioning method based on a 3D Gaussian splash model and video fusion according to claim 2, characterized in that, The optical flow algorithm tracking in S3 adopts a hybrid tracking strategy that combines the Farneback dense optical flow and the LK sparse optical flow, including: Using the Farneback algorithm to perform global dense optical flow estimation, calculating the optical flow information of each pixel point in the entire image through the information of the front and back two frames of images, approximating the image frames through polynomial expansion, and estimating the displacement of the pixel points using the least squares method; Using the LK algorithm to perform sparse optical flow local detail tracking, selecting feature points for local matching, and only tracking the motion of some key points in the image; During the tracking process, if it is detected that the displacement of the Gaussian ellipsoid exceeds the preset threshold, the system will start the re-initialization process, re-detect the target feature points, and update the covariance matrix and color feature parameters; In each frame of the video sequence, calculate the color histogram similarity measure between the target area of the current frame and the target areas of at least one adjacent frame. When the similarity measure is lower than a preset threshold, it is determined that a significant change or occlusion event has occurred to the target; In response to the determination, a re-initialization operation of the tracker is triggered.

5. A personnel positioning method based on a 3D Gaussian splash model and video fusion according to claim 1, characterized in that The timing optimization of S4 includes: Establishing a Markov constraint on the change of Gaussian parameters between adjacent frames: Apply a second-order difference constraint on the position parameter to prevent the parameter from changing drastically in a short period of time. The constraint is expressed as: ， Among them, is a constraint threshold for controlling the amplitude of parameter change, respectively represent the position parameters at adjacent times ; The historical frame parameter influence weights are adjusted using exponential decay weighting, with small weights assigned to the parameters of distant frames and large weights assigned to the parameters of close frames. The weights are defined as: ， wherein is an attenuation factor, used to control the attenuation speed of the weight, to pay more attention to recent data in the time series while avoiding premature forgetting of historical information; To minimize parameter drift, an L2 regularization term is introduced to penalize the change of parameters, and the optimization objective function is expressed as: ， wherein, is the weight of the first-order difference constraint, is the L2 regularization weight, is the average parameter value of the current frame.

6. The personnel positioning method based on the 3D Gaussian splash model and video fusion according to claim 1, characterized in that, The spatial matching of S6 includes: Calculating the density-weighted average value of all Gaussian ellipsoids within the human bounding box; Selecting Gaussian clusters with opacity > 0.9 and density ranking in the top 5% as matching targets; Removing abnormal Gaussian distributions through the RANSAC algorithm.

7. A personnel positioning method based on a 3D Gaussian splash model and video fusion according to claim 1, characterized in that During multi-view triangulation in S7: Selecting a view combination with a baseline distance greater than 2 meters. A large baseline distance can significantly increase the angle between back-projected rays, thereby enhancing the geometric constraint of triangulation and reducing calculation errors; Performing a depth consistency check on matching points with an angle between back-projected rays less than 15°. By comparing the depth relationship between back-projected rays and target points from multiple views, it is judged whether it conforms to physical laws. If not, the matching point is removed to avoid misleading the estimation of the final three-dimensional coordinates; Using the LM algorithm to perform non-linear optimization on the three-dimensional coordinates. By minimizing the reprojection error, the matching points from multiple views are jointly optimized to make the final three-dimensional coordinates close to the true value.

8. A personnel positioning method based on a 3D Gaussian splash model and video fusion according to claim 1, characterized in that The specific process of fusing IMU data in S8 is as follows: Establishing a loose-coupling fusion framework based on error-state Kalman filtering; Switching to IMU dead reckoning when the positioning confidence of the Gaussian model is lower than the threshold; Predicting the periodic positioning deviation caused by the swinging of the lower limbs of the person through a kinematic model.

9. A personnel positioning system based on a 3D Gaussian splash model and video fusion, which is applied to the personnel positioning method based on a 3D Gaussian splash model and video fusion according to any one of claims 1 to 8, and is characterized in that, Including: A multi-view video stream acquisition module for acquiring video streams from at least two views as input data; A sparse point cloud initialization module for generating an initial Gaussian distribution through sparse initialization based on SFM point cloud for the first-frame video, constructing the pose and density parameters of three-dimensional Gaussian ellipsoids. Each Gaussian ellipsoid is defined by its spatial position, covariance matrix, color, and opacity parameters, and constructing an initial three-dimensional Gaussian model; A timing dynamic tracking module for tracking the displacement change of three-dimensional Gaussian ellipsoids through an optical flow algorithm for subsequent frames, and dynamically updating their position, covariance matrix, and color feature parameters; A timing consistency optimization module for maintaining the parameter consistency of the Gaussian model in the time dimension through an L2 regularization timing optimization algorithm that minimizes parameter drift; A target detection module for using the Mask R-CNN target detection model to extract the bounding box of construction workers in the video frame; A three-dimensional to two-dimensional spatial matching module for spatially matching the detected human area with the area of the three-dimensional Gaussian ellipsoid with the highest density in the Gaussian model, and establishing the spatial correspondence between pixel coordinates and three-dimensional coordinates; A multi-view fusion positioning module for calculating the three-dimensional coordinates of the person through perspective geometric back-projection. If multi-view input is adopted, triangulation is performed to improve the positioning accuracy; A multi-sensor fusion optimization module for fusing IMU data and a motion prediction model to perform timing smoothing and error suppression on the positioning result.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program, and the program instructs the device to execute the personnel positioning method based on the 3D Gaussian splash model and video fusion according to any one of claims 1 to 8.

Images