Target point coordinate measurement method, device and equipment of 3D Gaussian sputtering model and medium

Abstract

The application provides a target point coordinate measurement method, device and equipment of a 3D Gaussian sputtering model and a medium. The method comprises the following steps: generating an initial image based on an initial observation angle; constructing an initial line of sight ray based on an initial pixel coordinate of a target point and an initial camera optical center; generating an observation direction based on the initial camera optical center, calculating a camera displacement step length based on a target point depth, obtaining a candidate auxiliary observation angle and an auxiliary image, matching the target point in the auxiliary image through an optical flow tracking algorithm to obtain a matching pixel coordinate, and taking the candidate auxiliary observation angle meeting a preset error requirement as an effective observation angle; constructing a line of sight ray based on the matching pixel coordinate and the camera optical center for each effective observation angle; extracting Gaussian particles contributing to the pixel, calculating depth uncertainty based on depth value distribution to calculate a weight; constructing a weighted triangulation target function based on the effective observation angle weight and the line of sight ray, solving a minimum value to obtain a target point three-dimensional space coordinate. The method realizes coordinate measurement of the target point.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and medium for measuring the coordinates of target points in a 3D Gaussian sputtering model. Background Technology

[0002] With the iterative development of 3D reconstruction technology, 3D Gaussian sputtering (3DGS) models have rapidly become the mainstream solution for high-precision 3D scene reconstruction due to their high-fidelity scene restoration capabilities and real-time rendering efficiency. They are widely used in core fields such as digital twin factory construction, digital archiving of cultural relics, and reconstruction of the surface morphology of industrial parts.

[0003] However, the core value of existing 3DGS models is concentrated on "immersive visualization". Their underlying discretized Gaussian particle set can only support pixel generation in the rendering pipeline and cannot directly provide three-dimensional coordinate information of any point in the scene. As a result, the application of 3DGS models is limited to the "viewing" level and cannot meet the core "measurement" needs such as industrial inspection and cultural relic dimension annotation. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, device, equipment and medium for measuring the coordinates of target points in a 3D Gaussian sputtering model, so as to realize the measurement of the three-dimensional spatial coordinates of target points in the 3DGS model.

[0005] In a first aspect, embodiments of this application provide a method for measuring the coordinates of target points in a 3D Gaussian sputtering model, including: An initial image of the 3D Gaussian sputtering model is generated based on the initial observation view of the camera on the model. Based on the initial pixel coordinates of the target point in the initial image and the initial camera optical center when the camera is in the initial observation view, an initial line of sight ray is constructed from the initial camera optical center and passes through the initial pixel coordinates. Based on the initial camera optical center, multiple observation directions are generated according to a preset direction strategy and angle constraints. For each observation direction, the camera displacement step size is calculated based on the depth of the target point on the initial line of sight to obtain candidate auxiliary observation viewpoints. Through an optical flow tracking algorithm, the target point in the auxiliary image of the 3D Gaussian sputtering model under the candidate auxiliary observation viewpoint is tracked and matched to obtain the matching pixel coordinates of the target point on the auxiliary image. The candidate auxiliary observation viewpoints whose matching pixel coordinates meet the preset error requirements are taken as effective viewpoints. For each effective viewpoint, based on the matching pixel coordinates corresponding to the effective viewpoint and the camera optical center when the camera is in the effective viewpoint, a line of sight ray is constructed from the camera optical center and passes through the matching pixel coordinates. From the auxiliary image corresponding to the effective viewpoint, extract all Gaussian particles that contribute to the pixel where the target point is located, and calculate the depth uncertainty of the effective viewpoint based on the depth value distribution of each Gaussian particle. Based on the depth uncertainty of each effective viewpoint, the weight of each effective viewpoint is calculated; the depth uncertainty is negatively correlated with the weight. Based on the weights of all the effective viewpoints and the line-of-sight rays, a weighted triangulation objective function is constructed, and the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function are obtained as the three-dimensional spatial coordinates of the target point.

[0006] In conjunction with the first aspect, this application provides a first possible implementation of the first aspect, wherein generating an initial image of the 3D Gaussian sputtering model based on the initial observation view of the camera on the 3D Gaussian sputtering model includes: Set the initial pose of the camera in the three-dimensional space where the 3D Gaussian sputtering model is located; Obtain the intrinsic parameter matrix and resolution parameters of the camera; Based on the initial pose, the intrinsic parameter matrix, and the resolution parameters, the 3D Gaussian sputtering model is rendered using a rendering function to generate an initial image of the 3D Gaussian sputtering model from the initial observation viewpoint; the initial observation viewpoint is determined by the initial pose.

[0007] In conjunction with the first aspect, this application provides a second possible implementation of the first aspect, wherein, based on the initial camera optical center, multiple observation directions are generated according to a preset direction strategy and angle constraints; for each observation direction, the camera displacement step size is calculated based on the depth of the target point on the initial line-of-sight ray to obtain candidate auxiliary observation viewpoints; the target point in the auxiliary image of the 3D Gaussian sputtering model under the candidate auxiliary observation viewpoint is tracked and matched using an optical flow tracking algorithm to obtain the matching pixel coordinates of the target point on the auxiliary image; and the candidate auxiliary observation viewpoint whose matching pixel coordinates meet the preset error requirements is taken as the effective viewpoint, including: Starting from the initial camera optical center, multiple observation directions are determined according to an alternating direction strategy and angle constraints; wherein, the angle constraint is the camera optical axis direction vector of the ray corresponding to the observation direction, and the angle between the camera optical axis direction vector of the initial line of sight ray is between 15 degrees and 30 degrees. For each observation direction, the camera displacement step size along the current observation direction is calculated based on the depth of the target point on the initial line of sight and a preset coefficient. Starting from the initial camera optical center, the camera displacement step is moved along the current observation direction to obtain the candidate camera optical center. Based on the candidate camera optical center and the current observation direction, the candidate auxiliary observation angle of the camera is determined. Generate auxiliary images of the 3D Gaussian sputtering model from the candidate auxiliary observation viewpoints; The target point is tracked and matched in the auxiliary image using an optical flow tracking algorithm to obtain the matching pixel coordinates of the target point in the auxiliary image; Starting from the matched pixel coordinates, the optical flow tracing algorithm is used in reverse to trace back the initial pixel coordinates of the target point in the initial image, thus obtaining the reverse-tracked pixel coordinates. Calculate the consistency error between the matched pixel coordinates and the reverse-tracked pixel coordinates; If the consistency error is greater than the preset error threshold, the camera displacement step size is reduced by a preset reduction factor to obtain a new camera displacement step size. Based on the new camera displacement step size, the steps are re-executed, starting from the initial camera optical center and moving the camera displacement step size along the current observation direction to obtain candidate camera optical centers and subsequent steps, until the consistency error is less than or equal to the preset error threshold. Then, the candidate auxiliary observation angle of the current iteration is determined as the effective angle.

[0008] In conjunction with the first aspect, this application provides a third possible implementation of the first aspect, wherein calculating the depth uncertainty of the effective viewpoint based on the depth value distribution of each of the Gaussian particles includes: The depth uncertainty of this effective viewpoint is calculated using the following formula:

[0009]

[0010]

[0011] in, The depth uncertainty of the m-th effective viewpoint is represented by N; N represents the total number of all Gaussian particles that contribute to the pixel containing the target point. This represents the depth value of the i-th Gaussian particle; represents the three-dimensional center coordinates of the i-th Gaussian particle in three-dimensional space; t represents the optical center of the camera when the camera is in the m-th effective viewpoint; n represents the optical axis direction vector of the camera when the camera is in the m-th effective viewpoint; This represents the transparency weight of the pixel containing the target point on the auxiliary image at the m-th effective viewpoint for the i-th Gaussian particle. This is the variance calculation function.

[0012] In conjunction with the first aspect, this application provides a fourth possible implementation of the first aspect, wherein calculating the weight of each effective viewpoint based on the depth uncertainty of each effective viewpoint includes: The weight of each effective viewpoint is calculated using the following formula:

[0013] in, This represents the depth uncertainty of the m-th effective viewpoint; M' represents the weight of the m-th effective viewpoint; M' represents the total number of effective viewpoints. This represents the sum of the reciprocals of the depth uncertainty of all valid viewpoints.

[0014] In conjunction with the first aspect, this application provides a fifth possible implementation of the first aspect, wherein the step of constructing a weighted triangulation objective function based on the weights of all effective viewpoints and the line-of-sight ray, and solving for the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function, as the three-dimensional spatial coordinates of the target point, includes: The weighted triangulation objective function is:

[0015] in, This represents the total number of effective viewpoints; This represents the weight of the m-th effective viewpoint; Represents the three-dimensional spatial coordinates of the target point; This represents the camera optical center when the camera is in the m-th effective viewing angle; This represents the distance traveled along the line of sight corresponding to the m-th effective viewpoint. ; This represents the unit vector of the direction of the line-of-sight ray corresponding to the m-th effective viewpoint; By adjusting ,turn up and line of sight When the distance between the corresponding points is minimized , which serves as the three-dimensional spatial coordinates of the target point.

[0016] In conjunction with the first aspect, this application provides a sixth possible implementation of the first aspect, wherein, after constructing a weighted triangulation objective function based on the weights of all effective viewpoints and the line-of-sight ray, and solving for the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function, as the three-dimensional spatial coordinates of the target point, the method further includes: Calculate the average vertical distance from the three-dimensional spatial coordinates of the target point to each line of sight ray; If the average vertical distance is less than the preset accuracy threshold, it means that the three-dimensional spatial coordinate verification of the target point has passed; if the average vertical distance is greater than or equal to the preset accuracy threshold, it means that the three-dimensional spatial coordinate verification of the target point has failed.

[0017] Secondly, embodiments of this application also provide a target point coordinate measuring device for a 3D Gaussian sputtering model, comprising: The first generation module is used to generate an initial image of the 3D Gaussian sputtering model based on the initial observation view of the camera on the 3D Gaussian sputtering model. The first construction module is used to construct an initial line-of-sight ray that originates from the initial camera optical center and passes through the initial pixel coordinates, based on the initial pixel coordinates of the target point in the initial image and the initial camera optical center when the camera is in the initial observation view. The second generation module is used to generate multiple observation directions based on the initial camera optical center according to a preset direction strategy and angle constraints. For each observation direction, the camera displacement step size is calculated based on the depth of the target point on the initial line of sight to obtain candidate auxiliary observation viewpoints. The target point in the auxiliary image of the 3D Gaussian sputtering model under the candidate auxiliary observation viewpoint is tracked and matched by an optical flow tracking algorithm to obtain the matching pixel coordinates of the target point on the auxiliary image. The candidate auxiliary observation viewpoints whose matching pixel coordinates meet the preset error requirements are taken as effective viewpoints. The second construction module is used to construct, for each effective viewpoint, a line of sight ray that originates from the camera optical center and passes through the matching pixel coordinates, based on the matching pixel coordinates corresponding to the effective viewpoint and the camera optical center when the camera is in the effective viewpoint. The first calculation module is used to extract all Gaussian particles that contribute to the pixel where the target point is located from the auxiliary image corresponding to the effective viewpoint, and calculate the depth uncertainty of the effective viewpoint based on the depth value distribution of each Gaussian particle. The second calculation module is used to calculate the weight of each effective viewpoint based on the depth uncertainty of each effective viewpoint; the depth uncertainty is negatively correlated with the weight. The third construction module is used to construct a weighted triangulation objective function based on the weights of all the effective viewpoints and the line-of-sight ray, and to solve for the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function, which are then used as the three-dimensional spatial coordinates of the target point.

[0018] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps in any of the possible implementations of the first aspect described above are performed.

[0019] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps in any of the possible implementations of the first aspect described above.

[0020] This application provides a method, apparatus, device, and medium for measuring target point coordinates using a 3D Gaussian sputtering model. First, an initial image is rendered using the 3D Gaussian sputtering model based on a set initial observation viewpoint. After the user specifies a target point, an initial line-of-sight ray is constructed from the initial camera optical center based on the target point's initial pixel coordinates and the initial camera optical center, thus constraining the target point onto this ray. Then, based on the initial camera optical center and preset direction strategies and angle constraints, multiple observation directions are generated, and the camera displacement step size is calculated in conjunction with the target point's depth to determine candidate auxiliary observation viewpoints. For each candidate auxiliary observation viewpoint, a corresponding auxiliary image is rendered using the 3D Gaussian sputtering model, and an optical flow tracing algorithm is used to find the same target point within it. By verifying the error of the matching pixel coordinates, a stable and effective viewpoint is selected. This process ensures that the multi-view data used for subsequent calculations is reliable and possesses good geometric relationships.

[0021] After obtaining multiple effective viewpoints and their matching pixel coordinates, a corresponding line-of-sight ray is constructed for each effective viewpoint. Instead of simply assuming all effective viewpoint observations are equally reliable, this scheme extracts all Gaussian particles that contribute to the formation of the target point pixel from the auxiliary imagery of each effective viewpoint. By analyzing the depth value distribution of these Gaussian particles, the depth uncertainty, used to characterize the reliability of the effective viewpoint observation, is calculated. Effective viewpoints with smaller depth uncertainties are considered more reliable. Finally, appropriate weights are assigned to each effective viewpoint based on its depth uncertainty, and a weighted triangulation objective function is constructed. The goal of this function is to find a 3D spatial point such that the sum of the weighted distances from that point to all line-of-sight rays is minimized. By solving this optimization problem, the 3D spatial coordinates of the target point are finally output.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This document illustrates a flowchart of a method for measuring the coordinates of a target point in a 3D Gaussian sputtering model, as provided in an embodiment of this application. Figure 2 A schematic diagram of the optical center of a candidate camera provided in an embodiment of this application is shown; Figure 3 This invention provides a schematic diagram of the structure of a target point coordinate measuring device for a 3D Gaussian sputtering model according to an embodiment of this application. Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0026] With the iterative development of 3D reconstruction technology, 3D Gaussian sputtering (3DGS) models have rapidly become the mainstream solution for high-precision 3D scene reconstruction due to their high-fidelity scene restoration capabilities and real-time rendering efficiency. They are widely used in core fields such as digital twin factory construction, digital archiving of cultural relics, and reconstruction of the surface morphology of industrial parts.

[0027] However, the core value of existing 3DGS models is concentrated on "immersive visualization". Their underlying discretized Gaussian particle set can only support pixel generation in the rendering pipeline and cannot directly provide three-dimensional coordinate information of any point in the scene. As a result, the application of 3DGS models is limited to the "viewing" level and cannot meet the core "measurement" needs such as industrial inspection and cultural relic dimension annotation.

[0028] There are key technical bottlenecks in measuring the three-dimensional spatial coordinates of target points in existing 3DGS models: On the one hand, 3DGS is composed of discrete Gaussian particles and has no actual geometric structure. Directly calculating the target point coordinates through particle interpolation will introduce significant errors. On the other hand, the depth map rendered by 3DGS is a pixel-level apparent depth. It is affected by the overlap of particle projection and the mixing of transparency, resulting in low accuracy and not reflecting the actual geometric relationship. Therefore, it cannot be directly used for triangulation measurement.

[0029] Considering the technical pain point that 3DGS models are "visible but not measurable" in the existing technology, this application provides a method, device, equipment and medium for measuring the target point coordinates of a 3D Gaussian sputtering model, which will be described below through embodiments.

[0030] To facilitate understanding of this embodiment, a method for measuring the target point coordinates of a 3D Gaussian sputtering model disclosed in this application will first be described in detail. For example... Figure 1 As shown, the process includes the following steps S101-S107: S101: Generate the initial image of the 3D Gaussian sputtering model based on the initial observation view of the camera on the 3D Gaussian sputtering model.

[0031] In this step, the 3D Gaussian Splatting (3DGS) model is a 3D scene model reconstructed from the target scene or object using a 3D Gaussian sputtering algorithm. The 3D Gaussian sputtering model consists of Z Gaussian particles with position, orientation, scale, and color, denoted as: , .in, This indicates the position of the Gaussian particle in the three-dimensional space of the 3D Gaussian sputtering model; This represents the orientation vector of a Gaussian particle in this three-dimensional space; Indicates the scale parameter; This represents the RGB color value.

[0032] When users view a 3D Gaussian sputtering model, by adjusting different viewing angles, the Gaussian particles in the 3D Gaussian sputtering model will be rendered in real time, thus displaying images from different viewing angles.

[0033] The camera is a virtual observer located in the three-dimensional space where the 3D Gaussian sputtering model is situated, used to control the viewing angle of the 3D Gaussian sputtering model. In this embodiment, based on the camera's initial viewing angle of the 3D Gaussian sputtering model in the three-dimensional space where the 3D Gaussian sputtering model is situated, an initial image (i.e., an initial 2D image) of the 3D Gaussian sputtering model under that initial viewing angle is generated.

[0034] In this step, the camera resolution is H. W, camera intrinsic parameter matrix (Describe the relationship between optical properties and pixel mapping,) ,in Focal length (Primary point coordinates); Camera extrinsic matrix (Describe the camera's attitude and position in three-dimensional space,) , For rotation matrix, (This is a translation vector).

[0035] In one possible implementation, when performing step S101, the following steps S1011-S1013 can be specifically performed: S1011: Set the initial pose of the camera in the three-dimensional space where the 3D Gaussian sputtering model is located.

[0036] S1012: Obtain the camera's intrinsic parameter matrix and resolution parameters.

[0037] S1013: Based on the initial pose, intrinsic parameter matrix and resolution parameters, the 3D Gaussian sputtering model is rendered through the rendering function to generate the initial image of the 3D Gaussian sputtering model under the initial observation view; the initial observation view is determined by the initial pose.

[0038] In this step, the initial pose of the camera in the three-dimensional space where the 3D Gaussian sputtering model S is located is set. Obtain the camera's intrinsic parameter matrix K and resolution parameter H. W, through the rendering function Generate initial image The specific formula is as follows:

[0039] in, These are the pixel coordinates of the initial image. Gaussian point mass The coordinates projected onto the image plane. It is a 2D Gaussian distribution. This represents the transparency of the Gaussian particle at that pixel.

[0040] S102: Based on the initial pixel coordinates of the target point in the initial image and the initial camera optical center when the camera is in the initial observation view, construct the initial line of sight ray starting from the initial camera optical center and passing through the initial pixel coordinates.

[0041] In this step, the user is in the initial image Select the target point to be measured, receive the user's selection operation on the initial image, and obtain the initial pixel coordinates of the target point in the initial image. ), and based on the initial pixel coordinates ( Initial camera optical center when the camera is at the initial observation angle (i.e., the initial camera pose), and the camera intrinsic parameter matrix K, in the 3D space of the 3D Gaussian sputtering model, construct an initial line-of-sight ray originating from the initial camera optical center and passing through the initial pixel coordinates. The ray equation of the initial line-of-sight ray is:

[0042] in, The initial camera optical center is represented by h; the depth parameter is h.

[0043] S103: Based on the initial camera optical center, multiple observation directions are generated according to the preset direction strategy and angle constraints. For each observation direction, the camera displacement step size is calculated based on the depth of the target point on the initial line of sight ray to obtain candidate auxiliary observation viewpoints. Through the optical flow tracking algorithm, the target point in the auxiliary image of the 3D Gaussian sputtering model under the candidate auxiliary observation viewpoint is tracked and matched to obtain the matching pixel coordinates of the target point on the auxiliary image. The candidate auxiliary observation viewpoints whose matching pixel coordinates meet the preset error requirements are taken as effective viewpoints.

[0044] In this step, to accurately measure the three-dimensional spatial coordinates of the target point, it is necessary to observe the target point from multiple different perspectives. Therefore, the purpose of this step is to automatically calculate several other (e.g., 3-5) optimal camera shooting positions and angles.

[0045] In one possible implementation, when performing step S103, the following steps S1031-S1038 can be specifically performed: S1031: Starting from the initial camera optical center, multiple observation directions are determined according to the alternating direction strategy and angle constraints; where the angle constraint is the camera optical axis direction vector of the ray corresponding to the observation direction, and the angle between the ray and the camera optical axis direction vector of the initial line of sight ray is between 15 degrees and 30 degrees.

[0046] In this step, the alternating direction strategy refers to alternately selecting directions in the horizontal (left-right) and vertical (up-down) directions. For example... Figure 2 As shown, two observation directions are selected in the vertical direction.

[0047] In this embodiment, considering that if the angle between the camera optical axis direction vector of the ray corresponding to the observation direction and the camera optical axis direction vector of the initial line-of-sight ray is too small (<15°), the two lines of sight are almost parallel, and their intersection is highly sensitive to noise, resulting in extremely poor triangulation accuracy. If the angle is too large (>30°), the difference between the new viewpoint and the initial viewpoint is too great, which may cause the optical flow tracking algorithm to be unable to stably find the same point, resulting in matching failure. Therefore, the angle is constrained to within... .

[0048] S1032: For each observation direction, calculate the camera displacement step size along the current observation direction based on the depth of the target point on the initial line of sight and the preset coefficient.

[0049] In this step, the camera displacement step size in the current observation direction is calculated using the following formula:

[0050] in, The depth of the target point on the initial line of sight (determined by the depth of the nearest Gaussian particle in the 3DGS model), which is the distance between the target point and the initial camera optical center; These are preset coefficients; This represents the camera displacement step size.

[0051] S1033: Starting from the initial camera optical center, move the camera displacement step size along the current observation direction to obtain the candidate camera optical center. Based on the candidate camera optical center and the current observation direction, determine the candidate auxiliary observation angle of the camera.

[0052] like Figure 2 As shown, the solid black sphere represents the optical center of the candidate camera.

[0053] In this step, based on the optical center of the candidate camera and the current observation direction, a candidate auxiliary observation angle of the camera and its corresponding candidate auxiliary camera pose are determined.

[0054] S1034: Generate auxiliary images of the 3D Gaussian sputtering model from candidate auxiliary observation perspectives.

[0055] In this step, the candidate auxiliary camera pose, camera intrinsic parameter matrix K, and resolution parameter H corresponding to the candidate auxiliary observation viewpoint are... W is input to the rendering function. In this process, auxiliary images of the 3D Gaussian sputtering model are generated from candidate auxiliary observation perspectives. .

[0056] S1035: Using an optical flow tracking algorithm, the target point is tracked and matched in the auxiliary image to obtain the matching pixel coordinates of the target point on the auxiliary image.

[0057] In this step, the initial image Initial pixel coordinates of the target point Using the reference point, in the auxiliary image Optical flow tracking and matching are performed to obtain the target point in the auxiliary image. Matching pixel coordinates .

[0058] In this embodiment, the optical flow tracking algorithm can be RAFT optical flow (for recursive full-pair field transformation of optical flow) or LK optical flow (Lucas-Cannard optical flow method), which can realize dynamic matching of target points between multi-view images and ensure matching accuracy.

[0059] S1036: Starting from the matching pixel coordinates, the optical flow tracing algorithm is used in reverse to trace back the initial pixel coordinates of the target point in the initial image, thus obtaining the reverse-tracked pixel coordinates.

[0060] In this step, from the initial image Tracking auxiliary images Get matching pixel coordinates Then, match the pixel coordinates. Starting from the initial image, the optical flow tracing algorithm is used in reverse to trace back to the initial image. This yields a theoretical reverse-tracking pixel coordinate. .

[0061] S1037: Calculate the consistency error between the matching pixel coordinates and the reverse-tracked pixel coordinates.

[0062] In this step, the matching pixel coordinates are calculated using the following formula. With reverse tracking pixel coordinates Consistency error between :

[0063] S1038: If the consistency error is greater than the preset error threshold, the camera displacement step size is reduced by the preset reduction factor to obtain a new camera displacement step size. Based on the new camera displacement step size, the steps are re-executed, starting from the initial camera optical center and moving the camera displacement step size along the current observation direction to obtain the candidate camera optical center and subsequent steps, until the consistency error is less than or equal to the preset error threshold. Then, the candidate auxiliary observation angle of the current iteration is determined as the effective angle.

[0064] In this step, if the consistency error is less than or equal to the preset error threshold (preset error requirement), the matched pixel coordinates are... The corresponding candidate auxiliary observation perspectives were determined as effective perspectives.

[0065] If the consistency error is greater than the preset error threshold ( Preferred If the value is less than 1.5 pixels, then discard the candidate auxiliary observation view and regenerate it.

[0066] In this step, the preset reduction factor is 0.7, and the new camera displacement step size is calculated using the following formula: Based on the new camera displacement step size Repeat steps S1033-S1037 until the consistency error is less than or equal to the preset error threshold. Then, determine the candidate auxiliary observation viewpoints of the current iteration as effective viewpoints, thereby obtaining the effective viewpoints corresponding to each observation direction.

[0067] S104: For each effective viewpoint, based on the matching pixel coordinates corresponding to the effective viewpoint and the camera optical center when the camera is in the effective viewpoint, construct a line of sight ray starting from the camera optical center and passing through the matching pixel coordinates.

[0068] In this step, based on the matching pixel coordinates Camera optical center when the camera is in the effective field of view (i.e., camera pose) and camera intrinsic parameter matrix K, in the 3D space of the 3D Gaussian sputtering model, construct matching pixel coordinates starting from the optical center of the camera and passing through (through) it. The line of sight.

[0069] S105: Extract all Gaussian particles that contribute to the pixel where the target point is located from the auxiliary image corresponding to the effective viewpoint, and calculate the depth uncertainty of the effective viewpoint based on the depth value distribution of each Gaussian particle.

[0070] In this step, the color of a pixel is formed by projecting and mixing multiple Gaussian particles in the 3DGS model. All Gaussian particles that contribute to the pixel containing the target point refer to all pixels within the rendered target point from the effective viewpoint. All Gaussian points that are used and contribute to the final color, i.e., the pixels covering the target point within this effective viewpoint. All Gaussian particles.

[0071] In this step, the smaller the depth uncertainty, the more reliable the depth is. It is defined as the weighted value of the Gaussian particle depth variance of the pixel covering the target point under the effective viewpoint and the number of Gaussian particles.

[0072] In one possible implementation, when performing step S105 to calculate the depth uncertainty of the effective viewpoint based on the depth value distribution of each Gaussian particle, the specific steps can be as follows: The depth uncertainty of this effective viewpoint is calculated using the following formula:

[0073]

[0074]

[0075] in, The depth uncertainty of the m-th effective viewpoint is represented by N; N represents the total number of all Gaussian particles that contribute to the pixel containing the target point. This represents the depth value of the i-th Gaussian particle; represents the three-dimensional center coordinates of the i-th Gaussian particle in three-dimensional space; t represents the camera optical center when the camera is at the m-th effective viewpoint; n represents the camera optical axis direction vector when the camera is at the m-th effective viewpoint; This represents the transparency weight of the pixel containing the target point on the auxiliary image at the m-th effective viewpoint for the i-th Gaussian particle. This is the variance calculation function.

[0076] S106: Calculate the weight of each effective perspective based on the depth uncertainty of each effective perspective; depth uncertainty and weight are negatively correlated.

[0077] In this embodiment, the weight of each effective viewpoint is calculated using the following formula:

[0078] in, This represents the depth uncertainty of the m-th effective viewpoint; M' represents the weight of the m-th effective viewpoint; M' represents the total number of effective viewpoints. This represents the sum of the reciprocals of the depth uncertainty of all valid viewpoints.

[0079] In this embodiment, each valid line of sight corresponds to a depth uncertainty. The smaller the depth uncertainty, the greater the weight and the stronger the contribution to the triangulation result.

[0080] S107: Based on the weights of all effective viewpoints and the line-of-sight rays, construct a weighted triangulation objective function, and solve for the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function, which are then used as the three-dimensional spatial coordinates of the target point.

[0081] In this embodiment, the weighted triangulation objective function is:

[0082] in, Indicates the total number of effective viewpoints; This represents the weight of the m-th effective viewpoint; Represents the three-dimensional spatial coordinates of the target point; This represents the camera optical center when the camera is in the m-th effective viewing angle; This represents the distance traveled along the line of sight corresponding to the m-th effective viewpoint. ; This represents the unit vector of the direction of the line-of-sight ray corresponding to the m-th effective viewpoint; By adjusting ,turn up and line of sight When the distance between the corresponding points is minimized , which serves as the three-dimensional spatial coordinates of the target point.

[0083] In one possible implementation, after performing step S107, the accuracy of the three-dimensional spatial coordinates of the target point can be verified further by following the steps S1081-S1082: S1081: Calculate the average vertical distance from the three-dimensional spatial coordinates of the target point to each line of sight ray.

[0084] In this step, the average vertical distance is calculated using the following formula. :

[0085] in, Indicates the total number of effective viewpoints; This represents the perpendicular distance from the target point's three-dimensional spatial coordinates to the m-th line of sight ray.

[0086] S1082: If the average vertical distance is less than the preset accuracy threshold, it means that the three-dimensional spatial coordinate verification of the target point has passed; if the average vertical distance is greater than or equal to the preset accuracy threshold, it means that the three-dimensional spatial coordinate verification of the target point has failed.

[0087] In this step, if the average vertical distance is less than the preset accuracy threshold, it means that the accuracy of the three-dimensional spatial coordinates of the target point is high and the verification is passed; if the average vertical distance is greater than or equal to the preset accuracy threshold, it means that the accuracy of the three-dimensional spatial coordinates of the target point is poor and the verification is not passed.

[0088] Based on the same technical concept, this application also provides a target point coordinate measuring device for a 3D Gaussian sputtering model, such as... Figure 3 As shown, the device includes: The first generation module 301 is used to generate an initial image of the 3D Gaussian sputtering model based on the initial observation view of the camera on the 3D Gaussian sputtering model. The first construction module 302 is used to construct an initial line of sight ray that originates from the initial camera optical center and passes through the initial pixel coordinates, based on the initial pixel coordinates of the target point in the initial image and the initial camera optical center when the camera is in the initial observation view. The second generation module 303 is used to generate multiple observation directions based on the initial camera optical center according to a preset direction strategy and angle constraints. For each observation direction, the camera displacement step size is calculated based on the depth of the target point on the initial line of sight to obtain candidate auxiliary observation viewpoints. The target point in the auxiliary image of the 3D Gaussian sputtering model under the candidate auxiliary observation viewpoint is tracked and matched by an optical flow tracking algorithm to obtain the matching pixel coordinates of the target point on the auxiliary image. The candidate auxiliary observation viewpoints whose matching pixel coordinates meet the preset error requirements are taken as effective viewpoints. The second construction module 304 is used to construct, for each effective viewpoint, a line of sight ray that originates from the camera optical center and passes through the matching pixel coordinates, based on the matching pixel coordinates corresponding to the effective viewpoint and the camera optical center when the camera is in the effective viewpoint. The first calculation module 305 is used to extract all Gaussian particles that contribute to the pixel where the target point is located from the auxiliary image corresponding to the effective viewpoint, and calculate the depth uncertainty of the effective viewpoint based on the depth value distribution of each Gaussian particle. The second calculation module 306 is used to calculate the weight of each effective viewpoint based on the depth uncertainty of each effective viewpoint; the depth uncertainty is negatively correlated with the weight. The third construction module 307 is used to construct a weighted triangulation objective function based on the weights of all the effective viewpoints and the line of sight ray, and to solve for the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function, which are then used as the three-dimensional spatial coordinates of the target point.

[0089] Optionally, when the first generation module 301 generates an initial image of the 3D Gaussian sputtering model based on the initial observation view of the 3D Gaussian sputtering model by the camera, it is specifically used for: Set the initial pose of the camera in the three-dimensional space where the 3D Gaussian sputtering model is located; Obtain the intrinsic parameter matrix and resolution parameters of the camera; Based on the initial pose, the intrinsic parameter matrix, and the resolution parameters, the 3D Gaussian sputtering model is rendered using a rendering function to generate an initial image of the 3D Gaussian sputtering model from the initial observation viewpoint; the initial observation viewpoint is determined by the initial pose.

[0090] Optionally, the second generation module 303 is used to generate multiple observation directions based on the initial camera optical center according to a preset direction strategy and angle constraints; for each observation direction, calculate the camera displacement step size based on the depth of the target point on the initial line-of-sight ray to obtain candidate auxiliary observation viewpoints; track and match the target point in the auxiliary image of the 3D Gaussian sputtering model under the candidate auxiliary observation viewpoint using an optical flow tracking algorithm to obtain the matching pixel coordinates of the target point on the auxiliary image; and take the candidate auxiliary observation viewpoint whose matching pixel coordinates meet the preset error requirements as the effective viewpoint. Specifically, this is used to: Starting from the initial camera optical center, multiple observation directions are determined according to an alternating direction strategy and angle constraints; wherein, the angle constraint is the camera optical axis direction vector of the ray corresponding to the observation direction, and the angle between the camera optical axis direction vector of the initial line of sight ray is between 15 degrees and 30 degrees. For each observation direction, the camera displacement step size along the current observation direction is calculated based on the depth of the target point on the initial line of sight and a preset coefficient. Starting from the initial camera optical center, the camera displacement step is moved along the current observation direction to obtain the candidate camera optical center. Based on the candidate camera optical center and the current observation direction, the candidate auxiliary observation angle of the camera is determined. Generate auxiliary images of the 3D Gaussian sputtering model from the candidate auxiliary observation viewpoints; The target point is tracked and matched in the auxiliary image using an optical flow tracking algorithm to obtain the matching pixel coordinates of the target point in the auxiliary image; Starting from the matched pixel coordinates, the optical flow tracing algorithm is used in reverse to trace back the initial pixel coordinates of the target point in the initial image, thus obtaining the reverse-tracked pixel coordinates. Calculate the consistency error between the matched pixel coordinates and the reverse-tracked pixel coordinates; If the consistency error is greater than the preset error threshold, the camera displacement step size is reduced by a preset reduction factor to obtain a new camera displacement step size. Based on the new camera displacement step size, the steps are re-executed, starting from the initial camera optical center and moving the camera displacement step size along the current observation direction to obtain candidate camera optical centers and subsequent steps, until the consistency error is less than or equal to the preset error threshold. Then, the candidate auxiliary observation angle of the current iteration is determined as the effective angle.

[0091] Optionally, when the first calculation module 305 calculates the depth uncertainty of the effective viewpoint based on the depth value distribution of each of the Gaussian particles, it is specifically used for: The depth uncertainty of this effective viewpoint is calculated using the following formula:

[0092]

[0093]

[0094] in, The depth uncertainty of the m-th effective viewpoint is represented by N; N represents the total number of all Gaussian particles that contribute to the pixel containing the target point. This represents the depth value of the i-th Gaussian particle; represents the three-dimensional center coordinates of the i-th Gaussian particle in three-dimensional space; t represents the optical center of the camera when the camera is in the m-th effective viewpoint; n represents the optical axis direction vector of the camera when the camera is in the m-th effective viewpoint; This represents the transparency weight of the pixel containing the target point on the auxiliary image at the m-th effective viewpoint for the i-th Gaussian particle. This is the variance calculation function.

[0095] Optionally, when the second calculation module 306 calculates the weights of each effective viewpoint based on the depth uncertainty of each effective viewpoint, it is specifically used for: The weight of each effective viewpoint is calculated using the following formula:

[0096] in, This represents the depth uncertainty of the m-th effective viewpoint; M' represents the weight of the m-th effective viewpoint; M' represents the total number of effective viewpoints. This represents the sum of the reciprocals of the depth uncertainty of all valid viewpoints.

[0097] Optionally, when the third construction module 307 is used to construct a weighted triangulation objective function based on the weights of all the effective viewpoints and the line-of-sight ray, and to solve for the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function, as the three-dimensional spatial coordinates of the target point, it is specifically used for: The weighted triangulation objective function is:

[0098] in, This represents the total number of effective viewpoints; This represents the weight of the m-th effective viewpoint; Represents the three-dimensional spatial coordinates of the target point; This represents the camera optical center when the camera is in the m-th effective viewing angle; This represents the distance traveled along the line of sight corresponding to the m-th effective viewpoint. ; This represents the unit vector of the direction of the line-of-sight ray corresponding to the m-th effective viewpoint; By adjusting ,turn up and line of sight When the distance between the corresponding points is minimized , which serves as the three-dimensional spatial coordinates of the target point.

[0099] Optionally, the device further includes: The third calculation module is used to construct a weighted triangulation objective function based on the weights of all the effective viewpoints and the line-of-sight rays in the third construction module 307, solve for the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function, and use them as the three-dimensional spatial coordinates of the target point, and then calculate the average vertical distance from the three-dimensional spatial coordinates of the target point to each line-of-sight ray. The verification module is used to indicate that the three-dimensional spatial coordinates of the target point have passed verification if the average vertical distance is less than a preset accuracy threshold, and to indicate that the three-dimensional spatial coordinates of the target point have failed verification if the average vertical distance is greater than or equal to the preset accuracy threshold.

[0100] Figure 4 A schematic diagram of an electronic device provided in this application embodiment includes: a processor 401, a memory 402, and a bus 403. The memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device runs the above-described information processing method, the processor 401 and the memory 402 communicate through the bus 403. The processor 401 executes the machine-readable instructions to perform the steps of the method described in Embodiment 1.

[0101] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps described in Embodiment 1.

[0102] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, electronic devices, and computer-readable storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, electronic devices, and computer-readable storage media can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or modules may be electrical, mechanical, or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0106] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the technical scope disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for measuring the coordinates of a target point in a 3D Gaussian sputtering model, characterized in that, include: An initial image of the 3D Gaussian sputtering model is generated based on the initial observation view of the camera on the model. Based on the initial pixel coordinates of the target point in the initial image and the initial camera optical center when the camera is in the initial observation view, an initial line of sight ray is constructed from the initial camera optical center and passes through the initial pixel coordinates. Based on the initial camera optical center, multiple observation directions are generated according to a preset direction strategy and angle constraints. For each observation direction, the camera displacement step size is calculated based on the depth of the target point on the initial line of sight to obtain candidate auxiliary observation viewpoints. Through an optical flow tracking algorithm, the target point in the auxiliary image of the 3D Gaussian sputtering model under the candidate auxiliary observation viewpoint is tracked and matched to obtain the matching pixel coordinates of the target point on the auxiliary image. The candidate auxiliary observation viewpoints whose matching pixel coordinates meet the preset error requirements are taken as effective viewpoints. For each effective viewpoint, based on the matching pixel coordinates corresponding to the effective viewpoint and the camera optical center when the camera is in the effective viewpoint, a line of sight ray is constructed from the camera optical center and passes through the matching pixel coordinates. From the auxiliary image corresponding to the effective viewpoint, extract all Gaussian particles that contribute to the pixel where the target point is located, and calculate the depth uncertainty of the effective viewpoint based on the depth value distribution of each Gaussian particle. Based on the depth uncertainty of each effective viewpoint, the weight of each effective viewpoint is calculated; the depth uncertainty is negatively correlated with the weight. Based on the weights of all the effective viewpoints and the line-of-sight rays, a weighted triangulation objective function is constructed, and the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function are obtained as the three-dimensional spatial coordinates of the target point.

2. The method according to claim 1, characterized in that, The process of generating an initial image of the 3D Gaussian sputtering model based on the initial observation view of the camera on the model includes: Set the initial pose of the camera in the three-dimensional space where the 3D Gaussian sputtering model is located; Obtain the intrinsic parameter matrix and resolution parameters of the camera; Based on the initial pose, the intrinsic parameter matrix, and the resolution parameters, the 3D Gaussian sputtering model is rendered using a rendering function to generate an initial image of the 3D Gaussian sputtering model from the initial observation viewpoint; the initial observation viewpoint is determined by the initial pose.

3. The method according to claim 1, characterized in that, Based on the initial camera optical center, multiple observation directions are generated according to a preset direction strategy and angle constraints. For each observation direction, the camera displacement step size is calculated based on the depth of the target point on the initial line-of-sight ray to obtain candidate auxiliary observation viewpoints. An optical flow tracking algorithm is used to track and match the target point in the auxiliary image of the 3D Gaussian sputtering model under the candidate auxiliary observation viewpoint, obtaining the matching pixel coordinates of the target point on the auxiliary image. Candidate auxiliary observation viewpoints whose matching pixel coordinates meet preset error requirements are taken as valid viewpoints, including: Starting from the initial camera optical center, multiple observation directions are determined according to an alternating direction strategy and angle constraints; wherein, the angle constraint is the camera optical axis direction vector of the ray corresponding to the observation direction, and the angle between the camera optical axis direction vector of the initial line of sight ray is between 15 degrees and 30 degrees. For each observation direction, the camera displacement step size along the current observation direction is calculated based on the depth of the target point on the initial line of sight and a preset coefficient. Starting from the initial camera optical center, the camera displacement step is moved along the current observation direction to obtain the candidate camera optical center. Based on the candidate camera optical center and the current observation direction, the candidate auxiliary observation angle of the camera is determined. Generate auxiliary images of the 3D Gaussian sputtering model from the candidate auxiliary observation viewpoints; The target point is tracked and matched in the auxiliary image using an optical flow tracking algorithm to obtain the matching pixel coordinates of the target point in the auxiliary image; Starting from the matched pixel coordinates, the optical flow tracing algorithm is used in reverse to trace back the initial pixel coordinates of the target point in the initial image, thus obtaining the reverse-tracked pixel coordinates. Calculate the consistency error between the matched pixel coordinates and the reverse-tracked pixel coordinates; If the consistency error is greater than the preset error threshold, the camera displacement step size is reduced by a preset reduction factor to obtain a new camera displacement step size. Based on the new camera displacement step size, the steps are re-executed, starting from the initial camera optical center and moving the camera displacement step size along the current observation direction to obtain candidate camera optical centers and subsequent steps, until the consistency error is less than or equal to the preset error threshold. Then, the candidate auxiliary observation angle of the current iteration is determined as the effective angle.

4. The method according to claim 1, characterized in that, The calculation of the depth uncertainty of the effective viewpoint based on the depth value distribution of each Gaussian particle includes: The depth uncertainty of this effective viewpoint is calculated using the following formula: in, The depth uncertainty of the m-th effective viewpoint is represented by N; N represents the total number of all Gaussian particles that contribute to the pixel containing the target point. This represents the depth value of the i-th Gaussian particle; represents the three-dimensional center coordinates of the i-th Gaussian particle in three-dimensional space; t represents the optical center of the camera when the camera is in the m-th effective viewpoint; n represents the optical axis direction vector of the camera when the camera is in the m-th effective viewpoint; This represents the transparency weight of the pixel containing the target point on the auxiliary image at the m-th effective viewpoint for the i-th Gaussian particle. This is the variance calculation function.

5. The method according to claim 1, characterized in that, The calculation of the weights of each effective viewpoint based on the depth uncertainty of each effective viewpoint includes: The weight of each effective viewpoint is calculated using the following formula: in, This represents the depth uncertainty of the m-th effective viewpoint; M' represents the weight of the m-th effective viewpoint; M' represents the total number of effective viewpoints. This represents the sum of the reciprocals of the depth uncertainty of all valid viewpoints.

6. The method according to claim 1, characterized in that, The process of constructing a weighted triangulation objective function based on the weights of all effective viewpoints and the line-of-sight rays, and then finding the coordinates of the three-dimensional point that minimizes the value of the weighted triangulation objective function, which are then used as the three-dimensional coordinates of the target point, includes: The weighted triangulation objective function is: in, This represents the total number of effective viewpoints; This represents the weight of the m-th effective viewpoint; Represents the three-dimensional spatial coordinates of the target point; This represents the camera optical center when the camera is in the m-th effective viewing angle; This represents the distance traveled along the line of sight corresponding to the m-th effective viewpoint. ; This represents the unit vector of the direction of the line-of-sight ray corresponding to the m-th effective viewpoint; By adjusting ,turn up and line of sight When the distance between the corresponding points is minimized , which serves as the three-dimensional spatial coordinates of the target point.

7. The method according to claim 1, characterized in that, The method further includes: constructing a weighted triangulation objective function based on the weights of all effective viewpoints and the line-of-sight ray; finding the coordinates of the three-dimensional point that minimizes the value of the weighted triangulation objective function; and using these coordinates as the three-dimensional coordinates of the target point. Calculate the average vertical distance from the three-dimensional spatial coordinates of the target point to each line of sight ray; If the average vertical distance is less than the preset accuracy threshold, it means that the three-dimensional spatial coordinate verification of the target point has passed; if the average vertical distance is greater than or equal to the preset accuracy threshold, it means that the three-dimensional spatial coordinate verification of the target point has failed.

8. A target point coordinate measuring device for a 3D Gaussian sputtering model, characterized in that, include: The first generation module is used to generate an initial image of the 3D Gaussian sputtering model based on the initial observation view of the camera on the 3D Gaussian sputtering model. The first construction module is used to construct an initial line-of-sight ray that originates from the initial camera optical center and passes through the initial pixel coordinates, based on the initial pixel coordinates of the target point in the initial image and the initial camera optical center when the camera is in the initial observation view. The second generation module is used to generate multiple observation directions based on the initial camera optical center according to a preset direction strategy and angle constraints. For each observation direction, the camera displacement step size is calculated based on the depth of the target point on the initial line of sight to obtain candidate auxiliary observation viewpoints. The target point in the auxiliary image of the 3D Gaussian sputtering model under the candidate auxiliary observation viewpoint is tracked and matched by an optical flow tracking algorithm to obtain the matching pixel coordinates of the target point on the auxiliary image. The candidate auxiliary observation viewpoints whose matching pixel coordinates meet the preset error requirements are taken as effective viewpoints. The second construction module is used to construct, for each effective viewpoint, a line of sight ray that originates from the camera optical center and passes through the matching pixel coordinates, based on the matching pixel coordinates corresponding to the effective viewpoint and the camera optical center when the camera is in the effective viewpoint. The first calculation module is used to extract all Gaussian particles that contribute to the pixel where the target point is located from the auxiliary image corresponding to the effective viewpoint, and calculate the depth uncertainty of the effective viewpoint based on the depth value distribution of each Gaussian particle. The second calculation module is used to calculate the weight of each effective viewpoint based on the depth uncertainty of each effective viewpoint; The depth uncertainty is negatively correlated with the weight; The third construction module is used to construct a weighted triangulation objective function based on the weights of all the effective viewpoints and the line-of-sight ray, and to solve for the coordinates of the three-dimensional spatial point that minimizes the value of the weighted triangulation objective function, which are then used as the three-dimensional spatial coordinates of the target point.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the memory via the bus, and the machine-readable instructions, when executed by the processor, perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method as described in any one of claims 1 to 7.

Images