Three-dimensional panoramic super-resolution positioning data splicing method, device, equipment and medium

Abstract

The application discloses a three-dimensional panoramic super-resolution positioning data splicing method and device, equipment and medium, relates to the technical field of image processing, the positioning point of three-dimensional panoramic image is layered, the layered positioning point is aggregated based on the optimal neighborhood range, the spatial geometric structure is determined, the multi-head self-attention module is used for feature aggregation and semantic information propagation of the spatial geometric structure, the attention weight is calculated, the local spatial geometric feature is adaptively reweighted and iteratively updated, the local spatial geometric feature and the updated feature are fused, and the semantic feature is obtained; the first semantic feature is matched in geometric structure and semantic feature, the matching confidence between the first semantic features is calculated, and the second semantic feature is screened; the spatial offset is calculated by using the weighted centroid, so that the target positioning point is spliced, the splicing misplacement problem caused by the density heterogeneity of the splicing joint positioning data is solved, and the accuracy and efficiency of the three-dimensional panoramic super-resolution positioning data splicing are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for stitching three-dimensional panoramic super-resolution positioning data. Background Technology

[0002] 3D panoramic super-resolution positioning images are formed by stitching together a large number of small FOV (Field of View) 3D super-resolution data. Traditional super-resolution panoramic image stitching mainly relies on pixel registration in grayscale images, and can only render super-resolution positioning data at a fixed resolution. When the rendering resolution is not high, detailed information will be lost, and manual input of parameters is required, which cannot achieve automation. The registration error is large, and serious stitching misalignment is likely to occur.

[0003] Currently, data registration is performed using super-resolution original positioning table data to complete panoramic super-resolution positioning image stitching. However, existing methods only utilize two-dimensional planar information, such as positioning points or background fluorescence noise, and cannot process three-dimensional super-resolution positioning data with significant density heterogeneity, resulting in stitching misalignment and affecting subsequent analysis.

[0004] As can be seen from the above, how to solve the problem of stitching misalignment caused by the heterogeneity of positioning data density at the seam, realize the stitching of 3D panoramic super-resolution positioning data, improve the accuracy and efficiency of 3D panoramic super-resolution positioning data stitching, and thus improve the quality of 3D panoramic super-resolution positioning images is a problem to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for stitching three-dimensional panoramic super-resolution positioning data, which can solve the problem of stitching misalignment caused by the heterogeneity of positioning data density at the stitching seam, realize three-dimensional panoramic super-resolution positioning data stitching, improve the accuracy and efficiency of three-dimensional panoramic super-resolution positioning data stitching, and thus improve the quality of three-dimensional panoramic super-resolution positioning images. The specific solution is as follows: In a first aspect, this application discloses a method for stitching three-dimensional panoramic super-resolution positioning data, including: The system acquires positioning points at adjacent seams in a 3D panoramic image, adaptively layers these positioning points to determine the optimal neighborhood range adjacent to each positioning point, aggregates the layered positioning points based on the optimal neighborhood range to generate local spatial geometric features, and calculates the contribution of these local spatial geometric features to the geometric structure; the 3D panoramic image is an image at a preset resolution. Based on the local spatial geometric features and using a multilayer perceptron to determine the spatial geometric structure, a multi-head self-attention module and the contribution degree are used to perform feature aggregation and semantic information propagation on the spatial geometric structure to calculate the attention weight. Based on the attention weight, the local spatial geometric features are adaptively reweighted and iteratively updated to obtain the updated features. The local spatial geometric features and the updated features are fused to obtain the semantic features under the preset dimension. First semantic features are selected from all the semantic features. Geometric structure and semantic feature matching are performed on the first semantic features to calculate the matching confidence between the first semantic features. Second semantic features are selected from the first semantic features based on the matching confidence. The spatial offset is calculated using the weighted centroid of the second semantic feature, and the target positioning point is spliced ​​based on the spatial offset; the target positioning point is the positioning point corresponding to the second semantic feature.

[0006] Optionally, the adaptive hierarchical layering of the positioning point to determine the optimal neighborhood range adjacent to the positioning point includes: The positioning points are adaptively layered to obtain positioning points in multiple density layers; Calculate the eigenvalues ​​and eigenvectors using the covariance matrix of any location point in multiple density layers; Using the criterion of minimizing local structure, the optimal neighborhood range adjacent to the positioning point is determined based on the feature value and the feature vector.

[0007] Optionally, the aggregation of the layered positioning points based on the optimal neighborhood range to generate local spatial geometric features, and the calculation of the contribution of the local spatial geometric features to the geometric structure, includes: The local geometric unit set is obtained by aggregating the positioning points that are within the optimal neighborhood and whose distance from the layered positioning points is less than a preset threshold. The local geometric unit set is encoded using a multilayer perceptron to generate local spatial geometric features; Calculate the initial contribution of the local spatial geometric features to the geometric structure; The initial contribution is normalized using the Softmax function to obtain the contribution value.

[0008] Optionally, determining the spatial geometric structure based on the local spatial geometric features and using a multilayer perceptron includes: The local geometric unit set, local spatial geometric features, and position encoding features corresponding to the positioning point are concatenated to obtain the concatenated features; The stitched features are mapped onto a spatial structure using a multilayer perceptron to obtain the spatial geometric structure.

[0009] Optionally, the formula for calculating the matching confidence is: ; in, Let the matching confidence be between structures a and b. and These are the semantic embedding vectors of structures a and b under preset dimensions, respectively. and These are the weighted centroids of structures a and b, respectively. This is the predicted registration offset. It is a learnable mapping function.

[0010] Optionally, the step of filtering second semantic features from the first semantic features based on the matching confidence, and calculating the spatial offset using the weighted centroid of the second semantic features, includes: The TOP-K algorithm is used to select the second semantic features from the first semantic features based on the matching confidence. The spatial offset is calculated using the weighted centroid of the second semantic feature; wherein, the formula for calculating the spatial offset is: ; ; ; in, For the weighted centroid, and All are spatial structure points. Let the matching confidence be between structures a and b. This is the spatial offset.

[0011] Optionally, the three-dimensional panoramic super-resolution positioning data stitching method further includes: A loss function is constructed, and the loss function is used to optimize the process of stitching together 3D panoramic super-resolution positioning data; The loss function is: ; in, For loss function, For positive sample structure pairs, and These are the weighted centroids of structures a and b, respectively. The predicted registration offset is... and Let be the feature vectors corresponding to features a and b to be matched, respectively, and T denotes transpose.

[0012] Secondly, this application discloses a three-dimensional panoramic super-resolution positioning data stitching device, comprising: The geometric feature generation module is used to acquire positioning points at adjacent stitching seams of a 3D panoramic image, adaptively layer the positioning points to determine the optimal neighborhood range adjacent to the positioning points, aggregate the layered positioning points based on the optimal neighborhood range to generate local spatial geometric features, and calculate the contribution of the local spatial geometric features to the geometric structure; the 3D panoramic image is an image at a preset resolution. The semantic feature extraction module is used to determine the spatial geometric structure based on the local spatial geometric features and using a multilayer perceptron, and to perform feature aggregation and semantic information propagation on the spatial geometric structure using a multi-head self-attention module and the contribution degree to calculate the attention weight. Based on the attention weight, the local spatial geometric features are adaptively reweighted and iteratively updated to obtain the updated features. The local spatial geometric features and the updated features are fused to obtain the semantic features under the preset dimension. The matching confidence calculation module is used to filter out a first semantic feature from all the semantic features, perform geometric structure and semantic feature matching on the first semantic feature to calculate the matching confidence between the first semantic features, and filter out a second semantic feature from the first semantic feature based on the matching confidence. The splicing module is used to calculate the spatial offset using the weighted centroid of the second semantic feature, and splice the target positioning point based on the spatial offset; the target positioning point is the positioning point corresponding to the second semantic feature.

[0013] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned three-dimensional panoramic super-resolution positioning data stitching method.

[0014] Fourthly, this application discloses a computer storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed three-dimensional panoramic super-resolution positioning data stitching method.

[0015] As can be seen, this application adaptively layers the positioning points at adjacent stitching seams of 3D panoramic images to determine the optimal neighborhood range adjacent to the positioning points. Based on the optimal neighborhood range, the layered positioning points are aggregated to generate local spatial geometric features, reducing the dependence on the density uniformity of the original point set and solving the stitching misalignment problem caused by spatial density heterogeneity. Based on the local spatial geometric features and using a multilayer perceptron, the spatial geometric structure is determined. Multi-head self-attention modules and contribution values ​​are used to aggregate features and propagate semantic information of the spatial geometric structure to calculate attention weights, reducing the accumulation of errors in point-level matching, avoiding semantic ambiguity caused by indiscriminate feature learning, and improving the reliability and recognizability of high-dimensional semantic features. Based on the attention weights, the local spatial geometric features are adaptively reweighted and iteratively updated to strengthen core structural features and weaken invalid interference features. The local spatial geometric features and the updated features are fused to obtain semantic features under the preset dimension, solving the problem of easy loss of geometric information. To address the disconnect between semantic features and physical structure, a first semantic feature is selected from all semantic features. This improves the computational efficiency of the algorithm while ensuring matching accuracy, avoiding the computational waste caused by full feature matching. Geometric structure and semantic feature matching are performed on the first semantic feature to calculate the matching confidence between them. Dual matching of geometric structure and semantic features is then performed on the first semantic feature to effectively solve the problem of erroneous matching pairs that are semantically similar but have completely different geometric positions. This ensures the geometric and semantic consistency of matching pairs from a quantitative perspective. Based on the matching confidence, a second semantic feature is selected from the first semantic feature. The spatial offset is calculated using the weighted centroid of the second semantic feature. The target positioning points are then stitched together based on the spatial offset, achieving accurate spatial alignment of 3D positioning data and seamless connection of small field-of-view 3D positioning data. This improves the accuracy and efficiency of 3D panoramic super-resolution positioning data stitching, thereby enhancing the quality of 3D panoramic super-resolution positioning images. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of a three-dimensional panoramic super-resolution positioning data stitching method disclosed in this application; Figure 2 This is a structural diagram of a three-dimensional panoramic super-resolution positioning data stitching system disclosed in this application; Figure 3 This is a diagram illustrating the effect of pairwise registration data disclosed in this application; Figure 4 This application discloses a composite image of three simulated data. Figure 5 This is a panoramic super-resolution 3D image stitching result disclosed in this application; Figure 6 This is a graph showing the generalization experiment results of Act protein data disclosed in this application; Figure 7 This is a schematic diagram of the structure of a three-dimensional panoramic super-resolution positioning data stitching device disclosed in this application; Figure 8 This application provides a structural diagram of an electronic device. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] 3D panoramic super-resolution positioning images are formed by stitching together a large number of small FOV (field of view) 3D super-resolution data. Traditional super-resolution panoramic image stitching mainly relies on pixel registration in grayscale images, and can only render super-resolution positioning data at a fixed resolution. When the rendering resolution is low, detailed information is lost, and manual parameter input is required, making automation impossible. Registration errors are also significant, easily leading to severe stitching misalignment. Currently, data registration is performed using original super-resolution positioning table data to complete panoramic super-resolution positioning image stitching. However, existing methods only utilize two-dimensional planar information, such as positioning points or background fluorescence noise, and cannot handle 3D super-resolution positioning data with significant density heterogeneity, resulting in stitching misalignment and affecting subsequent analysis. Therefore, how to solve the stitching misalignment problem caused by the density heterogeneity of positioning data at the stitching seams, achieve 3D panoramic super-resolution positioning data stitching, improve the accuracy and efficiency of 3D panoramic super-resolution positioning data stitching, and thus improve the quality of 3D panoramic super-resolution positioning images is a problem to be solved in this field.

[0020] See Figure 1 As shown in the figure, this invention discloses a method for stitching three-dimensional panoramic super-resolution positioning data, which may specifically include: Step S11: Obtain the positioning points at the adjacent stitching seams of the 3D panoramic image, perform adaptive layering on the positioning points to determine the optimal neighborhood range adjacent to the positioning points, aggregate the layered positioning points based on the optimal neighborhood range to generate local spatial geometric features, and calculate the contribution of the local spatial geometric features to the geometric structure; the 3D panoramic image is an image at a preset resolution.

[0021] In this embodiment, the positioning points are adaptively layered to obtain positioning points in multiple density layers; eigenvalues ​​and eigenvectors are calculated using the covariance matrix of any positioning point in each of the multiple density layers; based on the eigenvalues ​​and eigenvectors, the optimal neighborhood range adjacent to the positioning point is determined using the criterion of minimizing local structure; positioning points within the optimal neighborhood range and whose distance to the layered positioning points is less than a preset threshold are aggregated to obtain a local geometric unit set; the local geometric unit set is encoded using a multilayer perceptron to generate local spatial geometric features; the initial contribution of the local spatial geometric features to the geometric structure is calculated; and the initial contribution is normalized using the Softmax function to obtain the contribution value.

[0022] By analyzing the imaging process of 3D super-resolution positioning data, significant spatial density heterogeneity of positioning data at adjacent seams can be observed, leading to mismatches of key positioning point pairs at different spatial levels. This application innovatively transforms the problem from point feature space to structural feature space by utilizing the spatial structural characteristics inherent in biological samples, avoiding dependence on point-level matching. To mitigate the interference of spatial density heterogeneity in 3D positioning data, spatial adaptive layering of positioning points at adjacent seams in 3D panoramic images is performed using local density, and preliminary local structures are constructed by learning the geometric relationships of spatial points. ; ; in, For the local density of a point in space, and The first i The point and the first j One point, The smoothing parameter for density-adaptive sensing is set to the median of the calculated points. Let i be the set of neighboring points. Using a density threshold, spatial points are divided into K density layers based on percentiles to achieve adaptive stratification. For point i The set of neighboring points, For a subset of the divided density intervals, for The point in the middle.

[0023] Subsequently, for any location point within the local neighborhood The eigenvalues ​​and eigenvectors are calculated using its covariance matrix, and the optimal neighborhood range adjacent to the location point is adaptively determined based on minimizing the local structure. ; in, r The neighborhood radius, For the first l The set of all candidate neighborhood radii in each density layer. These are eigenvalues, used to prevent numerical divergence. Pick , k For feature value index, Let be the eigenvalue.

[0024] To describe the local spatial geometry, the above formula identifies points within the optimal neighborhood and whose distance from the location point is less than the neighborhood radius. r (e.g., preset threshold = neighborhood radius) r The local geometric unit set is obtained by aggregating the positioning points of the local geometric unit set. Then, the MLP (Multilayer Perceptron) is used to analyze the local geometric unit set. Encode to form local spatial geometric features : ; In addition, in order to determine whether the location point belongs to a structural feature This application utilizes learnable functions Measuring the initial contribution of local spatial geometric features to the geometric structure Finally, the initial contribution is normalized using the Softmax function to obtain the contribution score: ; in, Let be the centroid of the structural point.

[0025] This invention transforms spatial point information into a structural feature space through adaptive hierarchical and geometric descriptor learning, and uses a learnable mapping function to dynamically allocate point-level contributions to structural modeling, thereby overcoming the inherent defects of original point pair matching.

[0026] Step S12: Based on the local spatial geometric features and using a multilayer perceptron to determine the spatial geometric structure, use a multi-head self-attention module and the contribution to perform feature aggregation and semantic information propagation on the spatial geometric structure to calculate the attention weight. Based on the attention weight, perform adaptive reweighting and iterative updating on the local spatial geometric features to obtain the updated features. Then, fuse the local spatial geometric features and the updated features to obtain the semantic features under the preset dimension.

[0027] In this embodiment, the process of determining the spatial geometric structure is as follows: the local geometric unit set, local spatial geometric features, and position encoding features corresponding to the positioning point are concatenated to obtain the concatenated features; the concatenated features are mapped to the spatial structure using a multilayer perceptron to obtain the spatial geometric structure.

[0028] To fully utilize the high-dimensional semantic information of spatial structures for high-precision feature registration, this application proposes to use local spatial structures as basic units, employ the weights of the original positioning data, and capture the high-dimensional semantic relationships between spatial structure points through a weighted adjustment multi-head geometric attention mechanism, thereby generating stable semantic features for subsequent feature matching.

[0029] Specifically, the local geometric unit set Local spatial geometric features and the location coding features corresponding to the positioning points. By concatenating the features and mapping them to a spatial structure using an MLP, the model can explicitly capture the spatial geometry while learning semantic relationships. : ; Then, the unified features of each dimension in the spatial geometry are aggregated and input into the multi-head self-attention module to obtain Quarry, Key, and Value. Then, an explicit modulation of the attention distribution is used, leveraging the prior weights of the localization data (i.e., the weights of the localization data). The contribution value obtained after normalization is used to aggregate features and propagate semantic information of the spatial geometry to calculate the attention weight; the attention weight is calculated as follows: ; in, For attention weights, m The number of multi-head attention heads is used to achieve spatial multi-view feature learning. d For feature dimension, l For the current density layer, t This is the index in the key-value pair. This application sets... This is to avoid numerical instability caused by log(0).

[0030] This step assigns higher weights to spatial points with high structural contributions, thereby generating more robust spatial structural features during spatial structural semantic modeling. This application employs a multi-head attention mechanism to fuse multi-source features and further utilizes attention-derived weights to adaptively reweight and iteratively update the structural features, resulting in the updated features: .

[0031] To preserve the geometric prior information of the original features, residual connections are used to fuse local spatial geometric features and updated features, thereby obtaining semantic features in a preset dimension. .

[0032] By combining the multi-view spatial learning capability of the multi-head attention mechanism with prior weights of spatial structure points, a high-dimensional representation transformation from point-level geometric features to structure-level semantic features is achieved. This design not only effectively alleviates the problem of accumulated registration errors in large-scale point pairs, but also provides strong interpretability for the semantic feature registration process.

[0033] Step S13: Select a first semantic feature from all the semantic features, perform geometric structure and semantic feature matching on the first semantic feature to calculate the matching confidence between the first semantic features, and select a second semantic feature from the first semantic feature based on the matching confidence.

[0034] In this embodiment, the formula for calculating the matching confidence level is: ; in, Let the matching confidence be between structures a and b. and These are the semantic embedding vectors of structures a and b under preset dimensions, respectively. and These are the weighted centroids of structures a and b, respectively. This is the predicted registration offset. It is a learnable mapping function used to project geometric displacement residuals onto the semantic space.

[0035] To effectively evaluate the accuracy of matching two high-dimensional semantic structural features based on geometric constraints, this application introduces the concept of a geometric-semantic collaborative mechanism, which can calculate the matching confidence using the formula for calculating the matching confidence.

[0036] By embedding geometric consistency as an explicit semantic correction (rather than an independent regularization term), this metric achieves a unified semantic space distance measure, which can both suppress semantically reasonable geometrically mismatched feature correspondences and improve the robustness of structural matching under density heterogeneity conditions.

[0037] Step S14: Calculate the spatial offset using the weighted centroid of the second semantic feature, and stitch the target positioning points together based on the spatial offset; the target positioning point is the positioning point corresponding to the second semantic feature.

[0038] In this embodiment, the TOP-K algorithm is used, and second semantic features are selected from the first semantic features based on the matching confidence; the spatial offset is calculated using the weighted centroid of the second semantic features; wherein, the formula for calculating the spatial offset is: ; ; ; in, For the weighted centroid, and Features with high matching confidence and Spatial structural points, Let the matching confidence be between structures a and b. This represents the spatial offset of the final adjacent seam positioning data.

[0039] For example, using the Top-K algorithm, the top three semantically highly consistent features are selected from the first semantic feature to form the second semantic feature, such as... Using the weighted centroid of the second semantic features representing the spatial geometric structure This application estimates spatial offsets to achieve stable and physically interpretable registration. It employs high-dimensional semantically consistent spatial structural features for registration and utilizes the weighted centroids of the corresponding structures to estimate physical spatial offsets, thereby achieving robust 3D positioning data registration.

[0040] In addition, it also includes: constructing a loss function and using the loss function to optimize the process of stitching together 3D panoramic super-resolution positioning data; The loss function is: ; in, For loss function, For positive sample structure pairs, and These are the weighted centroids of structures a and b, respectively. This is the predicted registration offset. and Let be the feature vectors corresponding to features a and b to be matched, respectively, and T denotes transpose.

[0041] To effectively learn the high-dimensional semantic features of the structure and ensure its stability in the registration of adjacent seam features, this application designs a loss function that simultaneously enforces semantic consistency and geometric structure regularity, thereby enhancing the physical rationality of the final registration result.

[0042] By jointly optimizing semantic consistency and structural consistency losses through a loss function, we can achieve structure-driven 3D super-resolution localization point learning from local geometry to global semantics, enabling the model to obtain robust global structural semantic embedding under conditions of density heterogeneity.

[0043] This application proposes a 3D panoramic super-resolution positioning data stitching system, wherein the system structure is as follows: Figure 2 As shown, including The system comprises three modules: a Structure-aware Adaptive aggregation module, a Semantic Feature Aggregation (SFAM) module, and a Geometry-Semantic Cooperative Mechanism (GSCM) module. First, an adaptive structure-aware aggregation module is constructed to break away from the traditional local point aggregation paradigm. It guides spatial skeleton structure perception through adaptive hierarchical layering of spatial positioning points and geometric relationship learning, thus avoiding interference from the heterogeneity of spatial density of positioning points on the splicing quality. Second, to ensure the accuracy of structural feature matching, a semantic feature aggregation module is introduced. Positioning point weights are used as explicit priors to optimize the skeleton structure feature representation, enhancing the reliability of high-dimensional semantic features. Furthermore, a geometry-semantic cooperative mechanism is designed by combining semantic space metrics and geometric space constraints to further improve the robustness of high-dimensional features for matching. Finally, Top-k is used to select highly matched semantic feature pairs, and the spatial offset is estimated using the weight centroid of their corresponding spatial geometric structures, thereby achieving a more stable, accurate, and semantically consistent spatial alignment effect.

[0044] The proposed 3D panoramic super-resolution localization data stitching method was validated using experimental data. The imaging experimental sample was U-2OS cells, and the sample model was labeled using Alexa Fluor 647 tags. The detector was a Hamamatsu Flash 4.0 V3 camera. The camera's exposure time was set to 10ms, and the imaging FOV size was [missing information]. Each field of view (FOV) contains 1500 original images, and the molecular density in each original image is [missing information]. The cylindrical lens in the imaging channel provides 3D imaging based on astigmatism, with an imaging depth of -300nm to 300nm.

[0045] Experimental data were obtained from the laboratory, and simulation data were obtained from publicly available dataset websites, including Zola3D (microtubule data), MT, and ACT (actin) data. This application cropped a complete localization table into four 2×2 localization tables with smaller fields of view, and the seam width at the overlap was [missing information]. This was done to verify the stitching performance of the panoramic images.

[0046] Microtube Data Pair Registration Experiment: The performance of the TripStitcher framework was evaluated on simulated microtube data with pair registration, and quantitatively compared with several state-of-the-art models. These methods included classic point-to-point matching geometric algorithms, state-of-the-art point cloud registration algorithms, and super-resolution localization image stitching algorithms (NanoStitcher, PNanoStitcher). Point-to-point matching geometric algorithms included ICP (Iterative Closest Point) and CPD (Coherent Point Drift); point cloud registration algorithms included PointNet++ and HybridReg; and super-resolution localization image stitching algorithms included NanoStitcher and PNanoStitcher. The super-resolution localization image stitching algorithm was designed to stitch together two-dimensional super-resolution localization data. This application employs a distributed stitching strategy to calculate the axial and lateral registration offsets. Pair registration comparisons are shown in Table 1. Table 1. Comparison of Paired Registration

[0047] In Table 1, the underlined result indicates the second-best result, and the bolded result indicates the best result.

[0048] The effect of pairwise registration data is as follows Figure 3 As shown, the visual performance comparison results of all competing methods on pairwise registration and localization data are presented. The TripStitcher algorithm of this application (first image on the right) has the highest similarity to Groundtruth (first image on the left). The other results have unsatisfactory registration accuracy due to the heterogeneity of spatial localization point density, and all show varying degrees of misalignment. Among them, PointNet++ (fourth image from the left) has the most obvious misalignment, indicated by the dashed arrow. Both quantitative and qualitative analyses confirm that the panoramic image stitched by TripStitcher is closest to Groundtruth in terms of stitching quality and registration error.

[0049] This application's ablation studies on three datasets provide strong evidence for the complementary effects of core components. The results show that, when used alone... The module uses preliminary features for matching to register relevant stitching seams, which can slightly improve the stitching performance of panoramic images. However, due to the lack of precise semantic guidance, it may produce incorrect matches. (Used alone...) The module effectively utilizes prior knowledge of localization points to extract high-dimensional semantic features of the spatial skeleton, thereby significantly improving the stitching effect. Specifically, on the Microtuble dataset, the MSE (Mean Square Error) decreased from 2631.1 to 528.2, and the Chamfer Distance (CD) value decreased from 0.84 to 0.40. Crucially, combined with… The complete model with the SFAM module achieved optimal performance across all core metrics: MSE, SSIM (structural similarity), CD, RE (registration error), and TIME (computational efficiency). Specifically, the MSE value of the Zola data decreased from 708.6 to 28.5, and the CD value decreased from 0.54 to 0.06. This demonstrates the significant importance of introducing prior weights for localization data and accurate matching of biological sample structural features for spatial 3D panoramic super-resolution data stitching. Comparisons of ablation experimental data are shown in Table 2. Table 2 Comparison of ablation experimental data

[0050] Table 3 Comparison of Panoramic Image Stitching Performance

[0051] Table 3 summarizes the quantitative evaluation of the panoramic image stitching performance of the TripStitcher algorithm and other algorithms on three simulated datasets (Mt, Zola, and Microtuble). The TripStitcher algorithm outperforms the other algorithms in overall metrics. The only exception is computational efficiency; the CPD algorithm achieves the highest stitching efficiency of only about seven seconds, demonstrating a significant speed advantage. However, it performs poorly in terms of image stitching quality and spatial point registration error, which is not conducive to panoramic image stitching. In the Mt and Zola datasets, the ICP and CPD algorithms perform similarly. Besides computational efficiency, this is because traditional point-to-point registration algorithms, while capable of calculating spatial point offsets, cannot handle 3D super-resolution localization data with heterogeneous spatial density. Furthermore, for example, on the Microtuble dataset, the MSE score (233.3) is significantly higher than the second-best method (2546.1), with a gain of nearly ten times, and the registration error RE decreases from 0.9295 to 0.0015, indicating the limitations of the distributed stitching strategy in existing panoramic super-resolution localization image stitching algorithms. These quantitative results collectively demonstrate that TripStitcher has set a new benchmark in the field of 3D panoramic super-resolution positioning image stitching, providing a more reliable and accurate solution.

[0052] The effects of splicing three types of simulated data are as follows Figure 4 As shown, Figure 4 The visual comparison results of all competing methods on three datasets are presented. TripStitcher demonstrates a significant advantage in the panoramic super-resolution localization image stitching task, maintaining good structural consistency with GroundTruth. However, due to interference from parameters and noise artifacts during the experiment, the overall stitching quality of the Microtuble dataset is worse than that of other datasets. Nevertheless, the stitching performance of this application is still superior to the other two stitching algorithms. Specifically, the dashed boxes represent stitching details, clearly showing varying degrees of stitching misalignment in the other algorithms (arrows), making it difficult to maintain good consistency with GroundTruth. Both quantitative and qualitative analyses confirm that TripStitcher is closest to GroundTruth in terms of panoramic image stitching quality and spatial localization point registration error, which strongly demonstrates the effectiveness of the stitching strategy proposed in this paper.

[0053] To verify the stitching performance of the proposed TripStitcher algorithm on actual panoramic experimental data, experiments were conducted on 3×3 3D experimental data to be registered, achieving panoramic stitching of 3D positioning data. The NanoStitcher algorithm (a super-resolution positioning imaging algorithm) and the HybridReg algorithm (a point cloud algorithm) were selected for comparison with the algorithm in this application. The NanoStitcher algorithm employs a distributed axial-lateral registration strategy to stitch panoramic super-resolution 3D images. The results of the panoramic super-resolution 3D image stitching are shown below. Figure 5 As shown, the color bars next to the image represent the axial depth of the three-dimensional positioning point. The color bars from red to blue represent the depth of the positioning point from the focal plane from 300nm to -300nm.

[0054] Since the experimental data to be registered do not contain a true ground truth, a qualitative evaluation method is used to measure its stitching performance. The method proposed in this paper has no obvious misalignment in the details, while the stitching results of NanoStitcher have obvious structural misalignment and discontinuity.

[0055] To verify the generalization ability of TripStitcher on different biological samples, a generalization experiment was conducted on Act protein data, and the results were compared with several mainstream methods. The algorithm proposed in this application shows significant advantages in terms of panoramic image stitching quality and spatial point matching error. Specifically, TripStitcher achieved the lowest MSE (55.3), CD (0.03), and RE (0.002) scores, and the highest SSIM (0.99), indicating that the stitched panoramic images maintain strong competitiveness in terms of stitching quality and spatial point registration error, verifying that the proposed strategy has good generalization ability and practical value. The comparison results of the generalization experiment are shown in Table 4: Table 4 Comparison Results of Generalization Experiments

[0056] To further verify the registration performance of this algorithm in the generalization experiment, details at the stitching seams of the panoramic super-resolution localization image were magnified. The generalization experiment results of the Act protein data are as follows: Figure 6 As shown, the ICP, CPD, PointNet++, HybridReg, and NanoStitcher algorithms all exhibit varying degrees of misalignment in their details (indicated by arrows). However, the TripStitcher algorithm shows no significant differences in its details and maintains good consistency with Groundtruth.

[0057] This application breaks through the traditional local point aggregation paradigm in the 3D panoramic super-resolution localization image stitching neighborhood, transforming the 3D super-resolution localization point registration problem from point-pair matching to spatial structure feature registration. A structure-aware adaptive aggregation module is designed, which uses local density to perform spatial adaptive layering of localization point data and learns the geometric relationships between spatial points to construct a preliminary local structure. A semantic feature aggregation module is proposed, which uses the weight information of the original localization points to modulate a multi-head geometric attention mechanism, thereby capturing high-dimensional semantic relationships between spatial structure points and generating stable semantic features for subsequent feature matching. To effectively evaluate the accuracy of matching two high-dimensional semantic structure features based on geometric constraints, a geometric-semantic collaborative mechanism is introduced to calculate the similarity of semantic features. A complete closed-loop process is proposed: utilizing the designed... The module addresses point density heterogeneity and perceives spatial structure information → utilizes a semantic feature aggregation module to extract high-dimensional semantic features → employs a geometric-semantic collaboration mechanism to obtain semantically highly consistent feature pairs → calculates spatial registration offset using weighted geometric centroids → completes the stitching.

[0058] This invention breaks through the traditional local point aggregation paradigm, transforming the problem from spatial point pair matching to spatial structural feature registration; the proposed 3D panoramic super-resolution positioning image stitching algorithm can effectively solve the stitching misalignment problem caused by the heterogeneity of positioning data density at the stitching seam; and its performance is more stable than other super-resolution positioning image stitching algorithms.

[0059] This application adaptively hierarchically layers the localization points at adjacent stitching seams in 3D panoramic images to determine the optimal neighborhood range adjacent to the localization points. Based on the optimal neighborhood range, the layered localization points are aggregated to generate local spatial geometric features, reducing the dependence on the density uniformity of the original point set and solving the stitching misalignment problem caused by spatial density heterogeneity. Based on the local spatial geometric features and using a multilayer perceptron, the spatial geometric structure is determined. A multi-head self-attention module and contribution are used to aggregate features and propagate semantic information about the spatial geometric structure to calculate attention weights, reducing the accumulation of errors in point-level matching, avoiding semantic ambiguity caused by indiscriminate feature learning, and improving the reliability and recognizability of high-dimensional semantic features. Based on the attention weights, the local spatial geometric features are adaptively reweighted and iteratively updated to strengthen core structural features and weaken invalid interference features. The local spatial geometric features and the updated features are fused to obtain semantic features under a preset dimension, solving the problems of easy loss of geometric information and semantic ambiguity. To address the issue of the disconnect between features and physical structure, a first semantic feature is selected from all semantic features. This improves the computational efficiency of the algorithm while ensuring matching accuracy, avoiding the computational waste caused by full feature matching. Geometric structure and semantic feature matching are performed on the first semantic feature to calculate the matching confidence between them. Dual matching of geometric structure and semantic features is then performed on the first semantic feature to effectively solve the problem of erroneous matching pairs that are semantically similar but have completely different geometric positions. This ensures the geometric and semantic consistency of matching pairs from a quantitative perspective. Based on the matching confidence, a second semantic feature is selected from the first semantic feature. The spatial offset is calculated using the weighted centroid of the second semantic feature. The target positioning points are then stitched together based on the spatial offset, achieving accurate spatial alignment of 3D positioning data and seamless integration of small field-of-view 3D positioning data. This improves the accuracy and efficiency of 3D panoramic super-resolution positioning data stitching, thereby enhancing the quality of 3D panoramic super-resolution positioning images.

[0060] See Figure 7 As shown in the figure, an embodiment of the present invention discloses a three-dimensional panoramic super-resolution positioning data stitching device, which may specifically include: The geometric feature generation module 11 is used to acquire the positioning points at the adjacent stitching seams of the three-dimensional panoramic image, adaptively layer the positioning points to determine the optimal neighborhood range adjacent to the positioning points, aggregate the layered positioning points based on the optimal neighborhood range to generate local spatial geometric features, and calculate the contribution of the local spatial geometric features to the geometric structure; the three-dimensional panoramic image is an image at a preset resolution. The semantic feature extraction module 12 is used to determine the spatial geometric structure based on the local spatial geometric features and using a multilayer perceptron, to perform feature aggregation and semantic information propagation on the spatial geometric structure using a multi-head self-attention module and the contribution, to calculate the attention weight, to perform adaptive reweighting and iterative updating on the local spatial geometric features based on the attention weight, to obtain the updated features, and to fuse the local spatial geometric features and the updated features to obtain the semantic features under the preset dimension. The matching confidence calculation module 13 is used to filter out a first semantic feature from all the semantic features, perform geometric structure and semantic feature matching on the first semantic feature to calculate the matching confidence between the first semantic features, and filter out a second semantic feature from the first semantic feature based on the matching confidence. The splicing module 14 is used to calculate the spatial offset using the weighted centroid of the second semantic feature, and splice the target positioning point based on the spatial offset; the target positioning point is the positioning point corresponding to the second semantic feature.

[0061] In some specific embodiments, the geometric feature generation module 11 may specifically include: The adaptive layering module is used to adaptively layer the positioning points to obtain positioning points with multiple density layers. The eigenvalue and eigenvector calculation module is used to calculate eigenvalues ​​and eigenvectors from the covariance matrix of any location point in multiple density layers. The optimal neighborhood range determination module is used to determine the optimal neighborhood range adjacent to the positioning point based on the feature value and the feature vector, with the minimization of local structure as the criterion.

[0062] In some specific embodiments, the geometric feature generation module 11 may specifically include: The aggregation module is used to aggregate positioning points that are within the optimal neighborhood and whose distance to the layered positioning points is less than a preset threshold, to obtain a local geometric unit set. The encoding module is used to encode the set of local geometric units using a multilayer perceptron to generate local spatial geometric features; An initial contribution calculation module is used to calculate the initial contribution of the local spatial geometric features to the geometric structure; The normalization module is used to normalize the initial contribution using the Softmax function to obtain the contribution value.

[0063] In some specific embodiments, the semantic feature extraction module 12 may specifically include: The feature splicing module is used to splice the local geometric unit set, local spatial geometric features, and the position encoding features corresponding to the positioning point to obtain the spliced ​​features; The mapping module is used to map the stitched features to a spatial structure using a multilayer perceptron to obtain the spatial geometric structure.

[0064] In some specific embodiments, the formula for calculating the matching confidence is: ; in, Let the matching confidence be between structures a and b. and These are the semantic embedding vectors of structures a and b under preset dimensions, respectively. and These are the weighted centroids of structures a and b, respectively. This is the predicted registration offset. It is a learnable mapping function.

[0065] In some specific embodiments, the matching confidence calculation module 13 may specifically include: The filtering module is used to filter out second semantic features from the first semantic features based on the matching confidence using the TOP-K algorithm.

[0066] In some specific embodiments, the splicing module 14 may specifically include: The spatial offset calculation module is used to calculate the spatial offset using the weighted centroid of the second semantic feature; wherein, the formula for calculating the spatial offset is: ; ; ; in, For the weighted centroid, and All are spatial structure points. Let the matching confidence be between structures a and b. This is the spatial offset.

[0067] In some specific embodiments, the three-dimensional panoramic super-resolution positioning data stitching device may further include: An optimization module is used to construct a loss function and use the loss function to optimize the process of stitching together 3D panoramic super-resolution positioning data. The loss function is: ; in, For loss function, For positive sample structure pairs, and These are the weighted centroids of structures a and b, respectively. The predicted registration offset is... and Let be the feature vectors corresponding to features a and b to be matched, respectively, and T denotes transpose.

[0068] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the chip secure boot detection method performed by the electronic device disclosed in any of the foregoing embodiments.

[0069] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0070] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0071] The operating system 221 manages and controls the various hardware devices on the electronic device 20 and the computer program 222 to enable the processor 21 to perform calculations and processing on the data 223 in the memory 22. The operating system 221 can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the chip secure boot detection method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the chip secure boot detection device from external devices, as well as data collected by its own input / output interface 25.

[0072] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0073] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the chip secure boot detection method steps disclosed in any of the foregoing embodiments.

[0074] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] The present invention provides a detailed description of a chip secure boot detection method, apparatus, device, and storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for stitching three-dimensional panoramic super-resolution positioning data, characterized in that, include: The system acquires positioning points at adjacent seams in a 3D panoramic image, adaptively layers these positioning points to determine the optimal neighborhood range adjacent to each positioning point, aggregates the layered positioning points based on the optimal neighborhood range to generate local spatial geometric features, and calculates the contribution of these local spatial geometric features to the geometric structure; the 3D panoramic image is an image at a preset resolution. Based on the local spatial geometric features and using a multilayer perceptron to determine the spatial geometric structure, a multi-head self-attention module and the contribution degree are used to perform feature aggregation and semantic information propagation on the spatial geometric structure to calculate the attention weight. Based on the attention weight, the local spatial geometric features are adaptively reweighted and iteratively updated to obtain the updated features. The local spatial geometric features and the updated features are fused to obtain the semantic features under the preset dimension. First semantic features are selected from all the semantic features. Geometric structure and semantic feature matching are performed on the first semantic features to calculate the matching confidence between the first semantic features. Second semantic features are selected from the first semantic features based on the matching confidence. The spatial offset is calculated using the weighted centroid of the second semantic feature, and the target positioning point is spliced ​​based on the spatial offset; the target positioning point is the positioning point corresponding to the second semantic feature.

2. The three-dimensional panoramic super-resolution positioning data stitching method according to claim 1, characterized in that, The adaptive hierarchical layering of the positioning point to determine the optimal neighborhood range adjacent to the positioning point includes: The positioning points are adaptively layered to obtain positioning points in multiple density layers; Calculate the eigenvalues ​​and eigenvectors using the covariance matrix of any location point in multiple density layers; Using the criterion of minimizing local structure, the optimal neighborhood range adjacent to the positioning point is determined based on the feature value and the feature vector.

3. The three-dimensional panoramic super-resolution positioning data stitching method according to claim 1, characterized in that, The aggregation of the hierarchical positioning points based on the optimal neighborhood range to generate local spatial geometric features, and the calculation of the contribution of the local spatial geometric features to the geometric structure, includes: The local geometric unit set is obtained by aggregating the positioning points that are within the optimal neighborhood and whose distance from the layered positioning points is less than a preset threshold. The local geometric unit set is encoded using a multilayer perceptron to generate local spatial geometric features; Calculate the initial contribution of the local spatial geometric features to the geometric structure; The initial contribution is normalized using the Softmax function to obtain the contribution value.

4. The three-dimensional panoramic super-resolution positioning data stitching method according to claim 3, characterized in that, The determination of spatial geometric structure based on the local spatial geometric features and using a multilayer perceptron includes: The local geometric unit set, local spatial geometric features, and position encoding features corresponding to the positioning point are concatenated to obtain the concatenated features; The stitched features are mapped onto a spatial structure using a multilayer perceptron to obtain the spatial geometric structure.

5. The three-dimensional panoramic super-resolution positioning data stitching method according to claim 1, characterized in that, The formula for calculating the matching confidence is: ； in, Let the matching confidence be between structures a and b. and These are the semantic embedding vectors of structures a and b under preset dimensions, respectively. and These are the weighted centroids of structures a and b, respectively. This is the predicted registration offset. It is a learnable mapping function.

6. The three-dimensional panoramic super-resolution positioning data stitching method according to claim 1, characterized in that, The second semantic feature is selected from the first semantic feature based on the matching confidence. Calculating the spatial offset using the weighted centroid of the second semantic feature includes: The TOP-K algorithm is used to select the second semantic features from the first semantic features based on the matching confidence. The spatial offset is calculated using the weighted centroid of the second semantic feature; wherein, the formula for calculating the spatial offset is: ； ； ； in, For the weighted centroid, and All are spatial structure points. Let the matching confidence be between structures a and b. This is the spatial offset.

7. The three-dimensional panoramic super-resolution positioning data stitching method according to any one of claims 1 to 6, characterized in that, Also includes: A loss function is constructed, and the loss function is used to optimize the process of stitching together 3D panoramic super-resolution positioning data; The loss function is: ； in, For loss function, For positive sample structure pairs, and These are the weighted centroids of structures a and b, respectively. This is the predicted registration offset. and Let be the feature vectors corresponding to features a and b to be matched, respectively, and T denotes transpose.

8. A three-dimensional panoramic super-resolution positioning data stitching device, characterized in that, include: The geometric feature generation module is used to acquire positioning points at adjacent stitching seams of a 3D panoramic image, adaptively layer the positioning points to determine the optimal neighborhood range adjacent to the positioning points, aggregate the layered positioning points based on the optimal neighborhood range to generate local spatial geometric features, and calculate the contribution of the local spatial geometric features to the geometric structure; the 3D panoramic image is an image at a preset resolution. The semantic feature extraction module is used to determine the spatial geometric structure based on the local spatial geometric features and using a multilayer perceptron, perform feature aggregation and semantic information propagation on the spatial geometric structure using semantic feature aggregation and the contribution degree to calculate the attention weight, perform adaptive reweighting and iterative update on the local spatial geometric features based on the attention weight to obtain the updated features, and fuse the local spatial geometric features and the updated features to obtain the semantic features under the preset dimension. The matching confidence calculation module is used to filter out a first semantic feature from all the semantic features, perform geometric structure and semantic feature matching on the first semantic feature to calculate the matching confidence between the first semantic features, and filter out a second semantic feature from the first semantic feature based on the matching confidence. The splicing module is used to calculate the spatial offset using the weighted centroid of the second semantic feature, and splice the target positioning point based on the spatial offset; the target positioning point is the positioning point corresponding to the second semantic feature.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the three-dimensional panoramic super-resolution positioning data stitching method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the three-dimensional panoramic super-resolution positioning data stitching method as described in any one of claims 1 to 7.

Images