Point cloud digital service system and method based on digital twinning

Abstract

The invention discloses a point cloud digital service system and method based on digital twinning, and belongs to the field of digital twinning, and the system comprises an entity model construction module, a fusion updating module, a semantic segmentation labeling module, a digital interface construction module and a feedback optimization module. By constructing the user interaction behavior portrait and the service calling feature vector, accurate modeling of different user use preferences and model operation modes is realized, a point cloud processing algorithm can be differentially optimized according to individual requirements, and the processing precision and the user experience are remarkably improved. A federated learning mechanism is introduced, each edge device completes training and parameter updating of a point cloud processing model locally, only the model weight difference is uploaded, and original user data is not involved, so that cross-terminal joint optimization is realized while privacy security is guaranteed.

Description

Technical Field

[0001] This invention belongs to the field of digital twin technology, and more specifically, relates to a point cloud digital service system and method based on digital twins. Background Technology

[0002] With the widespread application of digital twin technology in industrial manufacturing, smart buildings, and urban management, point cloud data, as a three-dimensional spatial mapping of the real physical world, has become an important data source for building and maintaining digital twin models. Large amounts of point cloud data collected through methods such as LiDAR, structured light, and photogrammetry can accurately reconstruct the geometric shape and spatial structure of target objects, playing a core role in tasks such as digital modeling, defect detection, and deformation monitoring, and is mainly applied in power grid systems.

[0003] In existing technologies, point cloud data processing mainly relies on uniformly configured algorithm models, such as point cloud registration methods based on traditional ICP or deep neural networks. These methods typically assume a high degree of consistency between all scenarios and user needs, ignoring the behavioral differences among different users and the personalized characteristics of service call patterns during point cloud model usage. This leads to unstable matching accuracy between point clouds and digital twin models, difficulty in optimizing processing performance, and an inability to meet the personalized service requirements of large-scale distributed application environments. On the other hand, with the rise of edge computing and distributed intelligent terminals, more and more point cloud processing tasks are being completed at the edge. How to fully utilize data and feedback information from multiple terminals and scenarios to jointly train and optimize point cloud processing models while protecting user privacy has become a pressing problem to be solved. Furthermore, most existing point cloud processing strategies are statically configured, lacking autonomous learning and adaptive optimization capabilities. They cannot dynamically adjust processing strategies based on model complexity, user preferences, and real-time scenarios, limiting the intelligence and robustness of digital twin systems.

[0004] Therefore, there is an urgent need for a novel point cloud processing and twin matching optimization method that integrates user behavior profiling analysis, federated learning, and reinforcement learning mechanisms to achieve continuous learning and dynamic optimization of point cloud algorithms and improve the personalized intelligent service capabilities of digital twin systems in multi-user environments. Summary of the Invention

[0005] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0006] In view of the problems of the above-mentioned or existing point cloud digital service systems and methods based on digital twins, the present invention is proposed.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a point cloud digital service system based on digital twins, including: an entity model construction module, used to synchronously construct a digital twin based on a 3D CAD model and physical parameters, and to fuse and calibrate it with point cloud data to realize the construction of the digital twin model; The fusion update module is used to collect multi-source sensing data and update the point cloud model in the digital twin in real time through a multimodal data fusion algorithm, including capturing local geometric changes and identifying fine topological changes. The semantic segmentation and annotation module is used to perform semantic segmentation on point cloud model data. It extracts structural and functional features based on deep learning networks, intelligently annotates defects, and automatically matches them to the corresponding entities in the digital twin model. The digital interface building module is used to merge point cloud model data with the corresponding digital twin model, and encapsulate it into application-oriented digital service resources based on a microservice architecture, and build a unified standardized application programming interface. The feedback optimization module is used to construct a user behavior profile of the point cloud model based on user interaction data and service call logs, and to optimize the point cloud processing algorithm and the twin model matching strategy by combining federated learning and reinforcement learning.

[0008] As a preferred embodiment of the point cloud digital service system based on digital twins described in this invention, the construction of the digital twin model includes: synchronously constructing a digital twin based on a 3D CAD model and physical parameters, and fusing and calibrating it with point cloud data; selecting a 3D design model of the target physical entity, importing it into the digital twin platform to construct a basic geometry, adding an engineering attribute set to the geometry model to form a complete initial twin of the entity; aligning the point cloud data with the CAD model, and using a rigid registration algorithm to solve for the optimal rotation matrix and translation vector. Define the error distribution function between the point cloud and the CAD model: Where, q i Let R be the point cloud data points, t be the rotation matrix, and p be the translation vector. c (i) represents the corresponding CAD model points, and δ represents the point cloud error; Calculate the mean, standard deviation, and maximum offset of all errors, and for errors exceeding the threshold δ... th In local regions, local geometric compensation or model updates are performed to form the fused twin model T. fused : T fused =f(T init ,P clean ,Talign ,δ) Among them, T init For the initial digital twin model, P clean For the cleaned point cloud data, T align The transformation matrix is ​​used for registration of the point cloud.

[0009] As a preferred embodiment of the point cloud digital service system based on digital twins described in this invention, the point cloud data is aligned with the CAD model, and a rigid registration algorithm is used to solve for the optimal rotation matrix and translation vector, including: Calculate the geometric center of the point cloud data and the point set of the CAD model, spatially align the point cloud data to the CAD model, and solve for a set of rigid transformation parameters that minimize the following error terms: Where k is the number of matching point pairs; The optimal solution is obtained by constructing the covariance matrix and using singular value decomposition. After calculating the rotation matrix R, the translation vector is calculated according to the following formula: in, For the centroid of the point cloud data, Let be the centroid of the point set of the CAD model; Each point in the point cloud dataset is processed according to the calculated rotation and translation transformations to obtain aligned point cloud coordinates. After processing, the point cloud data will be spatially fitted to the CAD model, completing the rigid registration process.

[0010] As a preferred embodiment of the point cloud digital service system based on digital twins described in this invention, the point cloud model in the digital twin is updated in real time through a multimodal data fusion algorithm, including local geometric change capture and fine topological change recognition, including: For data of different modalities, point cloud geometric features, image semantic features, and state field change features are respectively used. A graph neural network is used to fuse and represent the features of different modalities to construct a cross-modal enhanced representation vector. By using a sliding time window mechanism, the fusion feature distribution of continuous time t and t-Δt is compared, the change response of each spatial region is calculated, a change response threshold θ is set, local regions that meet the conditions are screened, and candidate change regions are sent to the geometric update module as update priority regions. Within the changing region, the point cloud data at the current moment is extracted and compared with the corresponding region in the original twin model. The local difference alignment algorithm is used to identify geometric offsets and topological changes. If a topological change is detected, the local mesh topology map is updated.

[0011] As a preferred embodiment of the point cloud digital service system based on digital twins described in this invention, the system includes: semantic segmentation of point cloud model data, extraction of structural and functional features based on deep learning networks, intelligent labeling of defects, and automatic matching to corresponding entities in the digital twin model, comprising: A deep learning network model based on the native structure of point cloud is adopted. After inputting point cloud blocks, it outputs the semantic label of each point and the structural functional feature representation vector. In the semantic segmentation results, a defect detection network module is further introduced to identify defects. The defect region is output in the form of a segmentation mask with attached attribute information, and the defect region is semantically labeled and assigned a unique identifier ID. j By comparing structural and functional features with the entity feature database in the twin model, a mapping relationship between defect areas and digital twin entities is established. Defect annotation data is bound to the matched twin model entities, and the status labels and maintenance records of the twin are updated.

[0012] As a preferred embodiment of the point cloud digital service system based on digital twins according to the present invention, the fusion of point cloud model data with the corresponding digital twin model includes: An initial digital twin model corresponding to the physical object is constructed. The initial model is generated based on CAD design drawings, BIM data or 3D reconstruction results. The point cloud data and the initial twin model are spatially aligned using a 3D registration method so that the point cloud and the model overlap in the same coordinate system. Geometric registration is achieved through the correspondence relationship. Analyze the registration error and the difference region, repair the newly added or missing regions in the point cloud into the twin model, and complete the construction of the fusion model; the fusion model should retain the semantic entity structure and introduce new observation data features; perform topological consistency check and semantic entity matching on the fused twin.

[0013] As a preferred embodiment of the point cloud digital service system based on digital twins described in this invention, the following steps are taken: A user behavior profile of the point cloud model is constructed based on user interaction data and service call logs; and the point cloud processing algorithm and the twin model matching strategy are optimized through a combination of federated learning and reinforcement learning, including: A behavior matrix is ​​constructed by summarizing all user operations on each point cloud model. A clustering algorithm is then used to cluster the behavior matrix to obtain a set of typical user behavior patterns. Each terminal trains a point cloud processing model based on local user data. Among them, f θ For point cloud processing models, y i Let L be the training labels, and D be the loss function. i For user local data θi For local model parameters; Reinforcement learning environment state S t This includes current point cloud features, twin model structural features, and user behavior profile vectors; action space A t This includes matching algorithm selection, parameter configuration, and setting the trade-off between matching accuracy and performance; Reward value R t Based on the following indicators: R t =α⋅Accuracy Improvement -β⋅Time Consumption +γ⋅User Satisfaction Where α, β, and γ are adjustment factors; User behavior profiles are used as state inputs to participate in federated reinforcement training; after the server aggregates the policy network parameters, they are sent to the terminal for execution; the terminal further fine-tunes the policy network based on local user features.

[0014] A point cloud digital service method based on digital twins includes: synchronously constructing a digital twin based on a 3D CAD model and physical parameters, and fusing and calibrating it with point cloud data to realize the construction of the digital twin model; Collect multi-source sensing data and use multi-modal data fusion algorithms to update the point cloud model in the digital twin in real time, including capturing local geometric changes and identifying fine topological changes; Semantic segmentation is performed on point cloud model data, structural and functional features are extracted based on deep learning networks, defects are intelligently labeled, and automatically matched to the corresponding entities in the digital twin model; The point cloud model data is fused with the corresponding digital twin model, and then encapsulated into application-oriented digital service resources based on a microservice architecture, and a unified standardized application programming interface is built. Based on user interaction data and service call logs, a user behavior profile of the point cloud model is constructed, and the point cloud processing algorithm and the twin model matching strategy are optimized by combining federated learning and reinforcement learning.

[0015] A computing device, the computing device comprising: At least one processor, memory, and input / output unit; The memory is used to store computer programs, and the processor is used to call the computer programs stored in the memory to execute the steps of the point cloud digital service system based on digital twins.

[0016] A computer-readable storage medium includes instructions that, when executed on a computer, cause the computer to perform steps of a point cloud digital service system based on a digital twin.

[0017] The beneficial effects of this invention are as follows: By constructing user interaction behavior profiles and service call feature vectors, this invention achieves accurate modeling of different user preferences and model operation modes, enabling point cloud processing algorithms to be differentiated according to individual needs, significantly improving processing accuracy and user experience. By introducing a federated learning mechanism, each edge device completes the training and parameter updates of the point cloud processing model locally, uploading only the model weight differences without involving the original user data, thus achieving cross-terminal joint optimization while ensuring privacy and security. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of a point cloud digital service system based on digital twins, provided as an embodiment of the present invention.

[0020] Figure 2 A flowchart of a point cloud digital service method based on digital twins is provided for an embodiment of the present invention.

[0021] Figure 3 A schematic diagram of the structure of a medium according to an embodiment of the present invention is shown.

[0022] Figure 4 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown.

[0023] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0027] Example The following is for reference. Figures 1-4 , Figure 2 This is a flowchart illustrating a point cloud digital service method based on digital twins, as provided in one embodiment of the present invention. It should be noted that the embodiments of the present invention can be applied to any applicable scenario.

[0028] Figure 2 The flowchart of a point cloud digital service method based on digital twins provided in an embodiment of the present invention includes: S1: Construct a digital twin based on the 3D CAD model and physical parameters, and fuse and calibrate it with point cloud data to realize the construction of the digital twin model.

[0029] Preferably, a 3D design model of the target physical entity is selected, imported into a digital twin platform to construct a basic geometry, an engineering attribute set is added to the geometric model to form a complete initial twin of the entity; the point cloud data is aligned with the CAD model, and a rigid registration algorithm is used to solve for the optimal rotation matrix and translation vector; Define the error distribution function between the point cloud and the CAD model: Where, q i Let R be the point cloud data points, t be the rotation matrix, and p be the translation vector. c (i) represents the corresponding CAD model points, and δ represents the point cloud error; Calculate the mean, standard deviation, and maximum offset of all errors, and for errors exceeding the threshold δ... th In local regions, local geometric compensation or model updates are performed to form the fused twin model T. fused : T fused =f(T init ,P clean ,T align ,δ) Among them, T init For the initial digital twin model, P clean For the cleaned point cloud data, T align The transformation matrix is ​​used for registration of the point cloud.

[0030] Preferably, the geometric center of the point cloud data and the point set of the CAD model is calculated, the point cloud data is spatially aligned to the CAD model, and a set of rigid transformation parameters is solved to minimize the following error terms: Where k is the number of matching point pairs; The optimal solution is obtained by constructing the covariance matrix and using singular value decomposition. After calculating the rotation matrix R, the translation vector is calculated according to the following formula: in, For the centroid of the point cloud data, Let be the centroid of the point set of the CAD model; Each point in the point cloud dataset is processed according to the calculated rotation and translation transformations to obtain aligned point cloud coordinates. After processing, the point cloud data will be spatially fitted to the CAD model, completing the rigid registration process.

[0031] Furthermore, import the 3D CAD model T of the support component. init The model measures 1.5m × 0.4m × 0.5m; point cloud data P was acquired using a 3D laser scanner. raw A total of 84,392 points were obtained. Voxel filtering and statistical denoising were performed on the original point cloud to obtain the cleaned point cloud P. clean Approximately 68,050 points; point cloud data coordinate system: world coordinate system XYZ. Enhanced ICP algorithm was used for registration, iterating until the error convergence threshold was reached. The registration result is a rotation matrix: Translation vector: .

[0032] Furthermore, 1000 key area point pairs were selected, and the error statistics are as follows: average error μ δ The value is 4.7 mm, and the standard deviation σ is 4.7 mm. δ It is 2.3mm, with a maximum deviation δ max It is 11.4mm; Set the dynamic error threshold: δ th =μ δ +1.5⋅σ δ =4.7 + 1.5 ⋅ 2.3 = 8.15 mm In the registered model, extract all models that satisfy δ(q) iFor points with an error margin greater than 8.15mm, the results are as follows: Number of abnormal points: 6822 (accounting for 10.02% of the total points); the abnormalities are mainly concentrated at the root of the bracket and the upper flange, which are determined to be offset and welding deformation during actual installation. The offset in the root area is in the negative direction, and the system performs a 2.5mm topological shrinkage on the bottom contour line of the CAD model; the error in the flange mismatch area is concentrated at 8.5~11.4mm, and the system automatically selects point cloud points to fit the local surface, reconstructs a replacement surface based on NURBS surface, and replaces the original planar structure of the CAD model.

[0033] Furthermore, laser scanning equipment was used to conduct on-site measurements of the support structure, obtaining the original point cloud dataset Q={q1,q2,...,q k The point set P = {p1, p2, ..., p} is approximately 120,000 points, in millimeters. It is exported from pipeline design software as a CAD model point set. k The number of points is 120,000, and each point corresponds to a point cloud.

[0034] Calculate the geometric center of the point cloud dataset: Calculate the geometric center of the point set in the CAD model: The point set is shifted to the origin, i.e., each point is subtracted from its corresponding centroid, to obtain a centered dataset.

[0035] Calculate the covariance matrix H, perform singular value decomposition on matrix H, and further calculate the translation vector: The rotation matrix R and translation vector t are applied to all point cloud points to calculate the alignment error for each pair of points. The statistical error characteristics are: average error: 4.2 mm; maximum error: 10.8 mm; standard deviation: 2.6 mm. Set threshold δ th =μ+2σ=9.4mm, local compensation is performed on the model in the region exceeding the threshold: the root offset of the CAD model is obvious, the system fits the point cloud surface and corrects the model shape; the local mismatch area is marked as "suspicious deviation area" for subsequent maintenance annotation.

[0036] S2: Collect multi-source sensing data and update the point cloud model in the digital twin in real time through a multi-modal data fusion algorithm, including capturing local geometric changes and identifying fine topological changes.

[0037] Preferably, for data of different modalities, point cloud geometric features, image semantic features and state field change features are respectively used to fuse and represent the features of different modalities using graph neural networks, and construct cross-modal enhanced representation vectors; By using a sliding time window mechanism, the fusion feature distribution of continuous time t and t-Δt is compared, the change response of each spatial region is calculated, a change response threshold θ is set, local regions that meet the conditions are screened, and candidate change regions are sent to the geometric update module as update priority regions. Within the changing region, the point cloud data at the current moment is extracted and compared with the corresponding region in the original twin model. The local difference alignment algorithm is used to identify geometric offsets and topological changes. If a topological change is detected, the local mesh topology map is updated.

[0038] Furthermore, the PointNet++ network is used to extract the geometric features of local point cloud regions, outputting a 128-dimensional representation vector; ResNet-50+ DeepLab semantic segmentation is used to extract entity categories such as brackets, bolts, and cables in the image and their spatial distribution; the image segmentation results are embedded into a region-level semantic tensor (64-dimensional). LSTM encoding is performed on the time-series features of sensor nodes, such as stress values, temperature, and voltage, to extract the changing trends and embed them into a 32-dimensional state field feature vector. A heterogeneous graph neural network is constructed to interconnect the three modal feature nodes, and various features are aggregated through a graph attention mechanism; the output is a cross-modal enhanced representation vector ht∈R. 256 The geometric-semantic-state field of the covered structure is integrated. The sliding time window size is set to Δt = 15 min, and the enhanced representation vector h at time t is compared with that at t-Δt. t with h t-Δt For each spatial region r j Calculate the change response of its fused representation: Where, ρ j For region r j The amplitude of the multimodal response variation; Set the response threshold θ = 0.25, if ρ j If the response is ≥θ, then the region is considered to have a significant structural change and is marked as a candidate change region; a total of 3 regions with responses exceeding the threshold were identified in this cycle, located at the front right end and top of the track support; Local point cloud geometric comparison: Extract point cloud data of the response area at the current time; extract the corresponding region point set from the twin model; use the local difference algorithm for micro-alignment and calculate the residual field; a 6.4mm mean offset was detected at the top, indicating significant structural curvature. Compare the point cloud connectivity in the region to determine if there are breaks or new components on the surface; the system automatically reconstructs the local triangular mesh and inserts the mesh structure of that region into the twin, while recording the change log.

[0039] S3: Perform semantic segmentation on point cloud model data, extract structural and functional features based on deep learning networks, intelligently label defects, and automatically match them to the corresponding entities in the digital twin model.

[0040] Preferably, a deep learning network model based on the native structure of point cloud is adopted, which outputs the semantic label of each point after inputting point cloud blocks, and outputs the structural functional feature representation vector at the same time; In the semantic segmentation results, a defect detection network module is further introduced to identify defects. The defect region is output in the form of a segmentation mask with attached attribute information, and the defect region is semantically labeled and assigned a unique identifier ID. j By comparing structural and functional features with the entity feature database in the twin model, a mapping relationship between defect areas and digital twin entities is established. Defect annotation data is bound to the matched twin model entities, and the status labels and maintenance records of the twin are updated.

[0041] Furthermore, a handheld laser scanning device was used to acquire 3D point cloud data of the current substation area, totaling approximately 150,000 points. The point cloud was divided into several sub-blocks, each measuring 1m × 1m × 1m and containing approximately 2048 points. An improved network based on Point Transformer was employed; for each point q... i Output: Semantic category label c i ∈{bracket, bolt, door panel, cable…}; Point-level structural function representation vector f i ∈R 128 It encodes local geometry, structure, and contextual information. For each structural category, its representation vector is aggregated to obtain the global structural feature representation of each entity class. Based on the semantic segmentation results, the input point cloud blocks are fed into a dedicated defect detection subnetwork; the output is a mask region Mj={q} of the defect region. j1 ,q j2 ... and attribute information; each defect region is assigned a unique identifier ID. The structure of each defect region is represented by F... defect Compare with the entity database in the twin model; calculate the similarity score: sim(F defect ,F entityk )=cos(F defect ,F entityk ) Among them, F defect F is the functional feature vector of the defect structure. entityk Let be the feature vector of the k-th twin entity; If the similarity is greater than the threshold (e.g., 0.85), then the match is successful.

[0042] Logically bind the defect ID to the twin entity; in the twin platform, the entity's status label is automatically updated to "Abnormal - Crack," and corresponding maintenance suggestions and repair records are generated; maintenance logs are synchronized. { "entity": "Cabinet_Door_RB", "status": "CRACK_DETECTED", "linked_defect": "CRACK_001", "update_time": "2025-06-19 10:23:45" } Defective areas are highlighted in red in the 3D twin platform, displaying their ID, attributes, and update time; It supports linkage with structural heatmaps to display the evolution trend of defects at multiple times; it connects to the maintenance scheduling system to automatically generate maintenance tasks based on the physical status; and it integrates with the operation and maintenance knowledge base to provide standard handling procedures and historical case support.

[0043] S4: Merge point cloud model data with the corresponding digital twin model, encapsulate it into application-oriented digital service resources based on a microservice architecture, and build a unified, standardized application programming interface.

[0044] Preferably, an initial digital twin model corresponding to the physical object is constructed. The initial model is generated based on CAD design drawings, BIM data or 3D reconstruction results. A 3D registration method is used to spatially align the point cloud data with the initial twin model so that the point cloud and the model overlap in the same coordinate system. Geometric registration is achieved through the correspondence relationship. Analyze the registration error and the difference region, repair the newly added or missing regions in the point cloud into the twin model, and complete the construction of the fusion model; the fusion model should retain the semantic entity structure and introduce new observation data features; perform topological consistency check and semantic entity matching on the fused twin.

[0045] Furthermore, import the CAD drawings of the utility tunnel and read the IFC format data through the BIM platform to extract building components, component IDs, attribute sets, etc. Alternatively, reconstruct the model using photogrammetry or a 3D reconstruction system. Convert the CAD / BIM model to a common 3D mesh format (such as OBJ, PLY); standardize the naming conventions and semantic tags for components; and store it as an initial digital twin T. init It includes: geometric model; semantic structure tree; entity unique identifier; attribute set.

[0046] A panoramic point cloud P of the cable tunnel area was acquired using a ground-based laser scanning device; the point cloud was then denoised, downsampled, and its normals estimated to obtain a cleaned point set P. clean The process employs coarse registration (e.g., FPFH feature matching) + fine registration (ICP or GICP): After initial alignment, rigid registration is performed, calculating the rotation matrix R and translation vector t; the point cloud and the twin model are overlapped in a unified coordinate system; all registration error values ​​are statistically analyzed, and the mean, maximum, and standard deviation are calculated; a threshold δ is set. th Significant error identification areas: If there are areas in the point cloud that are not covered by the model, they are identified as newly added structures; if there are areas in the model that are not covered by the corresponding point cloud, they are identified as missing or altered structures.

[0047] Furthermore, the following consistency rules are applied to the fusion model: whether there are broken links or overlaps in the connections between all semantic entities; whether the spatial arrangement conforms to the design hierarchy; and if a new entity intersects with an existing entity, manual verification is prompted. For each fusion region, its structural functional features F are extracted. j Compare with the entity database in the twin platform; if the similarity sim(F) j ,F k If the value is greater than 0.9, then a binding is established with entity_k; Update the entity's attribute status to "Observed + Timestamp" to maintain historical synchronization records.

[0048] S5: Based on user interaction data and service call logs, construct a user behavior profile of the point cloud model, and optimize the point cloud processing algorithm and the twin model matching strategy by combining federated learning and reinforcement learning.

[0049] Preferably, a behavior matrix is ​​constructed by summarizing all user operations on each point cloud model, and a clustering algorithm is used to cluster the behavior matrix to obtain a set of typical user behavior patterns; each terminal trains a point cloud processing model based on local user data: Among them, f θ For point cloud processing models, y i Let L be the training labels, and D be the loss function. i For user local data θ i For local model parameters; Reinforcement learning environment state S t This includes current point cloud features, twin model structural features, and user behavior profile vectors; action space A t This includes matching algorithm selection, parameter configuration, and setting the trade-off between matching accuracy and performance; Reward value R t Based on the following indicators: Rt =α⋅Accuracy Improvement -β⋅Time Consumption +γ⋅User Satisfaction Where α, β, and γ are adjustment factors; User behavior profiles are used as state inputs to participate in federated reinforcement training; after the server aggregates the policy network parameters, they are sent to the terminal for execution; the terminal further fine-tunes the policy network based on local user features.

[0050] Furthermore, assuming the platform deploys 5 operation and maintenance sites, each site's operation and maintenance terminal records user behavior log data when using point cloud tools (such as point cloud registration, defect identification, etc.), with the following structure: User ID Operation type Algorithm selection Parameter settings Average time elapsed (s) Precision feedback Intervention frequency U001 Registration ICP ε=0.01 4.2 0.97 1 U002 Defect labeling PointNet++ radius=0.5 6.3 0.89 3 U003 Measurement RANSAC inlier=0.7 2.5 0.89 0 The above behavioral structures are encoded into vectors (dimension d=6) to form a user behavior matrix. K-means clustering is used to extract two typical user behavior profile patterns: Type A (accuracy-first): average feedback accuracy ≥ 0.95, fine parameter settings, and minimal intervention; Type B (speed-first): average time < 3.5s, with moderate compromise on accuracy.

[0051] Assuming the strategy achieves the following results: accuracy improvement: +4% (mIoU from 0.87 to 0.91), execution time: +1.2s, user satisfaction rating: 4.7 / 5, and setting α=100, β=5, γ=50, then: R t =100⋅0.04-5⋅1.2+50⋅0.94=4-6+47=45 Strategies that receive positive rewards are prioritized for retention.

[0052] After the federal policy parameters are aggregated, a unified policy network is formed; it is pushed to each terminal (such as U001, U002); U001 further fine-tunes the policy locally to adapt to its behavior profile; the adjusted policy selects FGR+ε=0.01 for it, giving priority to the accuracy-first path.

[0053] Test task: Substation cloud update task, test period 1 week: User ID Federal Precision Federal Post-Precision Time reduced (%) Satisfaction Improvement U001 0.87 0.91 -5.4% +1.1 points U002 0.82 0.88 -15.6% +0.7 points U003 0.84 0.89 -10.1% +0.9 points The overall system mIoU improved by 6.7%, the average latency decreased by 9.5%, and user satisfaction improved significantly.

[0054] After introducing the method of exemplary embodiments of the present invention, the following references are made. Figure 1 An exemplary embodiment of the present invention provides a point cloud digital service system based on digital twins, the system comprising: The entity model building module is used to synchronously build a digital twin based on a 3D CAD model and physical parameters, and to fuse and calibrate it with point cloud data to realize the construction of the digital twin model; The fusion update module is used to collect multi-source sensing data and update the point cloud model in the digital twin in real time through a multimodal data fusion algorithm, including capturing local geometric changes and identifying fine topological changes. The semantic segmentation and annotation module is used to perform semantic segmentation on point cloud model data. It extracts structural and functional features based on deep learning networks, intelligently annotates defects, and automatically matches them to the corresponding entities in the digital twin model. The digital interface building module is used to merge point cloud model data with the corresponding digital twin model, and encapsulate it into application-oriented digital service resources based on a microservice architecture, and build a unified standardized application programming interface. The feedback optimization module is used to construct a user behavior profile of the point cloud model based on user interaction data and service call logs, and to optimize the point cloud processing algorithm and the twin model matching strategy by combining federated learning and reinforcement learning.

[0055] After introducing the methods and systems of exemplary embodiments of the present invention, the following references are made. Figure 3 A computer-readable storage medium according to exemplary embodiments of the present invention will be described, please refer to... Figure 3 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it implements the steps described in the above method implementation, such as: synchronously constructing a digital twin based on a 3D CAD model and physical parameters, and fusing and calibrating it with point cloud data to realize the construction of the digital twin model; collecting multi-source sensing data, and updating the point cloud model in the digital twin in real time through a multi-modal data fusion algorithm, including capturing local geometric changes and identifying fine topological changes; performing semantic segmentation on the point cloud model data, extracting structural and functional features based on a deep learning network, intelligently labeling defects, and automatically matching them to the corresponding entities in the digital twin model; fusing the point cloud model data with the corresponding digital twin model, and encapsulating it into an application-oriented digital service resource based on a microservice architecture, and constructing a unified standardized application programming interface; constructing a user behavior profile of the point cloud model based on user interaction data and service call logs, and optimizing the point cloud processing algorithm and the twin model matching strategy through a combination of federated learning and reinforcement learning; the specific implementation methods of each step will not be repeated here.

[0056] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0057] After introducing the methods, apparatus, and media of exemplary embodiments of the present invention, the following references are made. Figure 4 A computing device for point cloud digital services based on digital twins, according to an exemplary embodiment of the present invention.

[0058] Figure 4 A block diagram is shown of an exemplary computing device 40 suitable for implementing embodiments of the present invention. The computing device 40 may be a computer system or a server. Figure 4 The computing device 40 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0059] like Figure 4 As shown, the components of computing device 40 may include, but are not limited to: one or more processors or processing units 401, system memory 402, and bus 403 connecting different system components (including system memory 402 and processing unit 401).

[0060] The computing device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computing device 40, including volatile and non-volatile media, and removable and non-removable media.

[0061] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 4021 and / or cache memory 4022. Computing device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, ROM 4023 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 4 (Not shown in the image, usually referred to as "hard drive"). Although not shown in... Figure 4The diagram illustrates that disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., CD-ROMs, DVD-ROMs, or other optical media) can be provided. In these cases, each drive can be connected to bus 403 via one or more data media interfaces. System memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0062] A program / utility 4025 having a set (at least one) of program modules 4024 may be stored, for example, in system memory 402, and such program modules 4024 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment. Program modules 4024 typically perform the functions and / or methods described in the embodiments of the present invention.

[0063] The computing device 40 can also communicate with one or more external devices 404 (such as a keyboard, pointing device, display, etc.). This communication can be performed via the input / output (I / O) interface 405. Furthermore, the computing device 40 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 406. Figure 4 As shown, network adapter 406 communicates with other modules of computing device 40 (such as processing unit 401) via bus 403. It should be understood that, although... Figure 4 As not shown, it can be used in conjunction with computing device 40 with other hardware and / or software modules.

[0064] The processing unit 401 executes various functional applications and data processing by running programs stored in the system memory 402. For example, it synchronously constructs a digital twin based on a 3D CAD model and physical parameters, and fuses and calibrates it with point cloud data to achieve the construction of the digital twin model; it collects multi-source sensing data and updates the point cloud model in the digital twin in real time using a multi-modal data fusion algorithm, including capturing local geometric changes and identifying fine topological changes; it performs semantic segmentation on the point cloud model data, extracts structural and functional features based on deep learning networks, intelligently labels defects, and automatically matches them to the corresponding entities in the digital twin model; it fuses the point cloud model data with the corresponding digital twin model, encapsulates it into application-oriented digital service resources based on a microservice architecture, and constructs a unified standardized application programming interface; it constructs a user behavior profile of the point cloud model based on user interaction data and service call logs, and optimizes the point cloud processing algorithm and twin model matching strategy through a combination of federated learning and reinforcement learning. The specific implementation methods of each step will not be repeated here.

[0065] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0066] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0067] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0068] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0069] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0070] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0071] Furthermore, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

Claims

1. A point cloud digital service system based on digital twins, characterized in that, include: The entity model building module is used to synchronously build a digital twin based on a 3D CAD model and physical parameters, and to fuse and calibrate it with point cloud data to realize the construction of the digital twin model; The fusion update module is used to collect multi-source sensing data and update the point cloud model in the digital twin in real time through a multimodal data fusion algorithm, including capturing local geometric changes and identifying fine topological changes. The semantic segmentation and annotation module is used to perform semantic segmentation on point cloud model data. It extracts structural and functional features based on deep learning networks, intelligently annotates defects, and automatically matches them to the corresponding entities in the digital twin model. The digital interface building module is used to merge point cloud model data with the corresponding digital twin model, and encapsulate it into application-oriented digital service resources based on a microservice architecture, and build a unified standardized application programming interface. The feedback optimization module is used to construct a user behavior profile of the point cloud model based on user interaction data and service call logs, and to optimize the point cloud processing algorithm and the twin model matching strategy by combining federated learning and reinforcement learning.

2. The point cloud digital service system based on digital twins as described in claim 1, characterized in that, The construction of a digital twin based on the synchronous construction of a 3D CAD model and physical parameters, and the fusion and calibration with point cloud data, to realize the construction of a digital twin model includes: selecting a 3D design model of the target physical entity, importing it into the digital twin platform to construct a basic geometry, adding an engineering attribute set to the geometry model to form a complete initial twin of the entity; aligning the point cloud data with the CAD model, and using a rigid registration algorithm to solve for the optimal rotation matrix and translation vector. Define the error distribution function between the point cloud and the CAD model: ； Where, q i Let R be the point cloud data points, t be the rotation matrix, and p be the translation vector. c (i) represents the corresponding CAD model points, and δ represents the point cloud error; Calculate the mean, standard deviation, and maximum offset of all errors, and for errors exceeding the threshold δ... th In local regions, local geometric compensation or model updates are performed to form the fused twin model T. fused : T fused =f(T init ,P clean ,T align ,δ); Among them, T init For the initial digital twin model, P clean For the cleaned point cloud data, T align The transformation matrix is ​​used for registration of the point cloud.

3. The point cloud digital service system based on digital twins as described in claim 2, characterized in that, The process of aligning point cloud data with the CAD model, using a rigid registration algorithm to solve for the optimal rotation matrix and translation vector, includes: Calculate the geometric center of the point cloud data and the point set of the CAD model, spatially align the point cloud data to the CAD model, and solve for a set of rigid transformation parameters that minimize the following error terms: ； Where k is the number of matching point pairs; The optimal solution is obtained by constructing the covariance matrix and using singular value decomposition. After calculating the rotation matrix R, the translation vector is calculated according to the following formula: ； in, For the centroid of the point cloud data, Let be the centroid of the point set of the CAD model; Each point in the point cloud dataset is processed according to the calculated rotation and translation transformations to obtain aligned point cloud coordinates. After processing, the point cloud data will be spatially fitted to the CAD model, completing the rigid registration process.

4. The point cloud digital service system based on digital twins as described in claim 1, characterized in that, The process involves real-time updating of the point cloud model in the digital twin using a multimodal data fusion algorithm, including local geometric change capture and fine-grained topological change recognition, including: For data of different modalities, point cloud geometric features, image semantic features, and state field change features are respectively used. A graph neural network is used to fuse and represent the features of different modalities to construct a cross-modal enhanced representation vector. By using a sliding time window mechanism, the fusion feature distribution of continuous time t and t-Δt is compared, the change response of each spatial region is calculated, a change response threshold θ is set, local regions that meet the conditions are screened, and candidate change regions are sent to the geometric update module as update priority regions. Within the changing region, the point cloud data at the current moment is extracted and compared with the corresponding region in the original twin model. The local difference alignment algorithm is used to identify geometric offsets and topological changes. If a topological change is detected, the local mesh topology map is updated.

5. The point cloud digital service system based on digital twins as described in claim 1, characterized in that, The process of semantic segmentation of point cloud model data, extraction of structural and functional features based on deep learning networks, intelligent labeling of defects, and automatic matching to corresponding entities in the digital twin model includes: A deep learning network model based on the native structure of point cloud is adopted. After inputting point cloud blocks, it outputs the semantic label of each point and the structural functional feature representation vector. In the semantic segmentation results, a defect detection network module is further introduced to identify defects. The defect region is output in the form of a segmentation mask with attached attribute information, and the defect region is semantically labeled and assigned a unique identifier ID. j By comparing structural and functional features with the entity feature database in the twin model, a mapping relationship between defect areas and digital twin entities is established. Defect annotation data is bound to the matched twin model entities, and the status labels and maintenance records of the twin are updated.

6. The point cloud digital service system based on digital twin as described in claim 1, characterized in that, The process of fusing point cloud model data with the corresponding digital twin model includes: An initial digital twin model corresponding to the physical object is constructed. The initial model is generated based on CAD design drawings, BIM data or 3D reconstruction results. The point cloud data and the initial twin model are spatially aligned using a 3D registration method so that the point cloud and the model overlap in the same coordinate system. Geometric registration is achieved through the correspondence relationship. Analyze the registration error and the difference region, repair the newly added or missing regions in the point cloud into the twin model, and complete the construction of the fusion model; the fusion model should retain the semantic entity structure and introduce new observation data features; perform topological consistency check and semantic entity matching on the fused twin.

7. The point cloud digital service system based on digital twins as described in claim 1, characterized in that, The process of constructing a user behavior profile of the point cloud model based on user interaction data and service call logs, and optimizing the point cloud processing algorithm and the twin model matching strategy through a combination of federated learning and reinforcement learning, includes: A behavior matrix is ​​constructed by summarizing all user operations on each point cloud model. A clustering algorithm is then used to cluster the behavior matrix to obtain a set of typical user behavior patterns. Each terminal trains a point cloud processing model based on local user data. ； Among them, f θ For point cloud processing models, y i Let L be the training labels, and D be the loss function. i For user local data θ i For local model parameters; Reinforcement learning environment state S t This includes current point cloud features, twin model structural features, and user behavior profile vectors; action space A t This includes matching algorithm selection, parameter configuration, and setting the trade-off between matching accuracy and performance; Reward value R t Based on the following indicators: R t =α⋅Accuracy Improvement -β⋅Time Consumption +γ⋅User Satisfaction Where α, β, and γ are adjustment factors; User behavior profiles are used as state inputs to participate in federated reinforcement training; after the server aggregates the policy network parameters, they are sent to the terminal for execution; the terminal further fine-tunes the policy network based on local user features.

8. A point cloud digital service method based on digital twins, characterized in that, include: A digital twin is constructed by synchronously building a 3D CAD model and physical parameters, and then fused and calibrated with point cloud data to realize the construction of the digital twin model. Collect multi-source sensing data and use multi-modal data fusion algorithms to update the point cloud model in the digital twin in real time, including capturing local geometric changes and identifying fine topological changes; Semantic segmentation is performed on point cloud model data, structural and functional features are extracted based on deep learning networks, defects are intelligently labeled, and automatically matched to the corresponding entities in the digital twin model; The point cloud model data is fused with the corresponding digital twin model, and then encapsulated into application-oriented digital service resources based on a microservice architecture, and a unified standardized application programming interface is built. Based on user interaction data and service call logs, a user behavior profile of the point cloud model is constructed, and the point cloud processing algorithm and the twin model matching strategy are optimized by combining federated learning and reinforcement learning.

9. A computing device, the computing device comprising: At least one processor, memory, and input / output unit; The memory is used to store computer programs, and the processor is used to call the computer programs stored in the memory to execute the steps of the point cloud digital service system based on digital twins as described in any one of claims 1 to 7.

10. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the steps of the point cloud digital service system based on digital twins as described in any one of claims 1 to 7.

Images