Digital twin management system and method for tourist attractions based on the Internet of Things

Through deep learning technology, the local perspective image features of cultural heritage are extracted and three-dimensional models are generated. Combined with digital twin technology, the problem that traditional management methods cannot provide a comprehensive experience is solved, the fine management and protection of cultural heritage is realized, and the immersive viewing is provided. Experience.

CN118691428BActive Publication Date: 2024-12-03SHAANXI TIRAIN TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411164347.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-12-03
Estimated Expiration
2044-08-23

AI Technical Summary

Technical Problem

Traditional tourist attractions management methods have challenges in the protection, display and education of cultural heritage. Excessive tourist flow may lead to physical damage. A single image display cannot provide a comprehensive and multi-angle perceptual experience and cannot fully convey the diverse value of cultural heritage.

Method used

Using computer vision technology based on deep learning to process multiple local perspective images of cultural heritage objects, extract visual features, and generate a three-dimensional model through feature aggregation analysis, combining digital twin technology to display virtual and reality, providing immersive, Multi-dimensional viewing experience and fine management.

Benefits of technology

It realizes the fine management and protection of cultural heritage, provides an immersive experience, enhances the vividness and authenticity of cultural heritage display, provides tourists with a comprehensive and multi-angle understanding, and reduces the risk of physical damage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118691428B_ABST
    Figure CN118691428B_ABST
Patent Text Reader

Abstract

The present application relates to the field of digital twin technology, and specifically discloses a digital twin management system and method for tourist attractions based on the Internet of Things, which uses computer vision technology based on deep learning to perform image processing on multiple local perspective images of cultural heritage objects to extract the visual features of the cultural heritage at each local perspective, and then obtains panoramic information of the cultural heritage objects by performing feature aggregation analysis on the visual features of the cultural heritage at multiple local perspectives, thereby generating a corresponding three-dimensional model of the cultural heritage object, and displaying the model through digital twin technology, which can provide tourists with an immersive, multi-dimensional viewing experience, and provide a platform that combines virtuality and reality to achieve fine management and protection of cultural heritage.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of digital twin technology, and more specifically, to a digital twin management system and method for tourist attractions based on the Internet of Things. Background Art

[0002] Due to the uniqueness and non-renewable nature of cultural heritage, traditional tourist attraction management methods face many challenges in terms of cultural heritage protection, display and education. For example, in the physical display of cultural heritage, excessive tourist traffic may cause physical damage to cultural heritage, while relying solely on a single image display method cannot provide tourists with a full range of multi-angle perceptual experience and cannot fully convey the diverse values ​​of cultural heritage.

[0003] Therefore, in order to meet the viewing needs of tourists and at the same time achieve effective protection and management of cultural heritage, we look forward to a digital twin management system and method for tourist attractions based on the Internet of Things. Summary of the invention

[0004] In order to solve the above technical problems, this application is proposed. The embodiment of this application provides a digital twin management system and method for tourist attractions based on the Internet of Things, which uses computer vision technology based on deep learning to process multiple local perspective images of cultural heritage objects to extract the visual features of cultural heritage at each local perspective, and then perform feature aggregation analysis on the visual features of cultural heritage at multiple local perspectives to obtain panoramic information of cultural heritage objects, thereby generating a corresponding three-dimensional model of the cultural heritage object, and displaying the model through digital twin technology, which can provide tourists with an immersive, multi-dimensional viewing experience, and provide a platform that combines virtuality and reality to achieve fine management and protection of cultural heritage.

[0005] Accordingly, according to one aspect of the present application, a digital twin management system for tourist attractions based on the Internet of Things is provided, which includes:

[0006] A multi-angle image acquisition module for cultural heritage objects, used to acquire multiple local perspective images of the monitored cultural heritage objects collected by the camera from multiple angles;

[0007] A local perspective image feature mining module, used for respectively performing image feature extraction and feature selection enhancement on the plurality of local perspective images to obtain a plurality of enhanced cultural heritage local perspective feature maps;

[0008] A local view feature aggregation module, used for performing feature reverse aggregation on a plurality of the enhanced cultural heritage local view feature maps to obtain a panoramic cultural heritage object representation feature vector;

[0009] A three-dimensional model generation module, used to generate a three-dimensional model of the cultural heritage object based on the panoramic cultural heritage object representation feature vector;

[0010] A digital twin display module is used to display the three-dimensional model of the cultural heritage object through a digital twin module.

[0011] In the above-mentioned digital twin management system for tourist attractions based on the Internet of Things, the local perspective image feature mining module includes: an image feature extraction unit, which is used to input multiple local perspective images into the MBCNet model including a backbone network and a boundary feature extraction branch to obtain multiple cultural heritage local perspective feature maps; a feature selection and enhancement unit, which is used to input multiple cultural heritage local perspective feature maps into the compression-suppression attention module enhanced by feature selection to obtain multiple enhanced cultural heritage local perspective feature maps.

[0012] In the above-mentioned digital twin management system of tourist attractions based on the Internet of Things, the backbone network in the MBCNet model includes multiple convolution layers based on 5x5 convolution kernels, activation layers and pooling layers based on ReLU activation functions; the boundary feature extraction branch in the MBCNet model includes a point convolution layer, an activation layer based on Sigmoid and an upsampling layer.

[0013] In the above-mentioned digital twin management system of tourist attractions based on the Internet of Things, the feature selection and enhancement unit includes: a global mean pooling subunit, which is used to calculate the global mean of each feature matrix of the local perspective feature map of the cultural heritage along the channel dimension to obtain a cultural heritage local perspective feature compression information representation vector; a channel information association interaction subunit, which is used to perform one-dimensional convolution encoding on the local perspective feature compression representation vector of the cultural heritage to obtain a cultural heritage local perspective feature compression information inter-association representation feature vector; a multi-scale fusion subunit, which is used to cascade the cultural heritage local perspective feature compression information representation vector and the cultural heritage local perspective feature compression information inter-association representation feature vector to obtain a cultural heritage local perspective feature compression information multi-scale representation vector; a multi-scale association feature extraction subunit, which is used to input the cultural heritage local perspective feature compression information multi-scale representation vector into a compression information feature extraction module including two fully connected layers and a SiLU activation function to obtain a cultural heritage local perspective feature compression information multi-scale association feature vector; a feature selection subunit, which is used to perform feature selection on the cultural heritage local perspective feature map based on the cultural heritage local perspective feature compression information multi-scale association feature vector to obtain the enhanced cultural heritage local perspective feature map.

[0014] In the above-mentioned digital twin management system of tourist attractions based on the Internet of Things, the feature selection subunit is used to: use the Sigmoid function to normalize the multi-scale correlation feature vector of the local perspective feature compression information of the cultural heritage to obtain the local perspective weight feature vector of the cultural heritage; use the various eigenvalues ​​of the local perspective weight feature vector of the cultural heritage as weights, and perform feature amplification and suppression operations on each feature channel of the local perspective feature map of the cultural heritage to obtain the enhanced local perspective feature map of the cultural heritage.

[0015] In the above-mentioned digital twin management system of tourist attractions based on the Internet of Things, the local perspective feature aggregation module includes: a feature shape reshaping unit, which is used to reshape the feature shapes of multiple enhanced cultural heritage local perspective feature maps to obtain multiple enhanced cultural heritage local perspective feature vectors; a reverse mask aggregation unit, which is used to input multiple enhanced cultural heritage local perspective feature vectors into a feature distribution reverse mask aggregation network to obtain the panoramic cultural heritage object representation feature vector.

[0016] In the above-mentioned digital twin management system of tourist attractions based on the Internet of Things, the reverse mask aggregation unit includes: a feature clustering subunit, which is used to cluster multiple local perspective feature vectors of the enhanced cultural heritage to obtain a cluster center vector; a semantic correlation coefficient calculation subunit, which is used to calculate the semantic correlation coefficient between each of the multiple local perspective feature vectors of the enhanced cultural heritage and the cluster center vector to obtain a sequence of semantic correlation coefficients; a semantic anti-correlation coefficient calculation subunit, which is used to take the inverse of each semantic correlation coefficient in the sequence of semantic correlation coefficients to obtain a sequence of semantic anti-correlation coefficients; a normalization subunit, which is used to normalize the sequence of semantic anti-correlation coefficients using a Sigmoid function. a reverse weighting subunit, for weighting each of the plurality of enhanced cultural heritage local perspective feature vectors with each semantic anti-correlation weight coefficient in the sequence of semantic anti-correlation weight coefficients as a weight, so as to obtain a plurality of suppressed cultural heritage local perspective feature vectors; a position difference subunit, for calculating the position difference between the plurality of enhanced cultural heritage local perspective feature vectors and the plurality of suppressed cultural heritage local perspective feature vectors, so as to obtain a plurality of optimized cultural heritage local perspective feature vectors; a panoramic aggregation subunit, for calculating the position mean vector of the plurality of optimized cultural heritage local perspective feature vectors, so as to obtain the panoramic cultural heritage object representation feature vector.

[0017] In the above-mentioned digital twin management system for tourist attractions based on the Internet of Things, the semantic correlation coefficient calculation subunit is used to: calculate the vector product between the transposed vector of the local perspective feature vector of the enhanced cultural heritage and the cluster center vector to obtain a semantic correlation factor; calculate the product between the length of the local perspective feature vector of the enhanced cultural heritage and the length of the cluster center vector to obtain a scale correlation factor; divide the semantic correlation factor by the scale correlation factor to obtain the semantic correlation coefficient.

[0018] In the above-mentioned digital twin management system for tourist attractions based on the Internet of Things, the three-dimensional model generation module is used to: input the panoramic cultural heritage object representation feature vector into a three-dimensional model generator based on a diffusion model to obtain the three-dimensional model of the cultural heritage object.

[0019] According to another aspect of the present application, a digital twin management method for tourist attractions based on the Internet of Things is provided, which includes:

[0020] Acquire multiple local perspective images of the monitored cultural heritage object collected by the camera from multiple angles;

[0021] Performing image feature extraction and feature selection enhancement on the plurality of local perspective images respectively to obtain a plurality of enhanced cultural heritage local perspective feature maps;

[0022] Performing feature reverse aggregation on a plurality of the enhanced cultural heritage local perspective feature maps to obtain a panoramic cultural heritage object representation feature vector;

[0023] generating a three-dimensional model of the cultural heritage object based on the panoramic cultural heritage object representation feature vector;

[0024] The three-dimensional model of the cultural heritage object is displayed through the digital twin module.

[0025] Compared with the prior art, the digital twin management system and method for tourist attractions based on the Internet of Things provided by the present application uses computer vision technology based on deep learning to perform image processing on multiple local perspective images of cultural heritage objects to extract the visual features of the cultural heritage at each local perspective, and then obtains panoramic information of the cultural heritage objects by performing feature aggregation analysis on the visual features of the cultural heritage at multiple local perspectives, thereby generating a corresponding three-dimensional model of the cultural heritage object, and displaying the model through digital twin technology, which can provide tourists with an immersive, multi-dimensional viewing experience, and provide a platform that combines virtual and reality to achieve fine management and protection of cultural heritage. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0027] Figure 1 It is a block diagram of a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application.

[0028] Figure 2 This is a schematic diagram of the architecture of a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application.

[0029] Figure 3 It is a block diagram of a local perspective image feature mining module in a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application.

[0030] Figure 4 This is a block diagram of a feature selection and enhancement unit in a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application.

[0031] Figure 5 It is a block diagram of the local perspective feature aggregation module in the digital twin management system of tourist attractions based on the Internet of Things according to an embodiment of the present application.

[0032] Figure 6 It is a block diagram of a reverse mask aggregation unit in a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application.

[0033] Figure 7 This is a flowchart of a digital twin management method for tourist attractions based on the Internet of Things according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] Below, the embodiments of the present application will be described in more detail in conjunction with the accompanying drawings, and the above and other purposes, features and advantages of the present application will become more apparent. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.

[0035] As mentioned in the above background technology, in view of the uniqueness and non-renewability of cultural heritage, traditional tourist attraction management strategies face major challenges in the protection, presentation and education of cultural heritage. For example, excessive influx of tourists may bring risks of damage to cultural heritage, and relying solely on a single image display method will make it difficult to provide tourists with a comprehensive and three-dimensional cognitive experience, and thus fail to fully demonstrate the rich value of cultural heritage. In response to the above technical problems, the technical concept of this application is to use deep learning-based computer vision technology to perform image processing on multiple local perspective images of cultural heritage objects to extract the visual features of cultural heritage from each local perspective, and then perform feature aggregation analysis on the visual features of cultural heritage from multiple local perspectives to obtain panoramic information of cultural heritage objects, thereby generating a corresponding three-dimensional model of the cultural heritage object, and displaying the model through digital twin technology, which can provide tourists with an immersive, multi-dimensional viewing experience, and provide a platform that combines virtuality and reality to achieve fine management and protection of cultural heritage.

[0036] Figure 1 It is a block diagram of a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of the architecture of a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the digital twin management system 100 of tourist attractions based on the Internet of Things includes: a cultural heritage object multi-angle image acquisition module 110, which is used to obtain multiple local perspective images of the monitored cultural heritage object collected by a camera from multiple angles; a local perspective image feature mining module 120, which is used to perform image feature extraction and feature selection enhancement on the multiple local perspective images to obtain multiple enhanced cultural heritage local perspective feature maps; a local perspective feature aggregation module 130, which is used to perform feature reverse aggregation on the multiple enhanced cultural heritage local perspective feature maps to obtain a panoramic cultural heritage object representation feature vector; a three-dimensional model generation module 140, which is used to generate a three-dimensional model of the cultural heritage object based on the panoramic cultural heritage object representation feature vector; a digital twin display module 150, which is used to display the three-dimensional model of the cultural heritage object through a digital twin module.

[0037] In the above-mentioned digital twin management system 100 of tourist attractions based on the Internet of Things, the cultural heritage object multi-angle image acquisition module 110 is used to obtain multiple local perspective images of the monitored cultural heritage object collected by the camera from multiple angles. It should be understood that cultural heritage objects usually have their inherent spatial structure and multi-dimensional characteristics, and two-dimensional images from a single perspective often cannot fully and accurately capture their overall characteristics. Therefore, in the technical solution of the present application, the appearance information of the cultural heritage is fully captured by collecting local images of cultural heritage objects from multiple perspectives, and then based on the collected multiple local perspective images, the three-dimensional structure and shape of the cultural heritage object are inferred using computer vision technology, and the conversion from two-dimensional images to three-dimensional models is realized, thereby improving the vividness and authenticity of the cultural heritage display, and providing an accurate digital basis for the protection and management of cultural heritage.

[0038] In the above-mentioned tourist attraction digital twin management system 100 based on the Internet of Things, the local view image feature mining module 120 is used to extract image features and select and enhance features of the multiple local view images to obtain multiple enhanced cultural heritage local view feature maps. Figure 3 FIG. 1 is a block diagram of a local perspective image feature mining module in a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application. Figure 3 As shown, the local perspective image feature mining module 120 includes: an image feature extraction unit 121, which is used to input the multiple local perspective images into the MBCNet model including the backbone network and the boundary feature extraction branch to obtain multiple cultural heritage local perspective feature maps; a feature selection and enhancement unit 122, which is used to input the multiple cultural heritage local perspective feature maps into the compression-suppression attention module of feature selection enhancement to obtain multiple enhanced cultural heritage local perspective feature maps.

[0039] Specifically, the image feature extraction unit 121 is used to input the multiple local perspective images into the MBCNet model including the backbone network and the boundary feature extraction branch to obtain multiple cultural heritage local perspective feature maps. It should be understood that, considering that in the feature extraction process of the local perspective image of the cultural heritage, as the multi-layer convolution operation goes deeper, some important boundary information may be lost, thereby affecting the fine modeling and detail display of the cultural heritage. Therefore, in the technical solution of the present application, the MBCNet model including the backbone network and the boundary feature extraction branch is used to process each local perspective image separately. Among them, the backbone network is mainly used to capture the global features of the image, while the boundary feature extraction branch is used to specifically extract and strengthen the edge and contour information in the cultural heritage image, and then generate more accurate cultural heritage local perspective features through fusion operations. Through such a design, it can be ensured that in the process of feature extraction, both the global information of the image can be retained and the boundary details of the cultural heritage can be emphasized, providing more comprehensive and accurate feature data for subsequent three-dimensional modeling. In a specific example of the present application, the backbone network in the MBCNet model includes multiple convolution layers based on 5x5 convolution kernels, activation layers and pooling layers based on ReLU activation functions; the boundary feature extraction branch in the MBCNet model includes a point convolution layer, a Sigmoid-based activation layer and an upsampling layer.

[0040] Specifically, the feature selection and strengthening unit 122 is used to input the plurality of local perspective feature maps of the cultural heritage into the compression-suppression attention module of feature selection and strengthening to obtain the plurality of enhanced local perspective feature maps of the cultural heritage. It should be understood that, considering that in each local perspective feature map of the cultural heritage, the contribution of different position features to three-dimensional modeling is not the same, for example, some features may have a more important role in the detailed display and overall structural description of the cultural heritage. Therefore, in the technical solution of the present application, the compression-suppression attention module of feature selection and strengthening is adopted to strengthen the features of each local perspective feature map of the cultural heritage. Specifically, the compression-suppression attention module introduces an attention mechanism, which performs feature compression and global information association interaction on the local perspective feature map of the cultural heritage to learn the feature importance of each feature channel of the local perspective feature map of the cultural heritage, and accordingly strengthens or suppresses each feature channel to different degrees, and by compressing non-critical information and suppressing noise, the model can focus more on representing the core features of the cultural heritage object, which helps to reduce information redundancy and interference, and improve the accuracy and reliability of feature representation.

[0041] Figure 4 FIG. 1 is a block diagram of a feature selection and enhancement unit in a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application. Figure 4As shown, the feature selection and enhancement unit 122 includes: a global mean pooling subunit 1221, which is used to calculate the global mean of each feature matrix along the channel dimension of the cultural heritage local perspective feature map to obtain a cultural heritage local perspective feature compression information representation vector; a channel information association interaction subunit 1222, which is used to perform one-dimensional convolution encoding on the cultural heritage local perspective feature compression representation vector to obtain a cultural heritage local perspective feature compression information correlation representation feature vector; a multi-scale fusion subunit 1223, which is used to combine the cultural heritage local perspective feature compression information representation vector and the cultural heritage local perspective feature compression information The associated representation feature vectors are cascaded to obtain a multi-scale representation vector of the compressed information of the local perspective feature of the cultural heritage; a multi-scale associated feature extraction subunit 1224 is used to input the multi-scale representation vector of the compressed information of the local perspective feature of the cultural heritage into a compression information feature extraction module including two fully connected layers and a SiLU activation function to obtain a multi-scale associated feature vector of the compressed information of the local perspective feature of the cultural heritage; a feature selection subunit 1225 is used to perform feature selection on the local perspective feature map of the cultural heritage based on the multi-scale associated feature vector of the compressed information of the local perspective feature of the cultural heritage to obtain the enhanced local perspective feature map of the cultural heritage.

[0042] Specifically, the feature selection subunit 1225 is used to: use a Sigmoid function to normalize the multi-scale associated feature vector of the cultural heritage local perspective feature compression information to obtain a cultural heritage local perspective weight feature vector; use each eigenvalue of the cultural heritage local perspective weight feature vector as a weight, and perform feature amplification and suppression operations on each feature channel of the cultural heritage local perspective feature map to obtain the enhanced cultural heritage local perspective feature map.

[0043] That is, the feature selection and enhancement unit 122 is used to: perform attention selection on the cultural heritage local perspective feature map according to the following feature enhancement formula to obtain the enhanced cultural heritage local perspective feature map, wherein the feature enhancement formula is:

[0044] ;

[0045] ;

[0046] ;

[0047] in, represents a local viewing angle feature map of the cultural heritage, The local visual feature map of the cultural heritage is represented by The coordinates in the channel are The eigenvalues ​​of a point, represents the height of the local visual feature map of the cultural heritage, represents the width of the local visual feature map of the cultural heritage, Represents the compressed information representation vector of the local visual feature of cultural heritage, The first vector representing the compressed information representation of the local visual feature of cultural heritage eigenvalues, Represents the multi-scale representation vector of the compressed information of the local perspective feature of cultural heritage. represents a one-dimensional convolution operation, Indicates cascade operation, represents a multi-layer perceptron, represents the activation function, represents channel-wise dot product, A local perspective feature map of the enhanced cultural heritage is represented.

[0048] In the above-mentioned tourist attraction digital twin management system 100 based on the Internet of Things, the local view feature aggregation module 130 is used to perform feature reverse aggregation on the multiple local view feature maps of the enhanced cultural heritage to obtain a panoramic cultural heritage object representation feature vector. Figure 5 FIG. 1 is a block diagram of a local view feature aggregation module in a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application. Figure 5 As shown, the local perspective feature aggregation module 130 includes: a feature shape reshaping unit 131, used to reshape the multiple enhanced cultural heritage local perspective feature maps to obtain multiple enhanced cultural heritage local perspective feature vectors; a reverse mask aggregation unit 132, used to input the multiple enhanced cultural heritage local perspective feature vectors into a feature distribution reverse mask aggregation network to obtain the panoramic cultural heritage object representation feature vector.

[0049] Specifically, the characteristic shape reshaping unit 131 is used to reshape the plurality of enhanced cultural heritage local perspective feature maps to obtain a plurality of enhanced cultural heritage local perspective feature vectors. That is, in order to unify the feature expression methods of the plurality of enhanced cultural heritage local perspective feature maps, reduce data complexity, and simplify the processing flow of feature data, the plurality of enhanced cultural heritage local perspective feature maps are reshaped to obtain a plurality of enhanced cultural heritage local perspective feature vectors with more compact and effective feature representation.

[0050] Specifically, the reverse mask aggregation unit 132 is used to input the multiple enhanced cultural heritage local perspective feature vectors into the feature distribution reverse mask aggregation network to obtain the panoramic cultural heritage object representation feature vector. It should be understood that since the cultural heritage object is a complex three-dimensional structure, its appearance information is distributed in multiple local perspectives. Therefore, in order to obtain the panoramic information of the cultural heritage object, it is necessary to further perform feature aggregation on the multiple enhanced cultural heritage local perspective feature vectors. In the technical solution of the present application, a feature distribution reverse mask aggregation network is used to fuse the multiple enhanced cultural heritage local perspective feature maps. Specifically, the feature distribution reverse mask aggregation network evaluates the correlation between the feature vectors and the multi-perspective overall features based on the distribution characteristics of each enhanced cultural heritage local perspective feature vector, identifies unnecessary elements therein, and enhances the unnecessary feature information through the reverse mask mechanism, and then calculates the difference between the multiple enhanced cultural heritage local perspective feature vectors and the unnecessary feature information to filter out the irrelevant features in the multiple enhanced cultural heritage local perspective feature vectors, thereby achieving feature purification and optimization. The purified and optimized enhanced local perspective feature vectors of cultural heritage are aggregated to obtain the overall structure and appearance information of the cultural heritage objects, thereby effectively avoiding duplication and redundancy of information and providing a solid foundation for subsequent 3D modeling and digital twin display.

[0051] Figure 6 FIG. 1 is a block diagram of a reverse mask aggregation unit in a digital twin management system for tourist attractions based on the Internet of Things according to an embodiment of the present application. Figure 6As shown, the reverse mask aggregation unit 132 includes: a feature clustering subunit 1321, which is used to cluster the multiple enhanced cultural heritage local perspective feature vectors to obtain a cluster center vector; a semantic correlation coefficient calculation subunit 1322, which is used to calculate the semantic correlation coefficient between each enhanced cultural heritage local perspective feature vector and the cluster center vector in the multiple enhanced cultural heritage local perspective feature vectors to obtain a sequence of semantic correlation coefficients; a semantic anti-correlation coefficient calculation subunit 1323, which is used to take the inverse of each semantic correlation coefficient in the sequence of semantic correlation coefficients to obtain a sequence of semantic anti-correlation coefficients; a normalization subunit 1324, which is used to use a Sigmoid function to normalize the sequence of semantic anti-correlation coefficients to obtain a semantic anti-correlation coefficient. A sequence of related weight coefficients; a reverse weighting subunit 1325, used to weight each of the multiple enhanced cultural heritage local perspective feature vectors with each semantic anti-correlation weight coefficient in the sequence of semantic anti-correlation weight coefficients as a weight to obtain multiple suppressed cultural heritage local perspective feature vectors; a position difference subunit 1326, used to calculate the position difference between the multiple enhanced cultural heritage local perspective feature vectors and the multiple suppressed cultural heritage local perspective feature vectors to obtain multiple optimized cultural heritage local perspective feature vectors; a panoramic aggregation subunit 1327, used to calculate the position mean vector of the multiple optimized cultural heritage local perspective feature vectors to obtain the panoramic cultural heritage object representation feature vector.

[0052] Specifically, the semantic correlation coefficient calculation subunit 1322 is used to: calculate the vector product between the transposed vector of the enhanced cultural heritage local perspective feature vector and the cluster center vector to obtain a semantic correlation factor; calculate the product between the length of the enhanced cultural heritage local perspective feature vector and the length of the cluster center vector to obtain a scale correlation factor; divide the semantic correlation factor by the scale correlation factor to obtain the semantic correlation coefficient.

[0053] That is, the reverse mask aggregation unit 132 is used to process the multiple enhanced cultural heritage local perspective feature vectors using the following reverse mask aggregation formula to obtain the panoramic cultural heritage object representation feature vector, wherein the reverse mask aggregation formula is:

[0054] ;

[0055] ;

[0056] ;

[0057] ;

[0058] ;

[0059] ;

[0060] ;

[0061] ;

[0062] in, are the multiple local perspective feature vectors of the enhanced cultural heritage, , , and They are respectively the first enhanced cultural heritage local perspective feature vector, the second enhanced cultural heritage local perspective feature vector, the The local perspective feature vector of the enhanced cultural heritage and the A local perspective feature vector of enhanced cultural heritage, The value of is the number of enhanced cultural heritage local perspective feature vectors in the multiple enhanced cultural heritage local perspective feature vectors, represents the transpose of the eigenvector, is the cluster center vector, represents the length of the vector, Indicates the The semantic correlation coefficient between the local perspective feature vector of the enhanced cultural heritage and the cluster center vector, For the The semantic anti-correlation coefficient, For the semantic anti-correlation weight coefficients, represents an exponential function with a natural constant as base, are multiple local view feature vectors that suppress cultural heritage, , , and They are respectively the first cultural heritage suppression local perspective feature vector, the second cultural heritage suppression local perspective feature vector, the The local perspective feature vector of the suppressed cultural heritage and the The local perspective feature vector of suppressed cultural heritage, are multiple optimized cultural heritage local perspective feature vectors, , , and They are the first one to optimize the local perspective feature vector of cultural heritage, the second one to optimize the local perspective feature vector of cultural heritage, The optimized local perspective feature vector of cultural heritage and Optimize the local perspective feature vector of cultural heritage, A feature vector is represented for the panoramic cultural heritage object.

[0063] In the above-mentioned tourist attraction digital twin management system 100 based on the Internet of Things, the three-dimensional model generation module 140 is used to generate a three-dimensional model of a cultural heritage object based on the panoramic cultural heritage object representation feature vector. In a specific example of the present application, the implementation method of generating a three-dimensional model of a cultural heritage object based on the panoramic cultural heritage object representation feature vector is to input the panoramic cultural heritage object representation feature vector into a three-dimensional model generator based on a diffusion model to obtain the three-dimensional model of the cultural heritage object. It should be understood that the three-dimensional model is a digital representation of a cultural heritage object in a virtual space, which can provide an immersive and multi-dimensional viewing experience for tourists. In the technical solution of the present application, a diffusion model is used to simulate the three-dimensional structure generation process of a cultural heritage object. Among them, the diffusion model is a powerful generation model that captures the potential structure of data by simulating the diffusion and inverse diffusion process of data. In the generation process of the three-dimensional model of the cultural heritage object, the diffusion model first destroys the original panoramic cultural heritage object representation feature vector by introducing noise, and then gradually removes the noise to restore the three-dimensional structure of the cultural heritage object. Through the iterative optimization process, each local detail of the cultural heritage object is accurately modeled to generate a high-quality three-dimensional model of the cultural heritage object.

[0064] In particular, in the technical solution of the present application, each of the multiple enhanced cultural heritage local perspective feature vectors respectively represents the enhanced image semantic features of each local perspective image of the monitored cultural heritage object. Considering that there are large differences in image source domain content between the local perspective images of the monitored cultural heritage object, after feature extraction and feature selection enhancement, the heterogeneous parts of the feature distribution structure between each of the multiple enhanced cultural heritage local perspective feature vectors will be highlighted. This means that when the multiple enhanced cultural heritage local perspective feature vectors are input into the feature distribution reverse mask aggregation network, the obtained panoramic cultural heritage object representation feature vector will also result in insufficient feature reverse aggregation coverage due to the significant structural heterogeneity of local features, affecting the final three-dimensional model diffusion generation effect.

[0065] Based on this, when the panoramic cultural heritage object representation feature vector is passed through a three-dimensional model generator based on a diffusion model to obtain a three-dimensional model of the cultural heritage object, the panoramic cultural heritage object representation feature vector is optimized, and the optimization process specifically includes:

[0066] Calculating a feature mean of the panoramic cultural heritage object representation feature vector, and dividing the feature mean by a difference between a maximum eigenvalue and a minimum eigenvalue of the panoramic cultural heritage object representation feature vector to obtain a panoramic cultural heritage object representation distribution representation value;

[0067] Subtract the panoramic cultural heritage object representation distribution representation value from one, and divide the result by the panoramic cultural heritage object representation distribution representation value to obtain a panoramic cultural heritage object representation distribution modulation value;

[0068] Activating the panoramic cultural heritage object representation feature vector through a probabilistic function to obtain a probabilistic panoramic cultural heritage object representation feature vector;

[0069] After performing point subtraction between the probabilistic panoramic cultural heritage object representation feature vector and the panoramic cultural heritage object representation distribution modulation value, taking the absolute value and calculating the negative of the logarithmic value with base 2 to obtain a probabilistic panoramic cultural heritage object representation distribution modulation information vector;

[0070] After dividing the panoramic cultural heritage object representation distribution characterization value by one minus the difference of each eigenvalue of the probabilistic panoramic cultural heritage object representation feature vector, summing all eigenvalues ​​of the probabilistic panoramic cultural heritage object representation feature vector and dividing by the length of the panoramic cultural heritage object representation feature vector to obtain a probabilistic panoramic cultural heritage object representation distribution modulation bias value;

[0071] The optimized panoramic cultural heritage object representation feature vector is obtained by multiplying the probabilistic panoramic cultural heritage object representation distribution modulation information vector and the probabilistic panoramic cultural heritage object representation distribution modulation bias value by a weight as a hyperparameter;

[0072] The optimized panoramic cultural heritage object representation feature vector is input into the diffusion model-based three-dimensional model generator to obtain the cultural heritage object three-dimensional model.

[0073] The optimized representation of the panoramic cultural heritage object representation feature vector is as follows:

[0074] ;

[0075] in, is the probabilistic panoramic cultural heritage object representation distribution modulation information vector, The probabilistic panoramic cultural heritage object represents the distribution modulation information vector eigenvalues, is the distribution representation value of the panoramic cultural heritage object, represents the logarithmic function value with base 2, is the length of the distribution modulation information vector representing the panoramic cultural heritage object, is the weight as a hyperparameter, It means to make a difference by position. It means adding by position. The optimized panoramic cultural heritage object represents a distribution modulation information vector.

[0076] And among them:

[0077] ;

[0078] in, is the distribution representation value of the panoramic cultural heritage object, represents the characteristic mean of the distribution modulation information vector of the panoramic cultural heritage object representation, and They respectively represent the maximum eigenvalue and the minimum eigenvalue of the distribution modulation information vector represented by the panoramic cultural heritage object.

[0079] Specifically, in the above preferred example, the eigenvalue-based probability information distribution planning of the panoramic cultural heritage object representation feature vector is performed by using the Bernoulli probability modulation distribution of the panoramic cultural heritage object representation feature vector relative to the eigenvalue distribution, and the probability reverse mapping of the probability characteristics of the panoramic cultural heritage object representation feature vector as a whole is used as an extended coverage of the set mapping space of the panoramic cultural heritage object representation feature vector to independently understand the interaction path between the intuitive probability information distribution and the abstract probability space mapping of the panoramic cultural heritage object representation feature vector, so as to improve the accuracy of the three-dimensional model of the cultural heritage object obtained by the optimized panoramic cultural heritage object representation feature vector through the three-dimensional model generator based on the diffusion model by avoiding the counterfactual reasoning mapping of the outlier feature distribution of the panoramic cultural heritage object representation feature vector to the generative probability.

[0080] In the above-mentioned digital twin management system 100 for tourist attractions based on the Internet of Things, the digital twin display module 150 is used to display the three-dimensional model of the cultural heritage object through the digital twin module. It should be understood that digital twin is a technology that maps and interacts with the physical world and the virtual world, and can provide a new solution for the protection, display and management of cultural heritage. Through the digital twin module, the generated three-dimensional model of the cultural heritage object is displayed, and tourists can freely browse and observe the cultural heritage object in the virtual space through terminal devices such as virtual reality devices or computer screens, thereby realizing an immersive experience of cultural heritage.

[0081] In summary, according to the embodiment of the present application, a digital twin management system for tourist attractions based on the Internet of Things is explained, which uses computer vision technology based on deep learning to perform image processing on multiple local perspective images of cultural heritage objects to extract the visual features of the cultural heritage at each local perspective, and then perform feature aggregation analysis on the visual features of the cultural heritage at multiple local perspectives to obtain panoramic information of the cultural heritage objects, thereby generating a corresponding three-dimensional model of the cultural heritage object, and displaying the model through digital twin technology, which can provide tourists with an immersive, multi-dimensional viewing experience, and provide a platform that combines virtual and reality to achieve fine management and protection of cultural heritage.

[0082] Figure 7 Flowchart of the digital twin management method of tourist attractions based on the Internet of Things according to an embodiment of the present application. Figure 7 As shown, according to the Internet of Things-based digital twin management method for tourist attractions according to an embodiment of the present application, the steps include: S1, obtaining multiple local perspective images of the monitored cultural heritage object collected by a camera from multiple angles; S2, performing image feature extraction and feature selection enhancement on the multiple local perspective images to obtain multiple enhanced cultural heritage local perspective feature maps; S3, performing feature reverse aggregation on the multiple enhanced cultural heritage local perspective feature maps to obtain a panoramic cultural heritage object representation feature vector; S4, generating a three-dimensional model of the cultural heritage object based on the panoramic cultural heritage object representation feature vector; S5, displaying the three-dimensional model of the cultural heritage object through a digital twin module.

[0083] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned tourist attraction digital twin management method based on the Internet of Things have been referred to above. Figures 1 to 6 The description of the IoT-based digital twin management system for tourist attractions has been introduced in detail, and therefore, its repeated description will be omitted.

[0084] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0085] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical sub-unit, that is, it may be located in one place, or it may be distributed on multiple network sub-units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0086] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0087] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. The digital twin management system of tourist attractions based on the Internet of Things is characterized by: include: A multi-angle image acquisition module for cultural heritage objects, used to acquire multiple local perspective images of the monitored cultural heritage objects collected by the camera from multiple angles; A local perspective image feature mining module, used to extract image features and select and enhance features of the plurality of local perspective images to obtain a plurality of enhanced cultural heritage local perspective feature maps; A local view feature aggregation module, used for performing feature reverse aggregation on a plurality of the enhanced cultural heritage local view feature maps to obtain a panoramic cultural heritage object representation feature vector; A three-dimensional model generation module, used to generate a three-dimensional model of the cultural heritage object based on the panoramic cultural heritage object representation feature vector; A digital twin display module, used to display the three-dimensional model of the cultural heritage object through a digital twin module; The local view feature aggregation module includes: A characteristic shape reshaping unit, used for reshaping the plurality of enhanced cultural heritage local perspective feature maps into characteristic shapes to obtain a plurality of enhanced cultural heritage local perspective feature vectors; A reverse mask aggregation unit, used for inputting a plurality of the enhanced cultural heritage local perspective feature vectors into a feature distribution reverse mask aggregation network to obtain the panoramic cultural heritage object representation feature vector; The reverse mask aggregation unit comprises: A feature clustering subunit, used for clustering a plurality of the local perspective feature vectors of the enhanced cultural heritage to obtain a cluster center vector; A semantic correlation coefficient calculation subunit, used for calculating the semantic correlation coefficient between each of the plurality of enhanced cultural heritage local perspective feature vectors and the cluster center vector to obtain a sequence of semantic correlation coefficients; A semantic anti-correlation coefficient calculation subunit, used for taking the reciprocal of each semantic correlation coefficient in the sequence of semantic correlation coefficients to obtain a sequence of semantic anti-correlation coefficients; A normalization subunit, used for normalizing the sequence of semantic anti-correlation coefficients using a Sigmoid function to obtain a sequence of semantic anti-correlation weight coefficients; a reverse weighting subunit, configured to weight each enhanced cultural heritage local perspective feature vector in the plurality of enhanced cultural heritage local perspective feature vectors by using each semantic anti-correlation weight coefficient in the sequence of the semantic anti-correlation weight coefficients as a weight, so as to obtain a plurality of suppressed cultural heritage local perspective feature vectors; A position difference subunit is used to calculate the position difference between the plurality of enhanced cultural heritage local perspective feature vectors and the plurality of suppressed cultural heritage local perspective feature vectors to obtain a plurality of optimized cultural heritage local perspective feature vectors; A panoramic aggregation subunit, used for calculating a position mean vector of a plurality of the optimized cultural heritage local perspective feature vectors to obtain the panoramic cultural heritage object representation feature vector; The semantic correlation coefficient calculation subunit is used to: Calculating the vector product between the transposed vector of the enhanced cultural heritage local perspective feature vector and the cluster center vector to obtain a semantic association factor; Calculating the product of the length of the local perspective feature vector of the enhanced cultural heritage and the length of the cluster center vector to obtain a scale-related factor; Dividing the semantic relevance factor by the scale relevance factor to obtain the semantic relevance coefficient; The three-dimensional model generation module is used to: Inputting the panoramic cultural heritage object representation feature vector into a three-dimensional model generator based on a diffusion model to obtain a three-dimensional model of the cultural heritage object; When the panoramic cultural heritage object representation feature vector is passed through a diffusion model-based 3D model generator to obtain a 3D model of the cultural heritage object, the panoramic cultural heritage object representation feature vector is optimized. The optimization process specifically includes: Calculating a feature mean of the panoramic cultural heritage object representation feature vector, and dividing the feature mean by a difference between a maximum eigenvalue and a minimum eigenvalue of the panoramic cultural heritage object representation feature vector to obtain a panoramic cultural heritage object representation distribution representation value; Subtracting the panoramic cultural heritage object representation distribution characterization value from one and dividing the result by the panoramic cultural heritage object representation distribution characterization value to obtain a panoramic cultural heritage object representation distribution modulation value; Activating the panoramic cultural heritage object representation feature vector through a probabilistic function to obtain a probabilistic panoramic cultural heritage object representation feature vector; After performing point subtraction between the probabilistic panoramic cultural heritage object representation feature vector and the panoramic cultural heritage object representation distribution modulation value, taking the absolute value and calculating the negative of the logarithmic value with base 2 to obtain a probabilistic panoramic cultural heritage object representation distribution modulation information vector; After dividing the panoramic cultural heritage object representation distribution characterization value by one minus the difference of each eigenvalue of the probabilistic panoramic cultural heritage object representation feature vector, summing all eigenvalues ​​of the probabilistic panoramic cultural heritage object representation feature vector and dividing by the length of the panoramic cultural heritage object representation feature vector to obtain a probabilistic panoramic cultural heritage object representation distribution modulation bias value; The optimized panoramic cultural heritage object representation feature vector is obtained by multiplying the probabilistic panoramic cultural heritage object representation distribution modulation information vector and the probabilistic panoramic cultural heritage object representation distribution modulation bias value by a weight as a hyperparameter; The optimized panoramic cultural heritage object representation feature vector is input into the diffusion model-based three-dimensional model generator to obtain the cultural heritage object three-dimensional model.

2. The digital twin management system for tourist attractions based on the Internet of Things according to claim 1 is characterized in that: The local view image feature mining module includes: An image feature extraction unit, used for inputting the plurality of local view images into an MBCNet model including a backbone network and a boundary feature extraction branch to obtain a plurality of cultural heritage local view feature maps; The feature selection and enhancement unit is used to input the multiple cultural heritage local perspective feature maps into the feature selection and enhancement compression-suppression attention module to obtain the multiple enhanced cultural heritage local perspective feature maps.

3. The digital twin management system for tourist attractions based on the Internet of Things according to claim 2 is characterized in that: The backbone network in the MBCNet model includes multiple convolution layers based on 5x5 convolution kernels, activation layers and pooling layers based on ReLU activation functions; the boundary feature extraction branch in the MBCNet model includes a point convolution layer, an activation layer based on Sigmoid and an upsampling layer.

4. The digital twin management system for tourist attractions based on the Internet of Things according to claim 3 is characterized in that: The feature selection and strengthening unit comprises: A global mean pooling subunit, used to calculate the global mean of each feature matrix along the channel dimension of the cultural heritage local perspective feature map to obtain a cultural heritage local perspective feature compression information representation vector; A channel information association interaction subunit is used to perform one-dimensional convolution coding on the cultural heritage local perspective feature compression representation vector to obtain a cultural heritage local perspective feature compression information association representation feature vector; A multi-scale fusion sub-unit, used for cascading the cultural heritage local perspective feature compression information representation vector and the cultural heritage local perspective feature compression information association representation feature vector to obtain a cultural heritage local perspective feature compression information multi-scale representation vector; A multi-scale correlation feature extraction subunit, used for inputting the multi-scale representation vector of the cultural heritage local perspective feature compression information into a compression information feature extraction module including two fully connected layers and a SiLU activation function to obtain a multi-scale correlation feature vector of the cultural heritage local perspective feature compression information; The feature selection subunit is used to perform feature selection on the cultural heritage local perspective feature map based on the multi-scale associated feature vector of the cultural heritage local perspective feature compression information to obtain the enhanced cultural heritage local perspective feature map.

5. The digital twin management system for tourist attractions based on the Internet of Things according to claim 4 is characterized in that: The feature selection subunit is used to: Using a Sigmoid function to normalize the multi-scale associated feature vector of the local perspective feature compression information of the cultural heritage to obtain a local perspective weight feature vector of the cultural heritage; Taking each eigenvalue of the cultural heritage local perspective weight feature vector as a weight, feature amplification and suppression operations are performed on each feature channel of the cultural heritage local perspective feature map to obtain the enhanced cultural heritage local perspective feature map.

6. A digital twin management method for tourist attractions based on the Internet of Things, used for a digital twin management system for tourist attractions based on the Internet of Things as claimed in any one of claims 1 to 5, characterized in that: include: Acquire multiple local perspective images of the monitored cultural heritage object collected by the camera from multiple angles; Performing image feature extraction and feature selection enhancement on the plurality of local perspective images respectively to obtain a plurality of enhanced cultural heritage local perspective feature maps; Performing feature reverse aggregation on a plurality of the enhanced cultural heritage local perspective feature maps to obtain a panoramic cultural heritage object representation feature vector; generating a three-dimensional model of the cultural heritage object based on the panoramic cultural heritage object representation feature vector; The three-dimensional model of the cultural heritage object is displayed through the digital twin module.

Citation Information

Patent Citations

  • Panoramic imaging device and method for tourism landscape display

    CN117726511A

  • Digital twinning-oriented urban refined road construction system and method

    CN118314296A