Digital twinborn visualization system based on three-dimensional grid

Abstract

The invention relates to the technical field of data twinning, and particularly discloses a digital twinning visualization system based on a three-dimensional grid, which comprises a plurality of data centers, a data processing module and a twinning data service module. Wherein the data center is arranged close to the data source, obtains attribute data of each physical object from the adjacent data source, associates the attribute data with three-dimensional grid units, and generates a three-dimensional grid unit set of each physical object; and for each physical object, filtering the corresponding three-dimensional grid unit set to screen out a plurality of three-dimensional grid units, and establishing an adjacent relationship among the screened three-dimensional grid units. The data processing module determines the number of required three-dimensional grid units according to the data service request, extracts corresponding three-dimensional grid units, encodes the three-dimensional grid units, and generates service response data; and the twin data service module comprises a rendering engine and an interaction unit. The system is helpful for improving intelligentization, maturity and large-scale application of digital twinning.

Description

Technical Field

[0001] The present application relates to the field of data twin technology, and in particular to a three-dimensional grid-based digital twin visualization system. Background Art

[0003] The visualization system is the core element and functional foundation of digital twins, and is the basic capability support for realizing data understanding, simulation deduction, resource optimization and operation management, and decision support. At present, the digital twin visualization system is mainly implemented by integrating static charts, 2D or 3D models, GIS systems, etc., and has problems such as insufficient real-time performance, difficulty in integrating and standardizing large-scale multimodal data, limited interactivity, poor scalability, rendering performance bottlenecks, low computability, and high implementation costs, which restrict the intelligence, maturity, and large-scale application of digital twins.

[0004] Therefore, a new digital twin visualization system is needed to solve the above problems. Summary of the invention

[0005] The present application provides a three-dimensional grid-based digital twin visualization system in an effort to solve or at least alleviate at least one of the above problems.

[0006] According to one aspect of the present application, a three-dimensional grid-based digital twin visualization system is provided, including: multiple data centers, arranged near a data source, suitable for obtaining attribute data of each physical object from a neighboring data source, and associating the attribute data with a three-dimensional grid unit to generate a three-dimensional grid unit set for each physical object; it is also suitable for filtering the corresponding three-dimensional grid unit set for each physical object, screening out multiple three-dimensional grid units, and establishing an adjacent relationship between the screened three-dimensional grid units, wherein the adjacent relationship describes the structure of the corresponding physical object from the dimension of spatial relationship; a data processing module, suitable for determining the required number of three-dimensional grid units according to a data service request, extracting and encoding the corresponding three-dimensional grid units from the data center, generating service response data and sending it to a twin data service module; the twin data service module, including: a rendering engine, suitable for decoding the service response data, and rendering according to the attribute data in the three-dimensional grid unit with the three-dimensional grid unit as the granularity and rendering object, and displaying the rendering effect through a visualization interface; an interaction unit, suitable for identifying and maintaining the interactive spatial relationship based on the adjacent relationship between the three-dimensional grid units involved in the interactive instruction when executing the interactive instruction.

[0007] Optionally, in the system according to the present application, each data center includes: a three-dimensional grid division unit, suitable for dividing the spatial range to obtain multi-level three-dimensional grid units, and encoding each three-dimensional grid unit respectively to obtain each unique grid code; an attribute data processing unit, suitable for aggregating and processing the attribute data of the same physical object to obtain processed attribute data; a grid data association unit, suitable for determining multiple three-dimensional grid units for displaying each of the physical objects; and associating and matching each determined three-dimensional grid unit with the processed attribute data of the corresponding physical object to obtain a three-dimensional grid unit set of each of the physical objects; a grid data linking unit, suitable for, for each three-dimensional grid unit set, filtering out multiple three-dimensional grid units from the three-dimensional grid unit set according to predetermined rules, and establishing an adjacent relationship between each three-dimensional grid unit by linking the filtered three-dimensional grid units in sequence.

[0008] Optionally, in the system according to the present application, the grid data association unit is further suitable for selecting a level of three-dimensional grid cells according to the scale of the map displaying each of the physical objects; determining the coordinates of each corner point of each three-dimensional grid cell within the spatial range according to the selected level; performing spatial position calculation on the processed attribute data of each of the physical objects and the three-dimensional grid cells of the corresponding level based on the coordinates to determine the three-dimensional grid cells corresponding to the processed attribute data; writing the processed attribute data in the corresponding three-dimensional grid cells by adding attribute fields, and after writing all the processed attribute data of the physical objects, obtaining the corresponding set of three-dimensional grid cells of the physical objects.

[0009] Optionally, in the system according to the present application, the grid data association unit is also suitable for determining the coordinates of the positioning point of the three-dimensional grid unit based on the encoding rule of the three-dimensional grid unit, wherein the positioning point is the corner point closest to the starting position of the subdivision; and calculating the coordinates of other corner points of the three-dimensional grid unit according to the accuracy corresponding to the level of the three-dimensional grid unit and the coordinates of the positioning point.

[0010] Optionally, in the system according to the present application, the three-dimensional grid subdivision unit is also suitable for subdividing the spatial range. For a larger spatial range, the minimum accuracy of the three-dimensional grid unit is at the meter level, and the maximum accuracy is at the kilometer level; for a smaller spatial range, the minimum accuracy of the three-dimensional grid unit is at the centimeter level, and the maximum accuracy is at the ten-meter level.

[0011] Optionally, in a system according to the present application, the predetermined rules include: if a three-dimensional grid cell is not associated and matched, the three-dimensional grid cell is filtered out; for high-precision three-dimensional grid cells, the three-dimensional grid cells are thinned out based on their associated attribute data to obtain thinned three-dimensional grid cells; for three-dimensional grid cells in which physical objects only occupy part of the space, the occupied part of the three-dimensional grid cells is retained by cropping.

[0012] Optionally, in the system according to the present application, the grid data link unit is also suitable for linking the selected three-dimensional grid units in combination with the logical order of the structural description of the physical object and the encoding order of the grid code, wherein the logical order of the structural description of the physical object includes: from macro to micro, from the whole to the part; and establishing the adjacent relationship between the selected three-dimensional grid units in the order pointed by the links.

[0013] Optionally, in the system according to the present application, the data service request includes the current map scale, the screen visible range and the physical objects within the screen visible range, and the data processing module is also suitable for determining the level of the three-dimensional grid unit according to the current map scale, and calculating the number of three-dimensional grid units required within the screen visible range.

[0014] Optionally, in the system according to the present application, the interaction unit is further adapted to feed back changes in attribute data occurring during the interaction operation to the physical object, thereby achieving virtual-reality interaction.

[0015] Optionally, in the system according to the present application, the twin data service module also includes a simulation unit, which is suitable for performing simulation based on three-dimensional grid units according to the configuration parameters of the physical object, wherein the configuration parameters include: physical object state configuration parameters, algorithm configuration parameters, data configuration parameters, and boundary configuration parameters.

[0016] Optionally, in the system according to the present application, the attribute data processing unit processes the attribute data including: aligning the geographic location information and time information of the attribute data; supplementing missing data points using an interpolation algorithm; identifying abnormal data values ​​and correcting the abnormal data values ​​based on historical data or adjacent data points; and detecting and removing duplicate data through hash mapping.

[0017] Optionally, in the system according to the present application, the attribute data processing unit is further suitable for storing the processed attribute data in a data lake architecture.

[0018] In summary, according to the solution of the present application, there is no need to make a three-dimensional model, and pure vectorized twin data can be used to describe and express physical objects, so as to achieve efficient organization, fusion processing and real-time calculation of large-scale multimodal twin data. Specifically, the data center integrates, filters, clips and other processes the attribute data of physical objects, associates and matches them with three-dimensional grid units, and expresses the attribute data of physical objects from the dimension of spatial relationships. The twin data service module uses three-dimensional grid units as cells for expressing physical objects, and dynamically updates and interacts with physical object status information in real time. This system helps to improve the intelligence, maturity and large-scale application of digital twins.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles applied herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. By reading the following detailed description in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. Throughout the application, the same reference numerals generally refer to the same parts or elements.

[0021] Figure 1 A schematic diagram of a three-dimensional grid-based digital twin visualization system 100 according to some embodiments of the present application is shown; Figure 2 A schematic diagram showing corner points of a three-dimensional grid unit according to some embodiments of the present application is shown; Figure 3 A schematic diagram showing the neighbor relationship between three-dimensional grid units according to some embodiments of the present application; Figure 4 An example diagram of above-ground and underground integrated visualization according to some embodiments of the present application is shown; Figure 5 An example diagram of topographic visualization according to some embodiments of the present application is shown; Figure 6 A schematic diagram of a computing device 600 according to some embodiments of the present application is shown. DETAILED DESCRIPTION

[0022] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0023] ‌Physical objects‌ refer to objects that actually exist, can be touched and sensed, including natural and man-made objects. According to the implementation of this application, physical objects mainly refer to buildings, bridges, roads, various equipment (such as vehicles, sensors, municipal facilities, etc.), people, etc.

[0024] Different from the voxelized expression method that divides the physical objects in three-dimensional space into small cubes (voxels) according to fixed physical sizes and represents the volume information of the physical objects by filling the voxels, the digital twin visualization system based on three-dimensional grids provided by this application realizes the elastic spatial precision expression of physical objects at multiple levels (up to 32 levels). Among them, the three-dimensional grid can be GeoSOT earth space grid, Beidou grid, etc.

[0025] In addition, voxel grids only record the physical features of physical objects, while 3D grids are not only associated with the physical features of physical objects, but also with information such as the real-time status, behavior, and operation of physical objects. Therefore, unlike voxelized expressions that focus on the expression of the volume and physical properties of objects, which are suitable for the detailed display and internal structure visualization of 3D models, this system focuses more on the overall digital mapping of the physical world and its operation. In addition to visual expression, it has outstanding application advantages in digital-analog interaction, simulation deduction, etc.

[0026] Among them, the three-dimensional grid location code is based on the earth segmentation model, which uniformly divides the earth's three-dimensional space into three-dimensional grid units with three-dimensional spatial location attributes (which can be solved for longitude and latitude coordinates) at multiple levels of different precision, and is identified and expressed according to unified coding rules, building a grid-based earth three-dimensional spatial data organization reference framework, supporting the aggregation of earth three-dimensional space and geographic spatial information, breaking through the bottlenecks of uniqueness, readability, scalability and relevance of large-scale multimodal data expression, and realizing efficient organization, fusion processing and real-time calculation of large-scale multimodal data. The three-dimensional grid location code includes GeoSOT (Geographical coordinate global Subdivision based on One-dimension-integer to n th power) geospatial grid code, Beidou grid location code, etc.

[0027] Figure 1FIG. 1 is a schematic diagram of a three-dimensional grid-based digital twin visualization system 100 according to some embodiments of the present application. Figure 1 The digital twin visualization system 100 includes: multiple data centers 110, a data processing module 120, and a twin data service module 130.

[0028] Among them, the data center 110 is arranged near the data source and stores data, forming a decentralized distributed data storage system with multi-data center collaboration and synchronization. Through the multi-data center approach, data applications can be seamlessly switched in multiple data centers to avoid data unavailability caused by data center node failures, and achieve real-time, reliability and high availability of data. The data source is, for example, a data storage system that stores architectural CAD data, text files, etc., a radar system, a positioning device, a video acquisition device, an Internet of Things sensing device, etc., which is used to collect or obtain relevant information of various physical objects within a certain spatial range. In this embodiment, the data representing the relevant information of the physical object is collectively referred to as the attribute data of the physical object. Attribute data, for example, includes: the location, material, internal and external shape, structure, status information, etc. of the physical object, which are not listed one by one here.

[0029] The data center 110 obtains the attribute data of each physical object from the adjacent data source through network transmission, file reading, database docking, Internet of Things data gateway, etc. After that, the data center 110 generates pure vectorized twin data of each physical object through a series of processing to describe and express the physical scene of the physical object within the spatial range. According to the implementation mode of the present application, the twin data consists of a number of three-dimensional grid units and their adjacent relationships.

[0030] The data processing module 120 is connected to each data center 110. The data processing module 120 receives data service requests, such as data loading requests, data query requests, data analysis requests, etc. Usually, the data service request includes request parameters such as the current map scale and physical objects within the visible range of the screen. According to the request parameters, the data processing module 120 extracts the corresponding twin data from the data center 110, encodes it, and sends it to the twin data service module 130 as service response data.

[0031] The twin data service module 130 is connected to the data processing module 120 and is usually used to process the service response data, using three-dimensional grid units as units to achieve rendering, query, operation interaction, simulation, etc.

[0032] The following is a detailed description of each component in the digital twin visualization system 100.

[0033] In general, the data center 110 associates the attribute data with the three-dimensional grid cells, and generates a set of three-dimensional grid cells for each physical object. Afterwards, the data center 110 will also filter the corresponding three-dimensional grid cell set for each physical object, screen out multiple three-dimensional grid cells, and establish an adjacent relationship between the screened three-dimensional grid cells. According to an embodiment of the present application, the adjacent relationship describes the structure of the corresponding physical object from the dimension of spatial relationship. In other words, this adjacent relationship and the three-dimensional grid cells involved are the vectorized twin data that characterize the physical object.

[0034] According to an embodiment of the present application, the data center 110 further includes: a three-dimensional grid partitioning unit 112 , an attribute data processing unit 114 , a grid data association unit 116 and a grid data linking unit 118 .

[0035] The three-dimensional grid subdivision unit 112 divides the spatial range to obtain multi-level three-dimensional grid units, and encodes each three-dimensional grid unit to obtain a unique grid code. The spatial range can be set according to the application space of the digital twin visualization system 100, and is not limited here. In some embodiments, when the three-dimensional grid subdivision unit 112 divides the spatial range, for a larger spatial range, the minimum accuracy of the three-dimensional grid unit is meter level, and the maximum accuracy is kilometer level, for example, the accuracy range is 0.97 meters to 1.85 kilometers; for a smaller spatial range, the minimum accuracy of the three-dimensional grid unit is centimeter level, and the maximum accuracy is ten meters level, for example, the accuracy range is 1.5 centimeters to 123.69 meters. Taking the Beidou grid location code technology as an example, for a larger spatial range (such as a city), the three-dimensional grid subdivision unit 112 adopts the 8th to 4th level subdivision coding; for a smaller spatial range (such as a factory park), the three-dimensional grid subdivision unit 112 adopts the 10th to 5th level subdivision coding. Afterwards, the grid code of each three-dimensional grid unit is generated according to the Beidou grid position code. It should be noted that the three-dimensional grid subdivision unit 112 can support multiple encoding methods, and here only the Beidou grid position code is taken as an example to illustrate the subdivision level of the grid unit.

[0036] Since the three-dimensional grid coding is unique and the multi-level grids represent expressions of different precisions, processing the multi-source data in the data center 110 based on the three-dimensional grid can solve the problem of difficulty in integrating and standardizing large-scale multimodal data.

[0037] The attribute data processing unit 114 collects and processes the attribute data of the same physical object to obtain processed attribute data.

[0038] As mentioned above, the attribute data obtained from the data source includes video images, geographic surveying and mapping data, text and other forms of data, so it is necessary to process the aggregated attribute data. In some embodiments, the processing includes the following aspects.

[0039] First, align the geographic location information and time information of the attribute data to achieve spatial and temporal synchronization of the attribute data. The geographic location information of the attribute data can be the data itself, or the geographic location information of the physical object it indicates. For example, if the attribute data is the location of the physical object, then it has its own geographic location information. For another example, if the attribute data is the state of the physical object, then the geographic location information of the physical object is the geographic location information of the attribute data. The time information of the attribute data is usually the timestamp of collecting or obtaining the attribute data.

[0040] Specifically, the geographic location information (such as longitude and latitude) of all attribute data is converted into a unified coordinate system (such as CGCS2000 or WGS84 coordinate system) to ensure spatial consistency. In addition, attribute data of different precisions are fused with corresponding or close-precision three-dimensional grid cells based on weight interpolation or spatial interpolation algorithms (such as Kriging interpolation and bilinear interpolation), ultimately achieving spatial synchronization of attribute data.

[0041] Use time synchronization algorithms (such as NTP-based time calibration) to unify the time base of each attribute data and achieve time synchronization of multi-modal attribute data.

[0042] In addition, data specifications are formulated or standard data formats are used to convert different types of attribute data into a unified and standardized expression to achieve format standardization of attribute data.

[0043] Next, we use the interpolation algorithm to fill in the missing data points. Of course, we can also use machine learning models (such as KNN or LSTM) to fill in the missing data points. I will not go into details here.

[0044] Next, identify the abnormal data values ​​and correct them based on historical data or adjacent data points. For example, abnormal data values ​​can be identified through rule-based detection (such as the reasonable range of geographic location changes) or statistical model-based detection (such as outlier detection algorithms, Z-score, DBSCAN clustering).

[0045] Furthermore, for redundant data, hash mapping is used to efficiently detect and remove duplicate data.

[0046] Finally, the attribute data processing unit 114 stores the processed attribute data in a data lake architecture and performs dynamic real-time updates. The data lake can store a large amount of data of any type, including structured, semi-structured, and unstructured data.

[0047] According to the attribute data processing unit 114 of the present application, not only the problem of difficulty in integrating and standardizing large-scale multimodal attribute data can be effectively solved, but also the problem of poor data scalability can be solved.

[0048] The grid data associating unit 116 determines a plurality of three-dimensional grid units for displaying each physical object, and associates and matches each determined three-dimensional grid unit with the processed attribute data of the physical object to obtain a set of three-dimensional grid units for each physical object.

[0049] Specifically, according to the scale of the map showing each physical object, the level of the three-dimensional grid unit is selected accordingly. In the present system 100, the map scale for showing each physical object is pre-set based on the national basic scale topographic map. Usually, the map scale ranges from 1:5000 to 1:250000. For different physical objects, the corresponding map scale is selected within the range for setting. At the same time, the correspondence between the map scale and the level of the three-dimensional grid unit is also set. For example, when the map scale is 1:5000, it corresponds to the 7th level grid. In short, the larger the map scale, the higher the grid level; the smaller the map scale, the lower the grid level. In this way, according to the map scale of the physical object, its corresponding three-dimensional grid level range or a single three-dimensional grid level can be determined.

[0050] Afterwards, the coordinates of each corner point of each three-dimensional grid unit within the spatial range are determined according to the selected level. In some embodiments, for each three-dimensional grid unit of the level, the coordinates of the positioning point of the three-dimensional grid unit are determined based on the coding rule of the three-dimensional grid unit, where the positioning point is the corner point closest to the starting position of the segmentation. Specifically, the positioning point of the three-dimensional grid unit can be obtained by reverse deduction and solution through the coding rule of the three-dimensional grid unit; then, according to the accuracy corresponding to the level of the three-dimensional grid unit and the coordinates of the positioning point, the coordinates of other corner points of the three-dimensional grid unit are calculated.

[0051] Figure 2 Schematic diagram of the corner points of a three-dimensional grid unit according to some embodiments of the present application is shown. Figure 2 Eight three-dimensional grid units of the same level are shown, among which points A1, A2, ..., H2, H3, etc. are the corner (vertex) points of the three-dimensional grid units, points O1, O2, O3 are the starting positions of the division of three-dimensional grid units at different heights, and the black points with arrows are the positioning points of the corresponding three-dimensional grid units, and the arrows indicate the directions of longitude and latitude.

[0052] According to the above method, the coordinates of each corner (vertex) point of all three-dimensional grid units in all levels within the spatial range can be calculated in sequence.

[0053] Afterwards, based on the calculated coordinates of the corner points of the three-dimensional grid cells, the processed attribute data of the physical object and the three-dimensional grid cells of the corresponding level of the physical object are spatially calculated to determine one or more three-dimensional grid cells corresponding to the processed attribute data.

[0054] Afterwards, the processed attribute data is written into the corresponding three-dimensional grid unit in a manner of adding attribute fields, and after all the processed attribute data of the physical object are written, a corresponding three-dimensional grid unit set of the physical object is obtained.

[0055] Similarly, the above process is performed on each physical object to determine one or more three-dimensional grid cells corresponding to the processed attribute data of each physical object; the processed attribute data is then written into the corresponding three-dimensional grid cells to obtain a corresponding three-dimensional grid cell set for each physical object.

[0056] For example, the following attribute data of a known physical object (building B): center point coordinates, plane shape coordinate set, height and number of floors, physical material of the outer surface, window position and material, internal CAD or laser scanning data, etc. The corresponding three-dimensional grid unit level is the 8th level of the Beidou grid position code (accuracy 0.97m), and the coordinates of each corner point of the three-dimensional grid unit at this level are solved. Then, the spatial position calculation is performed to obtain the three-dimensional grid unit corresponding to each attribute data (for example, according to the window position and the corner point coordinates of the three-dimensional grid unit, determine which one or more three-dimensional grid units the window position should be located in, and use it as the three-dimensional grid unit corresponding to the attribute data "window position and material"), and associate all attribute data of building B with the corresponding three-dimensional grid unit, that is, by adding attribute fields to this part of the three-dimensional grid unit, the corresponding attribute data is written into the associated three-dimensional grid unit. Finally, a multi-scale and multi-level three-dimensional grid unit aggregate that can be updated in real time is obtained, and a full vectorized data description of building B is performed.

[0057] By matching the attribute data of each physical object with the three-dimensional grid unit through the grid data association unit 116, the problem of insufficient real-time performance in the digital twin can be effectively solved, and technical support can be provided for subsequent rendering and calculation.

[0058] For each three-dimensional grid unit set, the grid data linking unit 118 selects multiple three-dimensional grid units from the three-dimensional grid unit set according to a predetermined rule, and establishes an adjacent relationship between the three-dimensional grid units by sequentially linking the selected three-dimensional grid units.

[0059] In some embodiments, the predetermined rules include the following (1)-(3).

[0060] (1) If a 3D grid cell is not associated and matched, the 3D grid cell is filtered out. In other words, the 3D grid cell that is not associated with physical object attribute data is directly discarded.

[0061] (2) For high-precision 3D grid cells (e.g., Beidou grid location code level 8 to 10), the 3D grid cells are thinned based on their associated attribute data to obtain thinned 3D grid cells. For example, for multiple 3D grid cells with the same attribute data, only one of the 3D grid cells is retained.

[0062] (3) For the three-dimensional grid cells where the physical object only occupies part of the space (such as the edge of the physical object), the occupied part of the three-dimensional grid cells is retained by clipping. In other words, for the three-dimensional grid cells where the edge of the physical object is located, the part occupied by the edge is clipped from the three-dimensional grid cells, and the unoccupied part is discarded.

[0063] The above processing is performed on the three-dimensional grid unit set of each physical object, and the remaining three-dimensional grid units are the selected three-dimensional grid units. The selected three-dimensional grid units are grid-linked to establish the adjacent relationship between the three-dimensional grid units representing the physical object.

[0064] According to an embodiment of the present application, the grid data linking unit 118 links the selected three-dimensional grid units in combination with the logical order of the structure description of the physical object and the coding order of the grid code; then, the adjacent relationship between the selected three-dimensional grid units is established according to the order pointed by the link. Specifically, the selected three-dimensional grid units are linked based on the logical order of the structure description of the physical object as the main method and the coding order of the grid code as the auxiliary method. Among them, the logical order of the structure description of the physical object includes: from macro to micro, from the whole to the part. For example, when describing the structure of a physical object such as a building, the order from the whole (height, number of floors, outer surface, etc.) to the part (windows, floors, households, etc.) can be followed. When describing the floors, multiple three-dimensional grid units may be involved, so the coding order of the grid codes of these multiple three-dimensional grid units is followed to link them one after another.

[0065] According to this embodiment, the logical order of the structural description of the physical object is converted into the spatial dimension according to the order in which the links point to and the established adjacent relationship.

[0066] Figure 3 A schematic diagram showing the neighbor relationship between three-dimensional grid units according to some embodiments of the present application is shown. Figure 3, G0→G1→G2→G3 represents a set of adjacent relationships, for example, corresponding to the structural description of physical object A; G4→G5→G6→G7 represents a set of adjacent relationships, for example, corresponding to the structural description of physical object B. "G0, G1, ..." identifies the corresponding three-dimensional grid units according to the order of linking.

[0067] In this way, when description information of physical object A is needed, the three-dimensional grid cells that sequentially describe the attribute data of physical object A can be quickly located through a set of adjacent relationships of “G0→G1→G2→G3”.

[0068] Furthermore, in some other embodiments, the neighbor relationship may include, in addition to indicating the order of the links pointing to, whether each face of the three-dimensional grid unit belongs to the three-dimensional grid unit that is filtered out. Figure 3 In the neighbor relationship, the depth of the color of the face of the three-dimensional grid unit is used to distinguish whether the face belongs to the filtered three-dimensional grid unit. For example, Figure 3 The medium-dark gray surface indicates that the surface belongs to the filtered out 3D grid unit.

[0069] In summary, the mesh data linking unit 118 filters and trims the set of three-dimensional mesh units that have associated physical object attribute data, thereby achieving a more detailed and realistic description and expression of the physical object in terms of internal and external shape, structure, material, scene effect, etc., while also greatly reducing the amount of data and improving rendering performance and computing efficiency.

[0070] In short, as the core foundation of the digital twin visualization system 100, the data center 110 integrates, filters, and clips the multimodal attribute data of the physical object, associates and matches it with the three-dimensional grid unit, and expresses the attribute data of the physical object from the dimension of spatial relationship. The data center 110 can solve the problems of insufficient real-time performance, rendering performance bottleneck, and poor computability in digital twins.

[0071] The data processing module 120 is connected to each data center 110, and the data processing module 120 receives data service requests, such as data loading requests, data query requests, data analysis requests, etc. Usually, the data service request includes request parameters such as the current map scale and physical objects within the visible range of the screen. The data processing module 120 determines the required number of three-dimensional grid units based on the data service request, extracts and encodes the corresponding three-dimensional grid units from the data center 110, generates service response data and sends it to the twin data service module 130.

[0072] Specifically, according to the correspondence between the map scale and the three-dimensional grid unit level, the data processing module 120 determines the level of the three-dimensional grid unit according to the current map scale, thereby obtaining the accuracy of the three-dimensional grid unit and calculating the number of three-dimensional grid units required within the visible range of the screen. Afterwards, a corresponding number of three-dimensional grid units are extracted from the multiple three-dimensional grid units corresponding to the physical object.

[0073] In some embodiments, the data processing module 120 performs binary encoding (such as Protocol Buffers) on the extracted three-dimensional grid units as service response data. In addition, according to actual application requirements, the service response data can also be sliced ​​and then sent to the twin data service module 130. For example, if the spatial scope is an urban area, the amount of data involved is large, and the service response data can be sliced ​​to produce data tiles to improve data transmission efficiency and rendering loading speed.

[0074] The twin data service module 130 is connected to the data processing module 120 and is generally used to process the received service response data to implement rendering, query, operation interaction, simulation, etc. In some embodiments, the twin data service module 130 includes: a rendering engine 132 and an interaction unit 134 .

[0075] According to some embodiments, a digital twin visualization system rendering engine with a B / S architecture (optional C / S architecture for mobile terminals) is designed using the new generation device graphics (GPU) interface WebGPU as the core of the Web rendering engine, and Vulkan and Metal as the core of the mobile rendering engine.

[0076] The rendering engine 132 decodes the service response data (for example, parses the service response data according to a decoding algorithm corresponding to the binary encoding algorithm adopted by the data processing module 120), uses three-dimensional grid units as granularity and rendering objects, renders according to the attribute data in the three-dimensional grid units, and displays the rendering effect through a visual interface.

[0077] First, the physical object shape, material, structure, etc. are rendered initially with three-dimensional grid units as the granularity and rendering object; then, the rendering engine 132 performs baking rendering based on the description of the physical object material and other attributes associated with the three-dimensional grid unit, combined with PBR, light and shadow, HDRI sky, etc. In addition, the rendering engine 132 will also establish a data channel for the physical entity to map the object in the twin scene to achieve real-time virtual-real linkage. In addition, the rendering engine 132 will also build an animation system, a particle effect system, a weather / environment system, and introduce visualization methods such as charts to enrich the expression and effect of the digital twin visualization system 100.

[0078] Figure 4An example diagram of above-ground and underground integrated visualization according to some embodiments of the present application is shown. Figure 5 An example of a topographic visualization diagram according to some embodiments of the present application is shown. Figure 4 For example, you can choose to display ground data, underground building data, geological layer data, grids, ground building cutting methods, etc. in the visualization interface, and you can also choose the display transparency of each data (such as ground data transparency, underground building data transparency, geological layer data transparency, etc.). Figure 4-5 It can be seen that the rendering engine 132 is capable of realizing multi-dimensional, multi-category and multi-level visualization, including physical object visualization and data visualization. In addition, the rendering engine 132 can also realize interactive visualization, algorithm visualization and simulation visualization.

[0079] When the interaction unit 134 receives the user's interaction instruction, it can determine the three-dimensional grid unit to be interacted with by coordinate conversion according to the physical objects, components in the physical objects, and three-dimensional grid units contained in the interaction instruction, and then execute the interaction instruction. Interaction instructions include selection, query, etc. For example, the user can select objects, components or three-dimensional grid units in the visual interface through a variety of selection modes (such as box selection, circle selection, voxel brush, etc.); VR / AR technology can also be introduced, and the user can select and operate the displayed physical objects, components in the physical objects, and three-dimensional grid units through gestures, eye tracking, etc.

[0080] When executing the interaction instruction, the interaction unit 134 identifies and maintains the spatial relationship of the interaction based on the adjacent relationship between the three-dimensional grid units involved in the interaction instruction. For example, if the interaction instruction is to zoom in on a room in a building, the interaction unit 134 can quickly identify the spatial relationship based on the adjacent relationship of the three-dimensional grid units corresponding to the room to ensure that the accurate spatial relationship is maintained during the interaction, avoiding the spatial misalignment display caused by zooming in on a room.

[0081] The three-dimensional grid unit has multi-level precision, which can ensure that the physical object, the components in the physical object, and the three-dimensional grid unit have their own precise coordinates and can switch between different scales. In addition, the interaction unit 134 will also feed back the changes in the attribute data that occur during the interactive operation to the physical object to achieve virtual-real interaction. For example, the state information of the physical object can be updated in real time through the attribute data of the corresponding three-dimensional grid unit, so as to achieve real-time synchronization of the state changes of the physical object, the components in the physical object, and the three-dimensional grid unit in the digital twin visualization expression. Similarly, the information changes of the physical object, the components in the physical object, and the three-dimensional grid unit caused by the interactive operation (such as the changes in the properties such as color, material, size, etc.) can also be fed back to the physical object in real time, thereby achieving virtual-real interaction.

[0082] In some other embodiments, the twin data service module 130 further includes a simulation unit 136. According to the configuration parameters of the physical object, a simulation is performed based on the three-dimensional grid unit. The configuration parameters include: physical object state configuration parameters, algorithm configuration parameters, data configuration parameters, and boundary configuration parameters.

[0083] For example, the data center 110 processes digital twin data of a chemical plant to realize pure vector digital expression of the chemical plant. The twin data service module 130 renders the chemical plant to realize visual expression and operation interaction. The simulation unit 136 receives configuration parameters about the chemical plant, including: setting dynamic parameters for the state of the factory fluid pipeline according to the physical object state configuration module, configuring pipeline leakage monitoring and pipeline fatigue point monitoring algorithms according to the algorithm configuration module, accessing and configuring the real-time sensor data of the pipeline based on the data configuration module, and setting boundary parameters such as pipeline flow, pressure, pipeline simulation nodes and range, simulation time, etc. Afterwards, the data-driven algorithm is run in units of three-dimensional grid units to realize pipeline fluid leakage prediction.

[0084] In addition, during the simulation process, the rendering engine 132 and the interactive unit 134 can be used to configure simulation parameters, as well as visualize the operation process, operation results, and other aspects and perform operational interactions.

[0085] According to the twin data service module 130 of the present application, the three-dimensional grid unit is used as the cell expressing the physical object, and the physical object state information is dynamically updated and interacted in real time. In addition, the grid code of the three-dimensional grid unit is used as the calculation unit. Because it is discrete and integer operation, the calculation complexity is much lower than the traditional longitude and latitude vector coordinate algorithm, and the calculation is more efficient and faster, generally several times or even dozens of times faster, and has nothing to do with the complexity of the physical object.

[0086] To summarize, according to the digital twin visualization system 100 of the present application, there is no need to make a three-dimensional model, and physical objects can be described and expressed with purely vectorized twin data, breaking through the bottlenecks of uniqueness, readability, scalability and correlation of large-scale multimodal twin data expression, and realizing efficient organization, fusion processing and real-time calculation of large-scale multimodal twin data. It solves or at least downgrades and alleviates the problems of traditional digital twin systems such as insufficient real-time performance, difficulty in integrating and standardizing large-scale multimodal data, limited interactivity, poor scalability, rendering performance bottleneck, low computability, and high implementation cost, which helps to improve the intelligence, maturity and large-scale application of digital twins.

[0087] The digital twin visualization system 100 according to the present application can be implemented by one or more computing devices. Figure 6FIG. 6 is a schematic diagram of a computing device 600 according to some embodiments of the present application. It should be noted that: Figure 6 The computing device 600 shown is only an example. In practice, the computing device used to implement the method of the present application can be any type of device, and its hardware configuration can be different from the above. Figure 6 The computing device 600 shown is the same as Figure 6 The computing device 600 shown is different. In practice, the computing device used to implement the embodiments of the present application can be Figure 6 The hardware components of the computing device 600 shown are increased or deleted, and the present application does not limit the specific hardware configuration of the computing device.

[0088] like Figure 6 In a basic configuration, the computing device 600 includes at least one processing unit 602 and a system memory 604. According to one aspect, depending on the configuration and type of the computing device, the processing unit 602 can be implemented as a processor. The system memory 604 includes, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, the system memory 604 includes an operating system 605 and program modules 606.

[0089] According to one aspect, operating system 605 is suitable, for example, for controlling the operation of computing device 600. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other application programs, and are not limited to any particular application or system. Figure 6 This basic configuration is illustrated in FIG. 6 by those components within dashed line 608. According to one aspect, computing device 600 has additional features or functionality. For example, according to one aspect, computing device 600 includes additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 6 6 is illustrated by removable storage 609 and non-removable storage 610.

[0090] As stated above, according to one aspect, program modules are stored in the system memory 604. According to one aspect, the program modules may include one or more application programs, and the present application does not limit the types of application programs, for example, the application programs may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slide show applications, painting or computer-aided applications, web browser applications, etc.

[0091] According to one aspect, the examples may be practiced on a circuit comprising discrete electronic components, a packaged or integrated electronic chip containing logic gates, a circuit utilizing a microprocessor, or a single chip containing electronic components or a microprocessor. Figure 6 Each or many components shown in can be integrated in a system on chip (SOC) on a single integrated circuit to practice examples. According to one aspect, such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") into a chip substrate as a single integrated circuit. When operated via SOC, the functions described in the present application can be operated via a dedicated logic integrated with other components of a computing device 600 on a single integrated circuit (chip). Other technologies that can perform logical operations (such as AND, OR, and NOT) can also be used to practice embodiments of the present application, including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, embodiments of the present application can be practiced in a general-purpose computer or in any other circuit or system.

[0092] According to one aspect, the computing device 600 may also have one or more input devices 612, such as a keyboard, a mouse, a pen, a voice input device, a touch input device, a VR motion capture input device, etc. Output devices 614 may also be included, such as a display, a speaker, a printer, etc. The aforementioned devices are examples and other devices may also be used. The computing device 600 may include one or more communication connections 616 that allow communication with other computing devices 618. Examples of suitable communication connections 616 include, but are not limited to: RF transmitters, receivers and / or transceiver circuits; Universal Serial Bus (USB), parallel and / or serial ports.

[0093] The term computer-readable medium as used in this application includes computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules). System memory 604, removable storage 609, and non-removable storage 610 are all examples of computer storage media (i.e., memory storage). Computer storage media may include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices, or any other products that can be used to store information and can be accessed by computing device 600. According to one aspect, any such computer storage medium can be part of computing device 600. Computer storage media do not include carrier waves or other propagated data signals.

[0094] According to one aspect, communication media is implemented by computer readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transport mechanism), and includes any information delivery media. According to one aspect, the term "modulated data signal" describes a signal that has one or more characteristics set or changed in a manner that encodes information in the signal. By way of example and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0095] The various techniques described herein may be implemented in combination with hardware or software, or a combination thereof. Thus, the method and apparatus of the present application, or certain aspects or portions of the method and apparatus of the present application may be in the form of program codes (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, a USB flash drive, a floppy disk, a CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes a device for practicing the present application.

[0096] In the case where the program code is executed on a programmable computer, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store the program code; the processor is configured to execute the method of the present application according to the instructions in the program code stored in the memory.

[0097] By way of example and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer readable instructions, data structures, program modules or other data. Communication media generally embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and include any information transfer medium. Any combination of the above is also included within the scope of readable media.

[0098] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the examples of the present application. According to the above description, it is obvious to construct the structure required for such systems. In addition, the present application is not directed to any specific programming language either. It should be understood that the content of the present application described herein can be realized by various programming languages, and the description of the above specific languages ​​is to disclose the preferred implementation of the present application.

[0099] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0100] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0101] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition they can be divided into multiple submodules or subunits or subassemblies. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise clearly stated, each feature disclosed in this specification can be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0102] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present application and to form different embodiments.

[0103] In addition, some of the embodiments are described herein as a combination of methods or method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices that are used to implement the functions performed by the elements for the purpose of implementing the disclosure.

[0104] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely refers to different instances of similar objects and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner. In addition, the quantifier "plurality" means "two" and / or "more than two."

Claims

1. A digital twin visualization system based on three-dimensional grid, characterized by: include: A plurality of data centers are arranged near a data source, adapted to obtain attribute data of each physical object from a neighboring data source, and associate the attribute data with a three-dimensional grid unit to generate a three-dimensional grid unit set of each physical object; and adapted to filter out a plurality of three-dimensional grid units for each physical object by filtering the corresponding three-dimensional grid unit set, and establish an adjacent relationship between the filtered three-dimensional grid units, wherein the adjacent relationship describes the structure of the corresponding physical object from the dimension of spatial relationship; A data processing module, adapted to determine the required number of three-dimensional grid units according to a data service request, extract and encode corresponding three-dimensional grid units from the data center, generate service response data and send it to the twin data service module; The twin data service module includes a rendering engine and an interaction unit, where: The rendering engine is adapted to decode the service response data, and use three-dimensional grid units as granularity and rendering objects, render according to attribute data in the three-dimensional grid units, and display the rendering effect through a visual interface; The interaction unit is adapted to identify and maintain the interactive spatial relationship based on the adjacent relationship between the three-dimensional grid units involved in the interaction instruction when executing the interaction instruction.

2. The system according to claim 1, characterized in that Each of the data centers comprises: The three-dimensional grid division unit is suitable for dividing the space range to obtain multi-level three-dimensional grid units, and each three-dimensional grid unit is encoded to obtain a unique grid code; The attribute data processing unit is adapted to aggregate and process the attribute data of the same physical object to obtain processed attribute data; A grid data association unit, adapted to determine a plurality of three-dimensional grid units for displaying each of the physical objects; and to associate and match each of the determined three-dimensional grid units with the processed attribute data of the corresponding physical object, to obtain a set of three-dimensional grid units of each of the physical objects; The grid data linking unit is adapted to select multiple three-dimensional grid units from each three-dimensional grid unit set according to a predetermined rule, and establish an adjacent relationship between each three-dimensional grid unit by sequentially linking the selected three-dimensional grid units.

3. The system according to claim 2, characterized in that The grid data association unit is further adapted to: According to the scale of the map showing each of the physical objects, a level of the three-dimensional grid unit is selected accordingly; According to the selected level, the coordinates of each corner point of each three-dimensional grid unit within the spatial range are correspondingly determined; Based on the coordinates, calculating the spatial position of the processed attribute data of each of the physical objects and the three-dimensional grid cells of the corresponding level to determine the three-dimensional grid cells corresponding to the processed attribute data; The processed attribute data is written in the corresponding three-dimensional grid unit in a manner of adding attribute fields, and after all processed attribute data of the physical object are written, the corresponding three-dimensional grid unit set of the physical object is obtained.

4. The system according to claim 3, characterized in that The grid data association unit is further adapted to: Based on the coding rule of the three-dimensional grid unit, the coordinates of the positioning point of the three-dimensional grid unit are determined, wherein the positioning point is the corner point closest to the starting position of the segmentation; The coordinates of other corner points of the three-dimensional grid unit are calculated according to the accuracy corresponding to the level of the three-dimensional grid unit and the coordinates of the positioning point.

5. The system according to claim 2, characterized in that The three-dimensional grid division unit is also suitable for dividing the spatial range. For a larger spatial range, the minimum accuracy of the three-dimensional grid unit is at the meter level, and the maximum accuracy is at the kilometer level; for a smaller spatial range, the minimum accuracy of the three-dimensional grid unit is at the centimeter level, and the maximum accuracy is at the ten-meter level.

6. The system according to claim 2, characterized in that The predetermined rules include: If the three-dimensional grid cell is not associated and matched, the three-dimensional grid cell is filtered out; For high-precision three-dimensional grid cells, the three-dimensional grid cells are thinned based on the attribute data associated therewith to obtain thinned three-dimensional grid cells; For the three-dimensional grid cells where the physical object only occupies part of the space, the occupied part of the three-dimensional grid cells is retained by clipping.

7. The system according to claim 2, characterized in that The grid data linking unit is also adapted to: The selected three-dimensional grid units are linked in combination with the structural description logic sequence of the physical object and the coding sequence of the grid code, wherein the structural description logic sequence of the physical object includes: from macroscopic to microscopic, from whole to part; According to the order in which the links point to, an adjacent relationship between the screened three-dimensional grid units is established.

8. The system of claim 1, wherein: The data service request includes the current map scale, the visible range of the screen, and the physical objects within the visible range of the screen. The data processing module is also adapted to determine the level of the three-dimensional grid unit according to the current map scale, and calculate the number of three-dimensional grid units required within the visible range of the screen.

9. The system according to claim 1, characterized in that The interaction unit is also suitable for feeding back changes in attribute data occurring during the interaction operation to the physical object, thereby realizing virtual-reality interaction.

10. The system according to claim 1, wherein: The twin data service module also includes: A simulation unit is adapted to perform simulation based on a three-dimensional grid unit according to configuration parameters of the physical object, The configuration parameters include: physical object state configuration parameters, algorithm configuration parameters, data configuration parameters, and boundary configuration parameters.

11. The system according to claim 2, characterized in that The attribute data processing unit is also adapted to: Aligning the geographical location information and time information of the attribute data; Use interpolation algorithms to fill in missing data points; Identify abnormal data values ​​and correct them based on historical data or adjacent data points; Detect and remove duplicate data through hash mapping.

12. The system according to claim 2, characterized in that The attribute data processing unit is also suitable for storing the processed attribute data in a data lake architecture.

Images