Yellow River simulator system based on space intelligence and domestic supercomputing and construction method

Abstract

The invention discloses a Yellow River simulator system based on space intelligence and domestic supercomputing and a construction method. The Yellow River basin intelligent simulation system covers links such as data perception convergence, storage management, intelligent simulation, calculation acceleration and platform construction, and aims to realize global intelligent simulation of the Yellow River basin and provide'one-stop 'intelligent service through organic fusion and collaborative application of the technologies, thereby effectively overcoming defects and deficiencies in the prior art. Through a data active discovery technology driven by double engines and an automatic convergence method under a three-dimensional net-shaped directory system, the limitation of traditional data acquisition is broken through, efficient integration of multi-source and multi-scale observation data is realized, a standardized data resource pool is formed, a comprehensive and high-quality data basis is provided for simulation and analysis of the Yellow River basin, and the Yellow River basin simulation and analysis efficiency is improved. The problems of watershed data dispersion, format isomerism and difficulty in cooperative utilization are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent simulation of the whole Yellow River Basin, and particularly relates to a Yellow River simulator system based on spatial intelligence and domestic supercomputing and a construction method thereof. BACKGROUND

[0002] As an important ecological barrier and economic zone of China, the Yellow River Basin has attracted much attention in terms of ecological protection, water resource management, disaster prevention and control, and sustainable development. With the rapid development of information technology, using digital and intelligent means to simulate and manage the whole Yellow River Basin has become a core direction to improve the governance capacity of the basin. However, the Yellow River Basin has characteristics such as wide geographical span, complex natural geographical conditions, fragile ecological environment, diverse hydro-meteorological changes, and far-reaching human activities, which bring great challenges to the fine simulation and intelligent management of the basin.

[0003] On the data level, the observation data of the Yellow River Basin is sourced from a wide range of channels, including satellite remote sensing, unmanned aerial photography, ground monitoring stations, Internet of Things sensors, and cameras, forming a comprehensive observation data of multiple sources, multiple scales, and massive amounts. However, in the existing technology, these data are often scattered and stored in different departments and platforms, resulting in a serious "data island" phenomenon and a prominent data fragmentation problem, making it extremely difficult to actively discover and aggregate data. At the same time, due to the complexity of data structure, significant differences in time and space scales, and weak element correlation, traditional data management methods cannot efficiently store, organize, and retrieve multi-source heterogeneous data, greatly restricting the full play of data value.

[0004] On the knowledge modeling and simulation level, the simulation of the Yellow River Basin involves hydrology and water resources, ecological environment, geology and geomorphology, meteorology and climate, and social economy, requiring the deep integration and collaboration of multi-disciplinary knowledge. However, the existing technology lacks an effective integration mechanism for multi-field knowledge, making it difficult to build a comprehensive and accurate basin knowledge system, resulting in weak multi-field knowledge collaboration simulation capabilities and an inability to accurately depict the complex ecological hydrological processes, disaster evolution laws, and human activity impact mechanisms of the Yellow River Basin. In addition, for complex scenario simulation of the Yellow River Basin, such as flood evolution, soil erosion, and water quality changes, existing simulation technologies often face the problems of large computation and long time consumption, making it difficult to meet the needs of real-time dynamic simulation and rapid decision support.

[0005] In the computing support layer, the rapid development of domestic supercomputers provides a strong hardware foundation for large-scale simulation of the Yellow River Basin, and its heterogeneous many-core architecture has high computing potential. However, how to fully utilize the performance advantages of domestic supercomputers and adapt to the diverse simulation scenario requirements of the Yellow River Basin is still a key problem. In the existing technology, there is a lack of reasonable computing power allocation mechanism for different simulation scenarios of the Yellow River Basin, and the cooperative computing efficiency between heterogeneous computing units such as central processing units and graphics processing units is low, resulting in low efficiency of data processing and complex model calculation. In the global large-scale simulation, the robustness of the calculation is difficult to guarantee, which seriously affects the reliability and timeliness of the simulation results.

[0006] In the platform construction layer, the existing simulation platforms related to the Yellow River Basin mostly focus on single function or single field application, and lack of integration of data perception aggregation, storage management, knowledge modeling, simulation calculation, decision service and other whole-process integration. The platform integration degree is insufficient. When users carry out basin simulation and management, they need to switch between multiple platforms, which is difficult to realize "one-stop" intelligent service, greatly reducing the work efficiency of basin governance.

[0007] In summary, the existing Yellow River Basin simulation technology has many key technical problems in data management, knowledge modeling, computing support and platform construction, and cannot meet the needs of the whole Yellow River Basin intelligent simulation and efficient management. SUMMARY

[0008] The purpose of the present application is to provide a systematic technical solution covering data perception aggregation, storage management, intelligent simulation, computing acceleration and platform construction, and to construct an efficient, accurate and integrated Yellow River simulator system and construction method based on spatial intelligence and domestic supercomputers.

[0009] The application aims to achieve the following: a construction method of a Yellow River simulator system based on spatial intelligence and domestic supercomputing, comprising the following steps: S1, constructing a data aggregation and intelligent management module, through a Yellow River basin comprehensive observation data active aggregation and intelligent management module, to realize active discovery, automatic aggregation and storage management of multi-source multi-scale comprehensive observation data, and to provide a data basis for the construction of the Yellow River simulator system; S2, constructing a domain knowledge reasoning and modeling module, based on the observation data obtained in S1, performing knowledge accurate extraction on multi-modal data, combining a Yellow River basin domain cross-modal knowledge fusion and reasoning method, and constructing a Yellow River basin complex scene domain knowledge intelligent model; S3, constructing a hardware and software integrated intelligent simulation module in a hybrid cloud computing environment of domestic supercomputing, combining the Yellow River basin complex scene domain knowledge intelligent model constructed in S2, designing a parallel computing module and an acceleration module of a many-core architecture, to realize high-precision and high-efficiency simulation calculation of a Yellow River basin complex coupled system; S4, constructing a Yellow River simulator platform based on a micro-service architecture, the platform comprising a data processing module, a calculation simulation module, a knowledge cognition module, a system service module and a visualization presentation module; the platform is used to simulate a super-large scale space-time scene on the basis of completing simulation calculation.

[0010] In S1, the active discovery method of multi-source multi-scale comprehensive observation data comprises the following steps: A "double-engine driven" data active discovery module is constructed, specifically as follows: (1) On the basis of a traditional vector space model, taking spatial place names and ordinary keywords as independent variables, mapping themes and data contents to a spatial vector space and an ordinary theme vector space composed of place name and ordinary keyword weights, to construct a multi-source data keyword extraction module; (2) Through calculation of time, space and theme correlation, a natural exponential function is used to realize data address priority allocation, and then the correlation value is used to schedule search priority, and a double-layer optimization method of surface data discovery and deep search is used to carry out search until the priority queue is empty, to construct a soft perception keyword search module; (3) Constructing a typical element extraction module of long-time series comprehensive observation data, specifically as follows: A time feature learning module is constructed by using a long short-term memory network to encode, at the same time, the feature vectors of different input time series are positionally encoded by using a sine-cosine function, and a self-attention mechanism is used to adaptively learn the spatial feature weight distribution, to construct a typical element extraction module of long-time series comprehensive observation data. (4) Constructing a typical element extraction module of high spatial resolution data, specifically as follows: From three different cognitive scales of pixel level, object level and target level, a method of discriminative feature learning and typical element extraction of high-dimensional data is established, and a mutual knowledge distillation strategy is adopted to realize the maximization of pixel-object-target feature representation at different levels, so as to construct a typical element extraction module of high spatial resolution data.

[0011] An automatic aggregation module of comprehensive observation data at basin scale is constructed, which is as follows: (1) Construct a three-dimensional network directory system of comprehensive observation data On the basis of existing directory system and standards, the classification and grading information of labeled observation data is used to reconstruct the original directory hierarchy relationship by using three-dimensional label network, and the multi-dimensional mapping mechanism is used to compatible the data organization structure of each center, so as to construct a three-dimensional network directory system of comprehensive observation data; (2) Establish a "time-space-theme" correlation module of comprehensive observation data The basic characteristics of mixed network are described by the distance measurement method, connectivity measurement method and centrality analysis method of complex network; the "community" with close relationship in mixed network is identified by community discovery algorithm; finally, the aggregation and hierarchy of the network are analyzed by using network clustering coefficient and hierarchical modularity, and the topological structure evolution of the whole network is analyzed, and the "time-space-theme" correlation module is established. (3) Construct a comprehensive observation data aggregation module From the aspects of starting time, general keywords and source keywords, the theme similarity calculation strategy considering starting time and the theme similarity calculation strategy based on bidirectional vector are adopted, and the time similarity, general theme similarity and spatial similarity of theme and data content are calculated based on multi-vector theme representation method, so as to construct a comprehensive observation data aggregation module.

[0012] A storage organization and management module of comprehensive observation data is constructed, which is as follows: (1) Construct a distributed storage model of comprehensive observation data based on multi-dimensional slicing mechanism, which is as follows: By constructing a distributed metadata organization method, metadata intelligent division mechanism and multi-copy consistency model, a storage organization and management module of observation data is constructed; (2) Construct a data dynamic organization model based on multi-dimensional three-dimensional sectioning, which is as follows: The time and space and use frequency of comprehensive observation data in the Yellow River Basin are calculated, each spatial data is given a unique multi-dimensional sectioning identifier of observation results based on sectioning grid, and a data dynamic organization model containing time and data hotness is constructed based on the multi-dimensional division scheme constructed by geographic sectioning grid; (3) Construct an intelligent retrieval model of comprehensive observation data based on asymmetric hash, which is as follows: A lightweight network FasterNet is introduced in the feature extraction stage, and an optimized structure embedding module is added in the backbone network; a comprehensive observation data intelligent retrieval model is constructed by using partial convolution PConv to apply convolution operation to part of the channels of the input feature map, and the characteristics of the remaining channels remain unchanged.

[0013] In S2, the specific process of the domain knowledge reasoning and modeling module is as follows: A WEAP-based water resources allocation model of the Yellow River Basin is constructed, a graph neural network-based temperature prediction model of the Yellow River Basin is proposed, a Bayesian-based rainfall simulation method of the Yellow River Basin is constructed, remote sensing images and deep learning technologies are deeply integrated, and domain knowledge modeling models in the fields of hydrology and water resources, basin water cycle, urban ecological-economic balanced development simulation, and major natural disasters are respectively established to form the domain knowledge reasoning and modeling module.

[0014] In S3, the implementation method of the simulator parallel scheduling module is as follows: by constructing a data scheduling module, a domestic many-core parallel computing module, and a simulator dynamic acceleration module, the parallel scheduling module of the simulator is formed, wherein: According to the spatiotemporal characteristics of data, massive remote sensing data is processed by slicing, and the data blocks are reasonably divided considering the spatiotemporal distribution characteristics and dynamic division; at the same time, based on the central data pool technology, the sliced data is distributedly mapped and loaded to construct the data scheduling module; The multi-source prior semantic knowledge in the data is mined, semantic deep expression is realized through multi-feature fusion, and the cost-sensitive learning is combined with the distributed multi-machine collaborative extraction model to construct a parallel computing module adapted to the domestic many-core architecture; The resources are processed in batches to reduce the data storage load of the work nodes, and the MD-LAMB algorithm is converted to SGDM in the early training stage to construct the simulator dynamic acceleration module.

[0015] The specific process of S4 is as follows: (1) Build a micro-service framework to service-encapsulate the data processing module, the calculation simulation module, the knowledge cognition module, the system service module, and the visualization presentation module; (2) In the data processing module, multi-source spatiotemporal data is collected, and data cleaning, format standardization, spatiotemporal registration, and feature extraction are sequentially performed; (3) In the calculation simulation module, based on the distributed parallel computing scheduling engine, multi-scale numerical simulation of hydrological, hydraulic, and geographical processes is performed; (4) In the knowledge cognition module, a semantic knowledge base is constructed, and a reasoning algorithm is used to perform semantic labeling and scenario analysis on the simulation results; (5) in the system service module, the task scheduling, the computing resource allocation and the message communication between microservices are realized; (6) in the visual presentation module, the simulation result is spatialized and displayed by using a three-dimensional rendering engine, and an interactive operation function is realized.

[0016] A Yellow River simulator system based on spatial intelligence and domestic supercomputing includes a data aggregation and intelligent management module, a domain knowledge reasoning and modeling module, a hardware and software integrated intelligent simulation module in a hybrid cloud computing environment for domestic supercomputing, and a Yellow River simulator platform based on a microservice architecture.

[0017] The comprehensive observation data active aggregation and intelligent management module is embedded in the data layer of the Yellow River simulator system, and is used for realizing active discovery, automatic aggregation and efficient storage management of multi-source and multi-scale comprehensive observation data, and providing data support for the Yellow River simulator system. The domain knowledge reasoning and modeling module is deployed in the platform cognitive layer, is used for accurately extracting knowledge from multi-modal data, combining cross-modal knowledge fusion and reasoning technology, and constructing a domain knowledge intelligent model of a complex scene in the Yellow River basin, and provides knowledge support for the Yellow River simulator system. The hardware and software integrated intelligent simulation module in the hybrid cloud computing environment for domestic supercomputing is integrated in the platform computing layer, is used for realizing efficient simulation of a large-scale complex scene in the Yellow River basin through a parallel computing module and an acceleration module of a many-core architecture, and provides computing support for the Yellow River simulator system. The Yellow River simulator platform based on the microservice architecture integrates the above modules into the Yellow River simulator platform with full-link functions of data, computing, cognition, service and presentation, realizes real-time dynamic simulation of a super-large scale space-time scene in the Yellow River basin and one-stop intelligent service, and realizes real-time dynamic simulation of a super-large scale space-time scene in the Yellow River basin and one-stop intelligent service. The Yellow River simulator platform includes a user interaction interface, a data interaction interface, a model calling interface and a service output interface, supports multi-terminal access and third-party system integration, and realizes visual display of simulation results and directional pushing of decision suggestions.

[0018] The Yellow River simulator platform based on the microservice architecture integrates the above modules into the Yellow River simulator platform with full-link functions of data, computing, cognition, service and presentation, realizes real-time dynamic simulation of a super-large scale space-time scene in the Yellow River basin and one-stop intelligent service, and realizes real-time dynamic simulation of a super-large scale space-time scene in the Yellow River basin and one-stop intelligent service. 2. Deep optimization of knowledge service level: The invention relies on multi-modal data knowledge precise extraction and cross-modal fusion reasoning technology, builds a knowledge intelligent model for complex scene fields in the Yellow River Basin, converts fragmented data into systematic knowledge, breaks through the limitations of traditional mathematical methods in complex system analysis, provides accurate knowledge support and reasoning rules for basin management decision-making, and improves the scientificity and intelligent level of decision-making; 3. Greatly enhanced computing efficiency and adaptability: The invention is based on the hybrid cloud computing environment of domestic supercomputers, and through the parallel computing module and acceleration module of the many-core architecture, it fully utilizes the computing potential of the domestic heterogeneous many-core architecture, realizes efficient simulation of large-scale complex scenes in the Yellow River Basin, guarantees the response speed of global large-scale scene simulation, improves the utilization rate of computing resources, provides strong computing support for super-large scale space-time scene simulation, and is adapted to the domestic supercomputing system, with the advantage of independent controllability; 4. Comprehensive integration of platform service functions: The invention integrates the deployment of the five-layer engine system of "data-computation-cognition-service-presentation", builds an integrated platform with "one-stop" intelligent service capability, realizes seamless connection of data interaction, simulation calculation, decision support and other functions, supports multi-terminal access and third-party system integration, can push simulation results and decision recommendations, provides convenient and efficient comprehensive solutions for ecological protection, water resources management, disaster prevention and control in the Yellow River Basin, and promotes the transformation of basin management from experience-driven to data and intelligent-driven. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A schematic diagram of the mixed association network of the invention is shown; Figure 2 A schematic diagram of the distributed meta-database organization of the invention is shown; Figure 3 A technical roadmap of the data division rule of the meta-data of the invention is shown; Figure 4 A data migration model diagram of the comprehensive observation result data of the invention is shown; Figure 5 A schematic diagram of the fast asymmetric hash method of the invention is shown; Figure 6 A schematic diagram of the data division of the gRPC distributed training of the invention is shown; Figure 7 A data parallel processing flowchart of the invention is shown; Figure 8 A data parallel processing flowchart of the invention is shown. DETAILED DESCRIPTION

[0020] The invention is further described below in conjunction with the drawings and examples.

[0021] A method for constructing a Yellow River simulator system based on spatial intelligence and domestic supercomputing, comprising the following steps: S1, constructing a data aggregation and intelligent management module, through the Yellow River Basin massive comprehensive observation data active aggregation and intelligent management module, to realize the active discovery, automatic aggregation and efficient storage management of multi-source multi-scale massive comprehensive observation data, and provide data basis for the construction of the Yellow River simulator system; the specific steps are as follows: Based on the traditional vector space model, the spatial place name and the general keyword (the keyword except the place name) are taken as independent variables, and the theme and the data content are uniformly mapped to the spatial vector space and the general theme vector space composed of the place name and the general keyword weight. Given theme T and data content D, T = <V Tk , V Tp > and D = <V Dk , V Dp >, V Tk and V Tp represent theme general vector and theme spatial vector respectively; V Dk and V Dp represent data content general vector and data address vector respectively. Theme general vector V Tk and spatial vector V Tp are V Tk = {(k1, w Tk1 ), (k2, w Tk2 ), …, (k s , w Tks )} and V Tp = {(p1, w Tp1 ), (p2, w Tp2 ), …, (p m , w Tpm )} respectively. k i represents the i-th general keyword in the theme; p i represents the i-th place name in the theme; w Tki represents the weight of the general keyword k i ; w Tpi represents the weight of the place name p i ; m represents the number of place names in the theme; s represents the number of general keywords in the theme.

[0022] When constructing the soft perception keyword search module, the natural exponential function is integrated to realize the priority allocation of data addresses by calculating time, space and theme relevance; the search priority is scheduled based on the relevance value, and the double-layer optimization method combining surface data discovery and deep search (including data keyword identification and data content understanding) is adopted to carry out search until the priority queue is empty. The data recognition work is completed by support vector machine data keyword classification algorithm; when the sample is small and the type is fixed, the rule matching or keyword matching method is used to identify the data keyword.

[0023] The time feature learning module is constructed to encode the time correlation in multi-temporal images. Specifically, the time correlation in multi-temporal images is encoded by using the time correlation in multi-temporal images. As sequence data, it is assumed that is the prediction probability of the i-th prediction unit, according to the Bayesian chain rule: And the long short-term memory network is used to model the time relationship in the growth process of crops. To embed the spatial position information of data, the K-means algorithm is further used to cluster the data in space: Where C k represents the k-th class cluster, and |C k | represents the number of data objects in the k-th class cluster. And by using the self-attention mechanism to adaptively learn the spatio-temporal feature weight distribution, the joint modeling of spatio-temporal heterogeneous features under different computing nodes is realized in a dynamic weighting manner. By concatenating the attention results obtained by different heads, the attention matrix oriented to multiple computing nodes is finally realized: MultiHead(G)&=Concat(heαd1,…,head h ),head k =Att(Q k ,K k ), Where Q k represents the query matrix of the k-th attention head, K k represents the key matrix of the k-th attention head, represents the key weight matrix of the k-th attention head.

[0024] The discriminative feature learning and typical element extraction method of high-dimensional data is established from three different cognitive scales of pixel level, object level and target level.

[0025] The convolution layer is used as the basic feature extraction component to perform spectral feature extraction operation on remote sensing images pixel by pixel. The output result of each layer in the convolution architecture is: Where i, M represent the feature map, (I-1) is the network layer. w, b are the weight and bias parameters of the network, and f is the convolution operation.

[0026] SLIC algorithm is used to segment the original image to generate a superpixel set, and then graph structures are constructed from two different perspectives of topology and spectrum.

[0027] The graph nodes with high similarity are further aggregated into a target node, and the higher the node weight, the higher the similarity. In order to identify NC nodes, a node degree variance index (DVI) is designed to measure the local centrality of the graph, and the DVI calculation method is as follows: Nei(v i )={v j |e ij ≠0} is the neighborhood set of node v i , φ(Nei(v i )) is the adjacent node degree of node v i , |Nei(v i )| represents the number of neighbor nodes, and Max is the value of the maximum graph node degree. Finally, the pixel-object-target feature representation maximization is realized by using the mutual knowledge distillation strategy.

[0028] Further, an automatic aggregation module of basin scale comprehensive observation data is constructed; On the basis of existing directory system and national and industry standards, the classification and grading information of labeled observation data is used to reconstruct the original directory hierarchy relationship by using the three-dimensional label network, and the multi-dimensional mapping mechanism is used to compatible the data organization structure of each center, and the resource directory system of each center data sharing is unified.

[0029] Firstly, the KP-Miner label extraction model is used for directory to label conversion. Based on the label extraction result, the label disambiguation algorithm based on rules, knowledge base, instance or statistics is used to extract the features of the label and fuse and refine the similar labels, and the label is uniformly stored, which provides the basis for the unified description, expression and storage of observation data.

[0030] The Quillia label disambiguation method is used to improve the quality of the label. This method determines the classification score of the ambiguous word according to the cluster formed by the surrounding words of the ambiguous word, and determines the classification of the ambiguous word through the final score. If the coexisting features are found in the context and these features are in the same cluster w, then w is selected as an instance.

[0031] According to the label category, the data is classified, and then the data is encoded according to the degree, grade, second, second below, time, forming a unified coding system of massive spatio-temporal data in the Yellow River Basin.

[0032] Spatial data is categorized into three types: points, lines, and polygons. Point data can be directly encoded using latitude and longitude to obtain its multidimensional mesh-based encoding. Line data can be encoded by converting it to polygon data using its minimum bounding rectangle (MBR). Polygon data is encoded by calculating the appropriate level based on its MBR, then determining the corresponding mesh area using spatial location information, thus obtaining the multidimensional mesh-based encoding of the spatial data.

[0033] Furthermore, a "time-space-theme" association module for comprehensive observation data is established. Algorithms such as distance measurement, connectivity measurement, and centrality analysis are used to describe the basic characteristics of the hybrid association network. Community detection algorithms are employed to identify closely related "communities" within the hybrid association network. Finally, network clustering coefficients and hierarchical modularity are used to analyze the clustering and hierarchy of the association network, and the evolution of the overall network topology is analyzed.

[0034] Let G(V,E) represent a mixed association network of observation data, where V is the set of nodes in the network, representing the associated outcome data, and E is the set of edges in the network, where each element e in E represents an edge. ij =(v i v j ), where v i v j ∈V; where v ij =v ji This indicates that the edges in the network are undirected, meaning the relationships between the resulting data are undirected; e ij The composite correlation degree between results in the network is calculated by combining spatial, temporal, and thematic correlation degrees according to certain rules, such as... Figure 1 As shown.

[0035] Furthermore, a comprehensive observation data aggregation module is constructed. In the traditional vector space model, starting from three aspects—start time, common keywords, and source keywords—and based on the idea of ​​simultaneously calculating the similarity of common query terms and spatial similarity in geographic information retrieval, a topic similarity calculation strategy that takes into account the start time and a topic similarity calculation strategy based on dual vectors are adopted. Specifically, based on the multi-vector topic representation method, the temporal similarity, common topic similarity, and spatial similarity of topic and data content are calculated separately.

[0036] The weights of the redundant dimensions corresponding to place names are transferred to the weights of other place names through spatial topological relationships, so as to form a vector with the same dimension as the topic place name vector. This vector is then applied to the cosine formula to represent the place name similarity matching value between the topic and the webpage.

[0037] Further, a comprehensive observation data efficient storage organization and management module is constructed; in the distributed meta database, XML, JSIN and other extensible data description modes are adopted, and binary stream is adopted for physical storage. After determining the storage structure of the metadata, the organization architecture of the distributed meta database needs to be determined, such as Figure 2 As shown in the organization architecture of the meta database in the distributed environment, each metadata has two copies of master and backup, wherein the master database (M) is used to provide metadata writing service, and the slave database (S) is mainly used to provide metadata reading service, so as to realize read-write separation during data access and improve the concurrent service capability of the system.

[0038] Based on the one-dimensional data division rule, a field in the metadata structure is used as the division basis, such as the storage time, administrative region, collection equipment and other fields of the observation results.

[0039] When the storage object itself does not have such a field, a new field needs to be constructed or the rotation method is used to store the data objects in each service node in turn according to the storage order. In the distributed environment with homogeneous performance of storage and computing nodes, good load balancing is achieved.

[0040] The two-dimensional data division rule is a spatial data division method for obtaining the storage position by operating two fields, such as in the aspect of vector data: for point elements such as GPS data, the spatial code of the point is calculated by the longitude and latitude of each point, and for surface elements and linear elements, the spatial code value is calculated by the center point of each element; in the aspect of image data: for image metadata containing four-to-position information, the spatial code of each image or tile is calculated by the center point expression; the two-dimensional database division rule adopts Peano and Hilbert filling curves in the aspect of spatial code.

[0041] In the data result calculation stage, virtual nodes are used to replace physical nodes, each virtual node corresponds to only one physical node, and one physical node can correspond to multiple physical nodes according to its computing performance or storage performance. When the data volume is huge and the pressure of the distributed database storage space is greater than the load pressure of the computing resources, the virtual nodes should be generated according to the storage performance such as the storage space of the hard disk or magnetic tape, and vice versa. The calculation formula of the virtual nodes corresponding to each physical node is: Where S is the performance index of the physical node, and U is the set basic load parameter, which has the same performance index unit as S, such as when the storage performance is used as the index, U can be set to 100G, so for a physical node containing 500G storage space, the corresponding amount of its virtual node is 5. The technical route of the data division rule of the metadata is shown in Figure 3 .

[0042] And the Raft-based metadata multi-copy consistency solution solves the need to implement multi-copy consistency in distributed environments in sharding replication.

[0043] Further, a multi-dimensional stereoscopic subdivision-based data dynamic organization model is constructed. By calculating the space-time and frequency of use of the comprehensive observation data of the Yellow River Basin, each spatial data is assigned a unique multi-dimensional subdivision identifier of the observation results based on the subdivision grid.

[0044] The average access interval of the file is calculated. If the file has not been accessed before, its average access interval is set to INFINITE; if the file is accessed for the second time, its average access interval is the current access interval, otherwise the current access interval and the old average access interval are weighted and summed according to the forgetting factor α to obtain the new average access interval.

[0045] The upgrade benefit of the file is calculated. Let access num, access bytes, filesize, avg interval represent the total number of accesses, the total number of access bytes, the file size and the average access interval of the file respectively, and the upgrade benefit of the file after T time is expressed as: represents the expected number of accesses of the file within T time, represents the average number of bytes accessed by the file each time.

[0046] A multi-dimensional subdivision model containing time and data heat is constructed, and through three times of earth expansion (expanding the earth to 512°x512°, expanding 1° to 64′, and expanding 1′ to 64″), a 4-way tree subdivision is realized, forming a multi-scale 4-way tree network from the earth (0 level) to the centimeter-level grid (32 level); on each level grid, further subdivision is performed according to time, with year, month and day as the basic unit to realize coarse division of the comprehensive observation data of the Yellow River Basin, and with seconds as the unit to realize further accurate division; on the spatial and temporal level, multi-dimensional division such as data heat latitude is provided.

[0047] A multi-dimensional subdivision logical index file based on the subdivision grid is generated for each spatial data to represent the internal information of the spatial data file. A logical subdivision file is formed within each image, and without changing the current image data organization mode, the remote sensing image data is virtually divided according to the subdivision grid spatial range, i.e. logical subdivision, by positioning the geographic coordinates of the four corner points of the subdivision grid-based multi-dimensional subdivision grid unit of the mass comprehensive observation results and the geographic coordinates of the remote sensing data, to establish a subdivision index of the remote sensing image data based on the subdivision grid coding.

[0048] Depend on Figure 4 It is known that disk saturation is the trigger condition for migration. When disk saturation reaches the threshold ε, the migration algorithm is triggered to judge the data according to its time attribute. Data that is first accessed at the current time (T1 > Y) is considered historical data. It is then determined whether the data belongs to the region of interest. If the data file l(X) = 1, the batch of data is directly stored on the primary storage medium; otherwise, the next step is executed. The data migration coefficient is calculated according to the migration function. If the data migration coefficient is greater than the threshold δ, data migration is performed. Let Y be the time distance between the data and the current time. When the time attribute of the data exceeds time Y, the batch of data is considered historical data. Let the access time of each access to the massive comprehensive observation results data file X from the start of storage be t1, t2, ..., tn, and the time distance between these time points and the current time t be t-t1, t-t2, ..., t-ti, ..., t-tn. Let the above time lengths be T1, T2, ..., Ti, ..., Tn.

[0049] During data application, the access frequency of data is constantly changing, and therefore the value of the data changes accordingly. Let fi be the access frequency of data within each Ti, then the access frequency of data within the time period Ti-Ti-1 is fi-fi-1. Whether data migration is necessary first requires considering the amount of remaining storage space. Because Ti-1 storage media has high performance, high price, and limited scalability, sufficient storage space must be reserved to handle the normal operation of storing and accessing new data. Let S0 be the disk's storage capacity, and S be the current disk space occupied. Then S / S0 represents the current disk saturation. A threshold ε is set; when S / S0 > ε, data migration is performed.

[0050] Furthermore, a comprehensive observation data intelligent retrieval model based on asymmetric hashing is constructed; such as... Figure 5 As shown, a lightweight network, FasterNet, is introduced in the feature extraction stage. Partial convolution PConv is used to apply convolution operations to some channels of the input feature map, while the characteristics of the remaining channels remain unchanged, effectively reducing computational redundancy and improving computational efficiency. Furthermore, an optimized structure embedding module is embedded into the FasterNet backbone network to achieve efficient extraction of deep features while capturing the internal structure of the image to enhance the expression of structural information, thereby improving the feature representation capability of the hash code and further improving retrieval accuracy. Finally, a linear classifier is introduced to capture the semantic features of image labels and combine them with asymmetric similarity information to generate more discriminative image hash codes, achieving effective improvement in similarity ranking.

[0051] The last three layers of FasterStruct are structure embedding module SSM, adaptive average pooling and Conv 1x1. Among them, the main task of the structure embedding module is to capture the local and global structure information in the image, thereby enhancing the network's understanding ability of image details and geometric structure and improving the image expression ability of the features. The adaptive average pooling (AAP) performs spatial dimension reduction on the feature map, maps the input features of any size to a fixed dimension, is suitable for input data of different resolutions, and enhances the adaptability of the model. Conv 1x1 further integrates and compresses the feature dimension in the final output layer, ensures the compactness of the feature expression, reduces the calculation overhead, and provides more recognizable feature representation for subsequent image classification and retrieval.

[0052] The structure embedding network module mainly consists of three sub-modules, namely self-similarity calculation module, self-similarity encoding module and feature fusion module. The self-similarity calculation module is mainly to reduce the computational complexity between channels. Before calculation, the original feature F ∈ R C×H×W is mapped to the feature map F' ∈ R C×H×W through linear layer operation. The self-similarity encoding module is composed of a series of convolution blocks such as convolution, batch normalization layer and ReLU function, which encodes the channel non-negative self-similarity descriptor S into a dense self-similarity descriptor D ∈ R C×H×W with the same size as the initial feature map F ∈ R C×H×W . The feature fusion module adjusts the contribution proportion of the original feature F and the self-similarity descriptor D by dynamically generating weights α and 1-α through a dynamic weight module DWG (Dynamic Weight Generator), so as to realize adaptive weighted fusion of features.

[0053] S2: Construction of domain knowledge reasoning and modeling module, based on the observation data obtained in S1, precise knowledge extraction is performed on the multi-modal data, combined with the cross-modal knowledge fusion and reasoning method in the Yellow River Basin field, an intelligent model of field knowledge in the complex scene of the Yellow River Basin is constructed.

[0054] The construction process of the multi-modal domain knowledge reasoning and modeling module is as follows: first, a multi-modal domain knowledge alignment method based on a contrast learning mechanism is adopted, multi-modal named entity recognition is realized through fine-grained text-image alignment, and a bilinear attention mechanism is used to complete the fusion of multi-modal knowledge to ensure the consistency of the representations between different modalities. On this basis, a contrast learning strategy is introduced to construct a unified feature space between different modalities, realize automatic alignment of shared knowledge and field features, and further reduce the semantic difference between modalities.

[0055] Secondly, a cross-modal domain knowledge fusion method based on dynamic association graph is adopted. The CasRel model is used to establish the semantic association between cross-modal domain knowledge. The topological relationship between nodes and edges in the graph is used to associate different modal data in order, effectively alleviating the inconsistency of information between modalities and overcoming the semantic gap of multi-modalities.

[0056] Further, combined with the deep coupling mechanism model and deep learning algorithm, a variety of reasoning and modeling methods are designed for the complex scene of the Yellow River Basin. Among them, a reasoning mechanism based on bilinear graph neural network is adopted, which optimizes the information transmission path and reasoning process by introducing bilinear mapping, improving the reasoning accuracy and efficiency of the model. At the same time, combined with fact-based reasoning and counterfactual reasoning, a knowledge reasoning explanation module is constructed, which generates potential contrast scenarios and possible reasoning results using counterfactual reasoning, thereby enhancing the explainability and reliability of the reasoning results.

[0057] The complex scene domain knowledge intelligent model includes hydrology and water resources model, Yellow River Basin rainfall simulation model, urban ecological-economic balanced development model, etc.

[0058] The hydrology and water resources model divides the optimization allocation goal of water resources in the Yellow River Basin into three aspects: economic benefit, social benefit, and ecological benefit.

[0059] The problem of water resources optimization allocation is transformed into a multi-objective decision-making mathematical problem, and its general mathematical expression is: Max(min)P=={f1(x), f2(x),..., f n (x)} T . Among them, f n (x) is the objective function, which represents the economic, social, and ecological indicators in the optimization allocation of water resources in the Yellow River Basin, and is the measurement value of the optimization allocation scheme. L is the feasible region of decision variables, which is composed of the constraints in the real water resources system.

[0060] The economic benefit objective function is: E(x) is the economic goal, n is the number of WEAP partitions; k(n) is the number of water demand points in the nth partition; j(n) is the number of independent water sources in the nth partition; i is the number of public water sources; x jkn is the water supply of the jth independent water source to the nth partition k demand point; x ink is the water supply of the ith public water source to the nth partition k demand point; m jkn is the unit output value of the jth independent water source to the nth partition k demand point; m ink is the unit output value of the ith public water source to the nth partition k demand point.

[0061] The social benefit objective function is: In the formula, S(x) is a social goal, n is the number of WEAP partitions; k(n) is the number of water demand points in the nth partition; j(n) is the number of independent water sources in the nth partition; i is the number of public water sources; x jkn is the water supply of the jth independent water source to the kth demand point in the nth partition; x ink is the water supply of the ith public water source to the kth demand point in the nth partition; D nk is the total water demand in the region.

[0062] The ecological benefit objective function is: In the formula, C(x) is an ecological goal, n is the number of WEAP partitions; a(n) is the number of forest, grassland, and other ecological water demand points in the nth partition; c an represents the water demand of the forest, grassland, and other ecological water demand points in the nth partition; a(n) is the number of lake and river ecological landscape water demand points in the nth partition; c bn is the water demand of the lake and river ecological landscape water demand points in the nth partition.

[0063] The comprehensive benefit objective function is: P = ω1P1 + ω2P2 + ω3P3, ω1 + ω2 + ω3 = 1, P is the comprehensive benefit target value, P1, P2, and P3 are the standardized values of the economic benefit target function calculation value (regional GDP), the social benefit target function calculation value (regional water shortage rate), and the ecological benefit target function calculation value (ecological water demand) respectively, ω1, ω2, and ω3 are the weight values of the economic, social, and ecological targets calculated by the analytic hierarchy process.

[0064] The spatial correlation of temperature sequence is extracted by graph attention network, and the feature fusion and position coding are performed before prediction to model the time series information, and the prediction output is performed through the Transformer architecture.

[0065] An adjacency matrix construction method based on temperature environment data is designed, each monitoring station is represented as a node in the graph, and the latitude, longitude, air quality data and meteorological data of each station are taken as the attributes of the node. Let denote the time series of each station i∈{1,2,…,n}, where n corresponds to the number of meteorological stations. Then, the attribute tensor X t of the graph can be represented as: The connection between these nodes is determined according to the geographical proximity of the stations, and the connection relationship is represented by the adjacency matrix A. The matrix A is an n×n matrix, where A ij denotes whether there is an edge between node i and node j. Considering that the earth is a sphere, the paper introduces the Haversine formula to calculate the shortest distance between two points on the sphere using the latitude and longitude of the stations and the radius of the earth.

[0066] The station and its neighbor station feature representations are attention-weighted using a graph attention mechanism to effectively extract the spatial correlation of the air temperature sequence. The GAT takes node features F and adjacency matrix A as input, and the output of GAT is the attention coefficient Y. F is composed of environmental features such as geographical information, air quality, and meteorological data, and these node features change over time. A is used to transmit the spatial relationship between all monitoring station nodes. GAT quantifies the strength of spatial relationships by calculating the attention coefficient of each edge. The specific formula for calculating the attention coefficient is: where N i represents all adjacent nodes of node i, j is a node in N i , F represents node features, W is a shared parameter matrix used by different nodes, || represents a concatenation operation after increasing the node dimension, a T (·) is a single-layer feedforward neural network.

[0067] The Transformer model is used for the spatio-temporal air temperature sequence prediction task, which has the ability to model local and long-range dependencies in sequences. Position encoding is required before prediction. The position encoding vector is added to the input embedding to complete the encoding. For the air temperature at time t, the position encoding can be obtained as shown in the formula: pos∈{1,2,…} represents the position in the time series, i∈{1,2,…} represents the position in the feature sequence, d k represents a predetermined hyperparameter that determines the length of the position encoding vector.

[0068] The time feature matrix of the air temperature at time step t can be obtained from the equation. Finally, the input feature matrix can be calculated by the following formula, where Z t is the feature matrix output after feature fusion.

[0069] The Yellow River Basin rainfall simulation model first converts a given number of hidden states into a state sequence through a Markov chain, and then randomly generates the corresponding observation sequence from the state sequence. The joint probability between the observation value and the state in Bayesian-HMM can be represented as the formula: Z1 represents the initial probability of the first state, P represents the probability, Z represents the state, R t represents the observation value at time t, and Z t represents the state at time t. External influence factors directly affect the transition probability between states, thereby indirectly affecting the observation value. Assuming that the state is i at time t-1 and there are B sequences of external influence factors, the probability that the state is j at time t is represented as: qijt represents an element in the state transition probability matrix, where X t = (X t，1 , X t，2 …X t，B ) is the impact factor variable of B dimensions, sigma ij is the residual error of the regression function, rho j is the regression coefficient of B dimensions, used to evaluate the weight of each impact factor. The optimal number of states in the model is determined by the Bayesian information criterion (BIC), and the state number with the minimum BIC value is the optimal state number.

[0070] The urban ecological-economic balanced development model simulates the dynamic carbon density changes of the Central Plains urban agglomeration in multiple years through a grey prediction model, and dynamically evaluates the influence of land use changes on carbon storage and the spatio-temporal distribution characteristics under the economic development scenario, ecological protection scenario and economic-ecological coordinated development scenario of the Central Plains urban agglomeration from 2005 to 2030 in combination with the Dyna-CLUE model and the InVEST model.

[0071] Further, the application provides a construction method of a flood monitoring model based on a spatio-temporal neural network, comprising the following steps: collecting multi-temporal remote sensing image data of the Yellow River Basin, and extracting hydrological feature parameters such as terrain slope, river curvature and soil water content; using a remote sensing index calculation method including a normalized difference water index (NDWI) and a modified normalized difference water index (MNDWI) and a deep learning technology to identify the flood inundation range of the image; constructing a spatio-temporal neural network model, which comprises a convolutional long short-term memory network (ConvLSTM) layer for capturing the dynamic changes of floods in the spatio-temporal sequence and fusing the hydrological features of the Yellow River Basin through an attention mechanism to realize the training and optimization of the model.

[0072] The application also provides a construction method of a Yellow River ice flood monitoring model based on remote sensing indexes and deep learning, comprising the following steps: obtaining multi-source remote sensing image data of the Yellow River section, including optical and synthetic aperture radar (SAR) images; calculating remote sensing indexes such as a normalized difference ice index (NDII) and a texture feature index to preliminarily extract the ice flood area; fusing a spatio-temporal neural network framework, which adopts a three-dimensional convolutional neural network (3D-CNN) combined with a long short-term memory network (LSTM) for processing remote sensing image sequences to realize spatio-temporal tracking of the dynamic evolution of ice floods; adjusting the parameters of the model through a deep learning optimizer and integrating historical data of the Yellow River ice flood to enhance the spatio-temporal fusion capability of the model.

[0073] The application further provides a lake dynamic monitoring method based on multi-scale context information aggregation, comprising the following steps: collecting multi-scale remote sensing data of lakes in the Yellow River Basin, including high-resolution optical images and medium and low-resolution time series images; designing a multi-scale context aggregation module, which adopts a pyramid pooling structure and a dilated convolution layer to extract feature information of lake water bodies at different temporal and spatial scales; combining the temporal and spatial variation characteristics of water resources in the Yellow River Basin, dynamically monitoring the lake boundaries and water body areas through a semantic segmentation network such as a U-Net variant; and using an attention mechanism to aggregate multi-scale contexts to realize feature fusion and model iterative training.

[0074] The application provides a construction method of a disaster knowledge database, comprising the following steps: collecting historical disaster data of the Yellow River Basin, including related records, remote sensing images and hydrological observation data of flood, ice flood and lake change events; designing a knowledge graph structure, which comprises entity nodes and relationship edges, and organizes disaster field knowledge by using an ontology modeling tool; integrating multi-modal data sources, and filling the database by using natural language processing and image labeling technology; and providing an interface for a Yellow River Basin emergency management large model to support a knowledge query and updating mechanism.

[0075] S3: Construction of a hardware-software integrated intelligent simulation module in a hybrid cloud computing environment for a domestic supercomputer, in a hybrid cloud computing environment supported by a domestic supercomputer, combining the complex scene field knowledge intelligent model of the Yellow River Basin constructed in S2, designing a parallel computing module and an acceleration module of a many-core architecture to realize high-precision and high-efficiency simulation calculation of the complex coupled system of the Yellow River Basin, specifically as follows: a distributed data loading method based on data spatiotemporal characteristics reasonably divides data blocks according to the spatiotemporal distribution characteristics of data, and then realizes efficient parallel loading of massive remote sensing data by constructing a data mapping based on a central data pool. This strategy mainly includes two parts: data dynamic division considering spatiotemporal distribution characteristics and data mapping based on a central data pool.

[0076] The data dynamic division considering spatiotemporal distribution characteristics adopts a remote procedure call (RPC) framework as a specific method for cross-host communication layer, and simultaneously realizes an all-reduce algorithm based on synchronous ring gradient update through NCCL. Among them, gRPC is a special RPC protocol framework, which has better performance than conventional RPC, and has the characteristics of cross-language and cross-platform. Its data packet is based on the HTTP2.0 protocol standard and is designed by itself, which can support bidirectional flow, header compression, multiple request multiplexing and other characteristics, which makes gRPC not only reduce the number of TCP links but also save bandwidth. At the same time, it uses the lightweight and fast ProtoBuf protocol to define a unified service format to solve the addressing and serialization problems.

[0077] Dynamic data partitioning in multi-node distributed training is to shard the dataset to ensure the convergence and reproducibility of the model. The dataset shard considering the spatiotemporal distribution characteristics is shown in Figure 6 As shown in FIG. 1, assuming that there are N computing nodes in the cluster, each computing node is equipped with K blocks of graphics cards, each computing node is allocated a data copy representing the complete dataset, the length of each batch training data is L, and when training to the current batch data, the batch data is evenly divided to all nodes, that is, each node gets L / 4 length of data, and because each thread in the node is responsible for training a block of GPU, the L / 4 data in the node is further divided to each thread, and each thread is relative to the L / 16 length of the current batch data.

[0078] Based on the central data pool data mapping, each GPU is regarded as a process capable of independent training in a distributed system, each process is independently numbered, then the training data is divided according to the established batch data length (mini-batch) responsible for training of each block of GPU, and the arrangement order of the divided data block is shuffled, and at the same time, the update step of the global parameter gradient of the model is calculated according to the total length of the data block used by the cluster at each time during training initialization, so that the data block of the central data pool and the training process can be corresponded one by one through the step and the process number. Each LMDB data block stores a corresponding pointer, and each GPU directly and quickly reads the data block from the disk where the central data pool data is located into the memory through the pointer address by using the memory mapping technology.

[0079] Further, the memory mapping type database LMDB is used to improve the allocation and reading efficiency of the dataset, and the distributed deep learning framework Horovod is used to drive large-scale domestic heterogeneous many-core hardware for model training and inference. In the process of model parallel training or inference, an incremental data processing method is used for model calculation optimization, and the data amount processed by each process at a time is increased to improve the efficiency of model training and inference. Assuming that the number of processes is N, the dataset is divided in units of global batch size, and only one batch of data is taken out for model calculation each iteration, then each process only loads a batch of data with a local batch size of Global Batchsize / N in the memory each time to perform model training or inference, and the above process is repeated until all data in the dataset is processed.

[0080] Further, a parallel computing module adapted to the domestic many-core architecture is constructed. A CPU / DCU collaborative acceleration method of many-core collaborative acceleration of multi-source prior semantic knowledge is proposed, which is based on heterogeneous many-core architecture and analyzes edge prior information, spectral prior information, spatial prior knowledge, and temporal prior knowledge. The Sobel operator is used to perform edge detection on the image.

[0081] Design a differentiable SLIC deep clustering model to group pixels with similar texture, color, and brightness for transition edge segmentation based on acquired depth feature information. Develop a spatial relationship model based on land cover dependencies, acquiring spatial information of land covers through multiple hops along dependency arcs between them, providing prior knowledge for deep understanding of data semantic information. Construct a graph based on a superpixel segmentation algorithm, and design a graph encoder to extract the degree of land cover dependencies. Utilize a GATConv layer to measure the degree of dependency between adjacent objects, and construct a weighted adjacency matrix where different weights represent different degrees of dependency. A′ = dense(sparese(E, Z)), where Z is the sparse edge weights learned by the GATconv layer of the FSSC-GCN model, and A′ is a weighted adjacency matrix constructed from Z. A GRU network is used to mine deeper temporal features. The GRU consists of an update gate z. t Reset door r t and hidden state h t The model is composed of components, each considering the loss for each category in the multi-machine collaborative extraction model. A loss function is constructed for each category, and then the average error for each category is calculated, as shown in the equation: This represents the total loss of all samples in the nth class. Let M represent the label of the i-th sample in the n-th class and the model's predicted value, respectively. n This represents the total number of samples in the nth class. By adding the weights of adaptive sample features to the calculation of the loss function using the original class proportions in the dataset, the above formula is improved into a weighted cross-entropy loss function based on adaptive sample features.

[0082] Furthermore, a dynamic acceleration module for the simulator is constructed. This includes proposing a strip residual module and a strip pyramid pooling module. The strip residual module consists of a series of residual units and strip pooling operations to extract long-distance strip features from the input image. The residual units add dilated convolution operations to the convolutional blocks to effectively expand the receptive range without reducing the feature map size. Strip pooling is added in the last residual unit to capture long-range information features.

[0083] A batch generation strategy for incremental processing is designed to reduce the data storage load on worker nodes by processing resources in batches, thereby accelerating multi-machine parallelism in distributed computing clusters. Secondly, the MD-LAMB algorithm is proposed to transform LAMB into SGDM in the early stages of training, enabling model weights to utilize the accumulated gradients in the momentum mechanism to increase the learning rate and accelerate convergence. Simultaneously, to prevent deviation from the optimal update direction, layer-based gradient L2 norm information is incorporated, allowing the growth rate of the learning rate to be automatically adjusted according to the convergence degree of the model layers.

[0084] The remote sensing image building data is stored in the form of key-value in LMDB, and the pointer occupying a small amount of memory is used for image positioning and reading. Then, combined with the Horovod framework, the model training process is parallelized after loading the data, and multiple machines are used to process large capacity data in parallel to improve the overall computing efficiency, and the overall framework structure is as shown in Figure 7 .

[0085] The incremental data processing method is used for data set loading and distribution optimization, and the strategy design idea is as shown in Figure 8 . The data set is divided into global batches Global Batchsize, and only one batch of data is taken out for parallel processing each time, so that each process only loads local Batchsize=GIobal Batchsize / N batch data in memory each time to update the model weight, and the above process is repeated until the data set is completely processed. The advanced layered adaptive optimizer algorithm LAMB is used to replace the original Adam weight update method, and an adaptive preheating optimization strategy is proposed to further improve the early convergence speed of LAMB. The layer normalization idea is used to adjust the gradient size to further ensure the stability of the decay process.

[0086] S4: Construct a Yellow River simulator platform based on a micro-service architecture, the Yellow River simulator platform comprising a data processing module, a calculation simulation module, a knowledge cognition module, a system service module and a visual presentation module; the platform is used to simulate a super-large scale space-time scene on the basis of completing simulation calculation, and provides one-stop intelligent service, has data interaction, calculation simulation and decision support functions, forms an integrated comprehensive service platform, and specifically as follows: the application divides the functions of the Yellow River simulator platform into five independent micro-service modules, namely a data processing module, a calculation simulation module, a knowledge cognition module, a system service module and a visual presentation module. Each module is designed based on the single responsibility principle (Single Responsibility Principle) to ensure independent functions and clear responsibilities.

[0087] RESTful API specification is used to define standardized interfaces for each micro-service module, and OpenAPl is used to describe the interface specification, including input parameters, output format and error code. The interface supports JSON or Protobuf format data transmission to ensure efficiency and compatibility.

[0088] A service registry center is deployed, and Eureka or Consul is used to realize dynamic registration and discovery of micro-services. Each micro-service registers service information to the registry center when starting, and maintains service state update through a heartbeat mechanism.

[0089] Based on Docker container technology, each microservice module is encapsulated, and each microservice is packaged as an independent Docker image. Dockerfile is used to define the image building process, including the running environment (Java, Python), dependent libraries and configuration files.

[0090] Kubernetes is used as a container orchestration tool to configure Pod, Deployment and Service resources to achieve automatic deployment, elastic expansion and load balancing of microservices. Kubernetes' ConfigMap is used to manage microservice configuration files and dynamically adjust running parameters.

[0091] Kubernetes' replica set mechanism is used to configure multiple instances for each microservice to ensure high availability of the service. Combined with the health check mechanism, it automatically detects and restarts faulty instances.

[0092] gRPC and Kafka message queues are used to achieve efficient communication between microservices. For high-throughput scenarios, Kafka is used for asynchronous message passing; for low-latency requirements, gRPC is used for synchronous calls.

[0093] OAuth2.0 protocol is used to configure identity authentication and authorization mechanisms for microservice interfaces, and JWT (JSON Web Token) is used for secure communication between services. API gateway is deployed to manage external requests uniformly, implement traffic control and request routing.

[0094] Integrate distributed log system to collect, store and analyze the running logs of each microservice. Use Prometheus and Grafana to monitor the performance indicators of microservices and set alarm rules to deal with abnormal situations.

[0095] Use Git for microservice code version management, and use CI / CD tools to implement automated building, testing and deployment. Each code commit triggers automated testing to ensure the stability of microservice updates.

[0096] Further, through the data interface, multi-source spatio-temporal data is collected, including but not limited to hydrological data, meteorological data, geographic information data and remote sensing data. A distributed data collection framework (Apache NiFi) is used to implement data source access and management, supporting multiple data protocols to adapt to different data source transmission formats. Data preprocessing is implemented, using statistical filtering methods to identify and remove noise and outliers in the data, such as abnormal water level records or missing remote sensing pixel values. Through data consistency checking, logical errors in the data are detected and corrected, such as timestamp misalignment or geographic coordinate deviation, based on rule verification and anomaly detection model to achieve automated cleaning.

[0097] Further, on the Hadoop and Spark distributed computing framework, combined with the heterogeneous characteristics of comprehensive observation data and common simulation calculation and analysis scenarios, spatial sub-domain division, spatial sub-domain task calculation evaluation, and spatial parallel computing models are designed to form a high-performance computing module for the Yellow River Basin simulator.

[0098] A spatial sub-domain is a subset of spatial entities in a layer formed by a data division strategy. A spatial sub-domain is composed of computing entities and auxiliary entities. Computing entities refer to entities that need to perform spatial operation functions in the sub-domain, and auxiliary entities are entities that assist computing entities in performing spatial operation functions. Spatial entities in a layer are divided into independent spatial sub-domains, so that each computing entity can independently complete function operations.

[0099] Further, a knowledge cognition module is constructed. In the data preprocessing part, structured information, type information, and entity description information in the knowledge base are extracted, and the parameters of the model are initialized. The knowledge representation learning part uses structured information through a translation model, uses type information through a mapping matrix, and uses entity description information through a convolutional neural network. After joint training, the comprehensive vector representation of entities and relationships is obtained, and the obtained vector is used in the knowledge graph application part to implement the knowledge graph completion task and knowledge reasoning.

[0100] Based on ontology modeling technology, a semantic knowledge base related to the Yellow River Basin is constructed, covering professional knowledge in the fields of hydrology, geography, ecology, and meteorology. OWL (Web Ontology Language) and RDF (Resource Description Framework) formats are used to define the ontology structure, ensuring semantic representation of knowledge. Knowledge is extracted from structured data, semi-structured data, and unstructured data, and natural language processing techniques (named entity recognition NER, relationship extraction) are used for automated knowledge extraction. Multiple sources of knowledge are integrated to form a unified semantic knowledge graph. A graph database (Neo4j) is used to store the semantic knowledge base, supporting efficient querying of nodes (entities) and edges (relationships). A knowledge update mechanism is configured to periodically update the knowledge base content from external data sources.

[0101] Further, a system service module is constructed, and a task scheduling framework is designed to decompose complex simulation tasks (hydrological simulation, flood prediction) into sub-tasks based on task priority and dependency relationships. A distributed task scheduling framework is used to realize automatic arrangement and execution of tasks.

[0102] Priority queue algorithm and load balancing strategy are used to optimize task allocation, ensuring that high-priority tasks are executed first. Combined with a task execution time prediction model, the scheduling strategy is dynamically adjusted.

[0103] Implement task status monitoring, record task submission, execution, completion and failure status, and display task progress in real time through task logs and dashboards. Support task retry mechanism, automatically trigger retry or degradation processing for failed tasks.

[0104] Use dynamic resource allocation algorithm, based on demand resource preemption or proportional allocation, allocate appropriate computing resources according to task computing demand. Combined with resource usage monitoring, realize adaptive adjustment of resources.

[0105] Deploy load balancer in supercomputing cluster, evenly distribute computing tasks to multiple computing nodes to avoid single point overload. Support heterogeneous computing resource scheduling, optimize CPU and GPU collaborative use efficiency.

[0106] Implement resource isolation between microservices through containerization technology to prevent resource competition. Configure resource quota to limit the maximum resource usage of each module.

[0107] Use asynchronous message queue and synchronous communication protocol to realize efficient communication between microservices. Asynchronous communication is used for high throughput scenarios, and synchronous communication is used for low latency requirements.

[0108] Provide standardized RESTful API and gRPC interface for data interaction and task invocation between modules. The interface supports task submission, state query and result retrieval functions, and uses OpenAPI specification to describe interface details.

[0109] Further, in the visual presentation module, based on the dynamic organization model of massive data, combined with the principles and technologies related to spatio-temporal geographic information, a model that meets the multi-dimensional and multi-scale expression of spatio-temporal data is constructed; Then, by comprehensively using spatial information visualization related technologies, the static expression of multi-dimensional and multi-scale spatio-temporal data is realized, the discrete change expression of multi-dimensional and multi-scale spatio-temporal data is realized by using sequence map switching method and symbol discrete change method, and the dynamic visualization expression of multi-dimensional and multi-scale spatio-temporal data is realized by using dynamic symbol method, dynamic interaction method and image animation method. Based on the symbolization technology and graph matching technology, spatio-temporal object vector data, raster data, three-dimensional data rendering technology is developed, and a "parallel computing, progressive transmission, complementary" collaborative flow data rendering model is constructed. Each unit of storage data is stored in a certain spatial range of points, lines, surfaces, bodies, rasters, field data through spatial division; Using the adapter principle, the mapping between multi-source data and the model is established. The method of combining real-time compression with offline compression is used to improve the data compression and progressive transmission mode. In the memory, texture images are stored in formats with high compression ratio, and after being loaded into memory during rendering, they are quickly decompressed and compressed into formats suitable for three-dimensional rendering in real time.

[0110] The tile node-based chunk index tree technology divides the space into multiple tiles using tiles as the basic unit and organizes these tiles in the form of index trees. The entire space is divided into multiple tiles according to certain rules. Index information is established for each tile, including node range, data pointer, and statistical information, etc. Real-time data updating is achieved through the WebSocket protocol. Data requests triggered by user operations, such as loading high-resolution data after zooming in on an area, are quickly responded through micro-service interfaces. Browser caching and data compression techniques are used to reduce data transmission and optimize the loading speed of large-scale scenarios. Asynchronous rendering technology is adopted to avoid main thread blocking and ensure smooth interaction. New layer types or interactive tools are added through the plug-in mechanism. Containerized deployment is used to achieve rapid updating and expansion of modules.

Claims

1. A method for constructing a Yellow River simulator system based on space intelligence and domestically produced supercomputing, characterized in that, Includes the following steps: S1. Construct a data aggregation and intelligent management module to realize the proactive discovery, automatic aggregation, and storage management of multi-source, multi-scale comprehensive observation data; S2. Construct a domain knowledge reasoning and modeling module. Based on the observation data obtained in S1, extract knowledge from multimodal data and combine cross-modal knowledge fusion and reasoning methods to construct a domain knowledge model for complex scenarios in the Yellow River Basin. S3. Construct an integrated hardware and software intelligent simulation module for a hybrid cloud computing environment supported by domestic supercomputing. In a hybrid cloud computing environment supported by domestic supercomputing, and in conjunction with the domain knowledge model constructed in S2, design a parallel computing module and acceleration module with a many-core architecture to achieve high-precision and high-efficiency simulation calculation of complex coupled systems in the Yellow River Basin. S4. Construct a Yellow River simulator platform based on a microservice architecture. The Yellow River simulator platform includes a data processing module, a computational simulation module, a knowledge cognition module, a system service module, and a visualization presentation module, which are used to simulate ultra-large-scale spatiotemporal scenarios.

2. The method for constructing the Yellow River simulator system based on space intelligence and domestic supercomputing according to claim 1, characterized in that, In step S1, the method for actively discovering multi-source, multi-scale integrated observation data includes the following steps: The data discovery module is built with a "dual-engine drive" approach, as detailed below: (1) Based on the traditional vector space model, spatial place names and common keywords are used as independent variables to map the theme and data content to the spatial vector space and the common theme vector space composed of the weights of place names and common keywords, so as to construct a multi-source data keyword extraction module. (2) By calculating time, space and topic relevance, the natural exponential function is integrated to realize data address priority allocation, and then the search priority is scheduled according to the relevance value. The search is carried out by a two-layer optimization method of surface data discovery and deep search until the priority queue is empty, so as to build a soft-perception keyword search module. (3) Construct a module for extracting typical elements from long-term integrated observation data, as follows: A temporal feature learning module is constructed using a long short-term memory network for encoding. Simultaneously, the feature vectors of different input time series are positionally encoded using sine and cosine functions, and a self-attention mechanism is used to adaptively learn the spatiotemporal feature weight distribution to construct a typical element extraction module for long-term comprehensive observation data. (4) Construct a typical feature extraction module for high spatial resolution data, as follows: We establish discriminative feature learning and typical element extraction methods for high-dimensional data from three different cognitive scales: pixel level, object level and target level. We adopt a mutual knowledge distillation strategy to maximize the feature representation of different levels of pixel-object-target in order to construct a typical element extraction module for high spatial resolution data. An automatic aggregation module for comprehensive watershed-scale observation data is constructed, as detailed below: (1) Construct a comprehensive three-dimensional network catalog system for observation data Based on the existing catalog system and standards, the classification and grading information of observation data is tagged, the original catalog hierarchy is reconstructed using a three-dimensional tag network, and the data organization structure of each center is compatible through a multi-dimensional mapping mechanism, so as to construct a comprehensive three-dimensional network catalog system for observation data. (2) Establish a "time-space-theme" correlation module for comprehensive observation data. The basic characteristics of hybrid interconnected networks are described using algorithms such as distance measurement, connectivity measurement, and centrality analysis for complex networks. Community detection algorithms are used to identify closely related "communities" in the hybrid interconnected networks. Finally, the clustering and hierarchy of the interconnected networks are analyzed using network clustering coefficients and hierarchical modularity, and the evolution of the overall interconnected network topology is analyzed to establish "time-space-theme" interconnected modules. (3) Construct a comprehensive observation data aggregation module Starting from three aspects—start time, common keywords, and source keywords—this paper adopts a topic similarity calculation strategy that takes into account the start time and a topic similarity calculation strategy based on dual vectors. Based on the multi-vector topic representation method, the paper calculates the temporal similarity, common topic similarity, and spatial similarity of topics and data content respectively, in order to construct a comprehensive observation data aggregation module. A comprehensive observation data storage organization and management module is constructed, as detailed below: (1) Construct a distributed storage model for comprehensive observation data based on a multi-dimensional sharding mechanism, as follows: By constructing a distributed metadata organization method, a metadata intelligent partitioning mechanism, and a multi-replica consistency model, a module for storing, organizing, and managing observation data is built. (2) Construct a dynamic data organization model based on multidimensional stereolithography, as follows: The spatiotemporal and usage frequency of comprehensive observation data in the Yellow River Basin are calculated. Each spatial data point is assigned a unique multidimensional segmentation identifier based on a segmentation grid. Based on the multidimensional partitioning scheme constructed by the geographic segmentation grid, a dynamic data organization model incorporating time and data popularity is constructed. (3) Construct an intelligent retrieval model for comprehensive observation data based on asymmetric hashing, as follows: In the feature extraction stage, a lightweight network FasterNet is introduced and an optimized structural embedding module is added to the backbone network. Partial convolution PConv is used to apply convolution operations to some channels of the input feature map, while the characteristics of the remaining channels remain unchanged, in order to build a comprehensive observation data intelligent retrieval model.

3. The method for constructing a Yellow River simulator system based on space intelligence and domestic supercomputing according to claim 1, characterized in that, In S2, the specific process of constructing the domain knowledge reasoning and modeling module is as follows: We constructed a water resources allocation model for the Yellow River Basin based on WEAP, designed a temperature prediction model for the Yellow River Basin based on graph neural networks, and developed a rainfall simulation method for the Yellow River Basin based on Bayesian methods. We also deeply integrated remote sensing imagery and deep learning technology to establish knowledge modeling models in the fields of hydrology and water resources, watershed water cycle, urban ecological-economic balance development, and major natural disasters.

4. The method for constructing a Yellow River simulator system based on space intelligence and domestic supercomputing according to claim 1, characterized in that, In S3, the implementation method of the simulator parallel scheduling module is as follows: The simulator's parallel scheduling module is formed by constructing a data scheduling module, a domestic many-core parallel computing module, and a simulator dynamic acceleration module, wherein: Based on the spatiotemporal characteristics of the data, massive remote sensing data is segmented into pieces, taking into account the spatiotemporal distribution characteristics and dynamically dividing the data into reasonable blocks; at the same time, based on the central data pool technology, the segmented data is distributed and loaded to build a data scheduling module. We mine multi-source prior semantic knowledge in the data, achieve deep semantic expression through multi-feature fusion, and combine cost-sensitive learning with a distributed multi-machine collaborative extraction model to build a parallel computing module adapted to the domestic many-core architecture. Resources are processed in batches to reduce the data storage load on worker nodes, and the MD-LAMB algorithm is converted from LAMB to SGDM in the early stage of training to build a dynamic acceleration module for the simulator.

5. The method for constructing a Yellow River simulator system based on space intelligence and domestic supercomputing according to claim 1, characterized in that, The specific process of S4 is as follows: (1) Build a microservice framework and encapsulate the data processing module, computation simulation module, knowledge cognition module, system service module and visualization presentation module into services; (2) In the data processing module, multi-source spatiotemporal data are collected, and data cleaning, format standardization, spatiotemporal registration and feature extraction are performed in sequence. (3) In the computational simulation module, multi-scale numerical simulations of hydrological, hydraulic and geographical processes are performed based on a distributed parallel computing scheduling engine. (4) In the knowledge cognition module, a semantic knowledge base is constructed and reasoning algorithms are used to perform semantic annotation and context analysis on the simulation results; (5) Implement task scheduling, computing resource allocation and message communication between microservices in the system service module; (6) In the visualization presentation module, the simulation results are displayed in a spatial manner using a 3D rendering engine, and interactive operation functions are implemented.

6. A Yellow River simulator system based on space intelligence and domestically produced supercomputing, characterized in that, It includes a data aggregation and intelligent management module, a domain knowledge reasoning and modeling module, a hardware and software integrated intelligent simulation module for hybrid cloud computing environment for domestic supercomputing, and a Yellow River simulator platform based on microservice architecture.

7. The Yellow River simulator system based on space intelligence and domestic supercomputing as described in claim 6, characterized in that, The integrated observation data active aggregation and intelligent management module is embedded in the data layer of the Yellow River simulator system. It is used to realize the active discovery, automatic aggregation and efficient storage management of multi-source and multi-scale integrated observation data, and to provide data support for the Yellow River simulator system. The domain knowledge reasoning and modeling module is deployed in the platform's cognitive layer. It is used to accurately extract knowledge from multimodal data and, combined with cross-modal knowledge fusion and reasoning technology, construct an intelligent domain knowledge model for complex scenarios in the Yellow River Basin, providing knowledge support for the Yellow River simulator system. The integrated hardware and software intelligent simulation module for hybrid cloud computing environment of domestic supercomputing is integrated into the platform computing layer. It is used to realize the efficient simulation of large-scale complex scenarios in the Yellow River Basin through parallel computing module and acceleration module of many-core architecture, and to provide computing support for the Yellow River simulator system. The Yellow River Simulator Platform, based on a microservice architecture, integrates and encapsulates the aforementioned modules into a Yellow River Simulator Platform with full-link functionality encompassing "data-computation-cognition-service-presentation," enabling real-time dynamic simulation and "one-stop" intelligent services for ultra-large-scale spatiotemporal scenarios in the Yellow River Basin.

8. The Yellow River simulator system based on space intelligence and domestic supercomputing as described in claim 6, characterized in that, The Yellow River simulator platform includes user interaction interfaces, data interaction interfaces, model call interfaces, and service output interfaces. It supports multi-terminal access and integration with third-party systems to achieve visualized display of simulation results and targeted push of decision-making suggestions.

Images