A multi-source, multi-dimensional and multi-scale data fusion method and system

Abstract

The present invention discloses a multi-source, multi-dimensional, and multi-scale data fusion method and system, which relates to the technical field of data fusion, including determining a data source according to a data source management principle, sending a session key to the data source, and collecting data after pairing; classifying and integrating the collected data according to the required purpose, separating the corresponding dimension value for each type of data, and separating the scale information from the dimension value; classifying and integrating the dimension value of each type of data again, sorting the various types of dimension values ​​horizontally according to the corresponding spatial information, and sorting them vertically based on the time information corresponding to the various types of dimension values ​​to construct a data space coordinate system; establishing an index for each type of dimension value on the spatial coordinate system, and when the required purpose begins, searching for the required data according to the index. The present invention associates the dimension value with the data and purpose by constructing a dimension data relationship table, and constructs a data space coordinate system based on the scale information in the dimension value, so that the data classification is more intuitive and neat.

Description

Technical Field

[0001] The present invention relates to the technical field of data fusion, and in particular to a multi-source, multi-dimensional and multi-scale data fusion method and system. Background Art

[0002] In the field of hydropower generation, in order to achieve effective water resource management and maximize power generation efficiency, it is necessary to build and apply a variety of models, including hydrological models, hydrodynamic models, and power generation prediction models. The construction of these models relies on data collected from a variety of data sources, which are complex and diverse, including but not limited to meteorological data, flow data, reservoir water level data, and terrain and geomorphology data.

[0003] First, the data collection process faces many challenges. The collection of multi-source data requires the formulation of clear data entry rules to ensure the consistency and integrity of the data. In addition, the diversity and volume of data often require highly complex processing procedures to maintain the accuracy and real-time nature of the data. During data transmission, data security faces severe challenges, including security risks such as unauthorized access and data tampering during data transmission.

[0004] Secondly, due to the diversity of data sources and the instability of the collection environment, data is often missing or erroneous. These abnormal problems may be caused by equipment failure, communication interruption or natural environmental factors, resulting in discontinuity and incompleteness of the collected data, which affects the accuracy and reliability of the model.

[0005] Finally, even if the data is successfully collected and transmitted, existing data processing and classification methods often show limited effectiveness when dealing with data with multi-dimensional and multi-scale characteristics. Although traditional database query indexing technology and linear regression statistical models optimize the efficiency of processing consistently structured data, they are inefficient and inflexible in processing multi-dimensional and multi-scale data, especially semi-structured or unstructured data, and have difficulty in handling nonlinear and complex relationships between data, especially in multi-dimensional data, where the interactions between variables may be extremely complex.

[0006] In order to solve the above problems, the present invention proposes a multi-source, multi-dimensional and multi-scale data fusion method and system, which aims to solve the problems of low efficiency and poor flexibility of existing methods when processing multi-dimensional and multi-scale data, especially semi-structured or unstructured data. Summary of the invention

[0007] In view of the above-mentioned problems, the present invention is proposed.

[0008] Therefore, the problem to be solved by the present invention is that traditional data classification and integration technologies are difficult to cope with the high-dimensional complexity of data and cannot effectively identify and utilize multi-scale information in the data, which further aggravates the difficulty of data management and affects the effective application of data.

[0009] To solve the above technical problems, the present invention provides the following technical solutions: a multi-source, multi-dimensional and multi-scale data fusion method, which includes: determining a data source according to a data source management principle, sending a session key to the data source, and collecting data after pairing; classifying and integrating the collected data according to the required purpose, separating the corresponding dimension value for each type of data, and separating scale information from the dimension value, wherein the scale information includes spatial information and time information; classifying and integrating the dimension values ​​of each type of data again, horizontally sorting each type of dimension value according to the corresponding spatial information, and on the basis of the horizontal sorting, vertically sorting based on the time information corresponding to each type of dimension value to construct a data space coordinate system; establishing an index for each type of dimension value on the spatial coordinate system, and when the required purpose begins, searching for the required data according to the index.

[0010] As a preferred solution of the multi-source, multi-dimensional and multi-scale data fusion method described in the present invention, the data source management principle includes the type of data to be collected and the source of data collection; the data source management principle is updated in real time as needed.

[0011] As a preferred solution of the multi-source, multi-dimensional and multi-scale data fusion method described in the present invention, the session key generation rule includes: after determining the data source, the data source and the acquisition end negotiate to determine the first session key, and after pairing based on the first session key, the data source and the acquisition end are connected through a transmission channel; according to the type of data required to be collected in the data source management principle, the corresponding type of data is packaged and a second session key is generated, and after the second session key is paired, data transmission is performed.

[0012] As a preferred solution of the multi-source, multi-dimensional and multi-scale data fusion method described in the present invention, wherein: the required uses include data for constructing a hydrological model, data for constructing a hydrodynamic model and data for constructing a power generation prediction model; the dimensional values ​​of each type of data are classified and integrated again, including defining the data collected from each of the data sources as multi-source data, and each data point in the multi-source data contains one or more dimensional values; determining the dimensional value of each data in all the multi-source data, and deduplicating the dimensional values ​​of the same type, and mapping the deduplicated dimensional values ​​to the multi-source data to form a dimensional data relationship table; the dimensional data relationship table indicates that one or more dimensional values ​​correspond to a data point; generating corresponding dimensional relationship identifiers for each dimensional relationship record in the dimensional data relationship table, and reversely binding the dimensional relationship identifiers corresponding to each dimensional relationship record to the data, and corresponding to the dimensional values ​​of each data; and classifying the dimensional values ​​according to the dimensional data relationship table.

[0013] As a preferred solution of the multi-source, multi-dimensional, and multi-scale data fusion method described in the present invention, the dimensional value of each type of data is classified and integrated again, and when the data source management principle remains unchanged, the re-collected data is matched one by one with the dimensional relationship records in the established dimensional data relationship table based on the dimensional relationship records extracted from the established dimensional data relationship table when data is collected again, and the dimensional value of the multi-source data to be fused is classified. If a new type of dimensional value that cannot be matched appears, it indicates that data redundancy occurs, and the redundant data type is fed back to the corresponding data source. The data source removes the redundant data type when packaging the data. If any dimensional relationship record is not matched after the classification is completed, it indicates that data is missing, and the missing data type is fed back to the corresponding data source. The data source repackages the missing data and generates a third session key. After the third session key is paired, the data is supplemented and transmitted. When the data source management principle changes and data is collected again, the newly collected new type of data is matched with the dimension relationship records in the established dimension data relationship table one by one. If there is a dimension relationship record matching the dimension value of the new type of data in the established dimension relationship record, the existing dimension relationship record is reused. If there is no dimension relationship record matching the dimension value of the new type of data in the established dimension relationship record, the new dimension relationship record is added to the established dimension relationship record. After the classification is completed, the dimension relationship records that have not been matched are deleted from the dimension data relationship table, and finally a dimension data relationship table that meets the new data source management principle is obtained.

[0014] As a preferred solution of the multi-source, multi-dimensional and multi-scale data fusion method described in the present invention, wherein: the horizontal sorting of various dimensional values ​​according to the corresponding spatial information includes: the dimensional value includes the corresponding spatial information, the spatial information reflects the collection location of the current dimensional value, and the spatial information of all dimensional values ​​is obtained to construct a geographical block topology map, that is, the plane map is divided into a number of blocks according to the collection location on the horizontal plane, one block corresponds to one collection location, and the dimensional value data is loaded into the corresponding block; the vertical sorting based on the time information corresponding to the various dimensional values ​​includes: after all dimensional values ​​are loaded into the corresponding block, different dimensional values ​​in the block are classified based on the dimensional data relationship table, and the time information of each type of dimensional value is collected; each type of dimensional value contains multiple independent dimensional value data, and the time information indicates the collection time of the current dimensional value data; based on the collection time, the dimensional value data is arranged vertically in a time sequence from bottom to top, if there is a time sequence interruption in the arrangement, that is, the corresponding time sequence data is missing, then feedback is fed back to the data source to request data supplement, if the data source reply information is that the data missing cannot be compensated due to an abnormal state, then the historical data is called to supplement the missing part.

[0015] As a preferred solution of the multi-source, multi-dimensional and multi-scale data fusion method described in the present invention, wherein: the indexing of each type of dimensional value in the spatial coordinate system includes, for each type of dimensional value in each block, index marking according to the results of classification and integration according to the required purposes, that is, one type of dimensional value corresponds to one or several required purposes; the searching for required data according to the index also includes, after searching for the required data, standardizing the data according to the requirements of the required model construction.

[0016] Another object of the present invention is to provide a multi-source, multi-dimensional, and multi-scale data fusion system, which can effectively fuse multi-source, multi-dimensional, and multi-scale data to make data utilization more efficient.

[0017] To solve the above technical problems, the present invention provides the following technical solutions: a system of a multi-source, multi-dimensional and multi-scale data fusion method, comprising: a data acquisition module, a preliminary integration module, a secondary integration module and an index establishment module; the data acquisition module determines the data source according to the data source management principle, sends a session key to the data source, and collects data after pairing; the preliminary integration module classifies and integrates the collected data according to the required purpose, separates the corresponding dimension value for each type of data, and separates the scale information from the dimension value, wherein the scale information includes spatial information and time information; the secondary integration module classifies and integrates the dimension value of each type of data again, sorts the various types of dimension values ​​horizontally according to the corresponding spatial information, and on the basis of the horizontal sorting, sorts vertically based on the time information corresponding to the various types of dimension values ​​to construct a data space coordinate system; the index establishment module establishes an index for each type of dimension value on the spatial coordinate system, and when the required purpose begins, the required data is searched according to the index.

[0018] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the multi-source, multi-dimensional and multi-scale data fusion method described above are implemented.

[0019] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-source, multi-dimensional, and multi-scale data fusion method as described above.

[0020] The beneficial effects of the present invention are as follows: the present invention preliminarily screens data by formulating data source management principles, thereby reducing the entry of useless information. The present invention adopts a double verification method, wherein the first verification first connects the data source and the acquisition end to the transmission channel, and the second verification is performed during data transmission, thereby improving the security of data transmission.

[0021] The present invention associates dimension values ​​with data and uses by constructing a dimension data relationship table, and according to the matching relationship between the new data and the dimension data relationship table, it can be determined whether there is data redundancy or data missing, and a data space coordinate system is constructed based on the scale information in the dimension value, so that data classification is more intuitive and neat, and an index is established for each type of data in the data space coordinate system, so that the required data can be directly found when constructing the corresponding model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0023] Figure 1 This is a flow chart of a multi-source, multi-dimensional, and multi-scale data fusion method in Example 1.

[0024] Figure 2 This is a module structure diagram of a multi-source, multi-dimensional, and multi-scale data fusion system in Example 3. DETAILED DESCRIPTION

[0025] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0027] Example 1, reference Figure 1 , which is the first embodiment of the present invention, and which provides a multi-source, multi-dimensional, and multi-scale data fusion method, including: Figure 1 As shown:

[0028] Step 1: Determine the data source according to the data source management principle, send the session key to the data source, and collect data after pairing.

[0029] Data source management principles include the type of data that needs to be collected and the source of data collection. Data source management principles are updated in real time as needed.

[0030] The generation rule of the session key includes that after determining the data source, the data source and the collection end negotiate to determine the first session key, and after pairing based on the first session key, the data source and the collection end establish a transmission channel connection.

[0031] According to the type of data to be collected in the data source management principle, the corresponding type of data is packaged and a second session key is generated. After the second session key is paired, data transmission is performed. The generation rule of the second session key is the same as that of the first session key.

[0032] Step 2: Classify and integrate the collected data according to the required purpose, separate the corresponding dimension value for each type of data, and separate the scale information from the dimension value. The scale information includes spatial information and temporal information.

[0033] The required uses include data for building a hydrological model, data for building a hydrodynamic model, and data for building a power generation prediction model. The details are as follows:

[0034] 1. The data required for the hydrological model include meteorological data, topographic data, watershed boundaries, hydrological and hydraulic data, soil data, and human activity data. Each type of data contains the following dimensional values:

[0035] 1. The dimensional values ​​contained in meteorological data include: rainfall: time and space distribution data.

[0036] Temperature: daily average, maximum and minimum temperatures.

[0037] Evaporation: Measured or estimated daily evaporation.

[0038] Wind speed and sunshine hours: affect evaporation and vegetation transpiration.

[0039] 2. The dimensional values ​​of terrain data include: Basin boundary: determines the working area of ​​the hydrological model.

[0040] Elevation and slope: affect runoff and surface flow velocity.

[0041] Land cover: Different types of land surfaces such as forests, farmlands, urban areas, etc.

[0042] 3. The dimensional values ​​contained in hydrological and hydraulic data include: River flow: historical flow data of each measuring station in the basin.

[0043] Water level data: Changes in water levels of important water bodies.

[0044] Water table: the depth and flow characteristics of groundwater.

[0045] 4. The dimensional values ​​contained in soil data include: Soil type: affects permeability and water retention capacity.

[0046] Soil Moisture: Current soil moisture conditions.

[0047] 5. The dimension values ​​contained in human activity data include: land use change: such as urban expansion, farmland irrigation, etc.

[0048] Water conservancy projects: such as construction and operation data of reservoirs, dams, and channels.

[0049] 2. The data required for the hydrodynamic model include fluid dynamics data, topography and geomorphology data, boundary conditions and initial conditions, water quality data and meteorological data. Each type of data contains the following dimensional values:

[0050] 1. The dimensional values ​​contained in the fluid dynamics data include: flow velocity and flow direction: used to correct the model output.

[0051] Water level data: Set boundary conditions for the model.

[0052] 2. The dimensional values ​​contained in the terrain and landform data include: River cross-section data: including width, depth and shape.

[0053] Riverbed and lake bottom topography: affects water flow dynamics.

[0054] 3. The dimensional values ​​contained in boundary conditions and initial conditions include: inflow and outflow conditions: flow rate or water level at the inlet and outlet.

[0055] Initial water flow state: The water dynamics when the model starts.

[0056] 4. The dimensional values ​​contained in water quality data include: pollutant concentration: water quality conditions at different locations.

[0057] Indicators such as biochemical oxygen demand (BOD) and chemical oxygen demand (COD).

[0058] 5. Meteorological data (which also affects hydrodynamic models) contains dimensional values ​​including: Wind direction and wind speed: especially affecting water flow in large water bodies.

[0059] Atmospheric pressure: can affect the water level in a body of water.

[0060] 3. The data required for the power generation forecast model include hydrological data, meteorological data, mechanical performance parameters, operating parameters, economic and dispatching parameters, environmental and regulatory parameters and historical operating data. Each type of data contains the following dimensional values:

[0061] 1. The dimensional values ​​contained in hydrological data include: Inflow: the amount of water flowing into the reservoir, usually measured in cubic meters per second (m³ / s).

[0062] Water level: Data on water levels in reservoirs and downstream, which affects power generation capacity and unit operation.

[0063] 2. The dimensional values ​​contained in meteorological data include: Rainfall: rainfall conditions in the basin directly affect inflow and reservoir water level.

[0064] Evaporation: Water loss that affects a reservoir.

[0065] 3. The dimensional values ​​contained in the mechanical performance parameters include: Turbine efficiency: turbine conversion efficiency under different flow rates and heads.

[0066] Generator Efficiency: How efficiently a generator converts mechanical energy into electrical energy.

[0067] Maximum output capacity: The maximum power generation capacity of the turbine generator.

[0068] 4. The dimensional values ​​contained in the operating parameters include: Head: The vertical distance between the reservoir water level and the turbine, which is an important factor in generating electrical energy.

[0069] Outflow: The amount of water flowing through the generator, which directly affects the amount of electricity generated.

[0070] 5. The dimensional values ​​contained in the economic and dispatch parameters include: Power demand and market price: predict power demand and electricity prices in different time periods to optimize power generation dispatch.

[0071] Operation and maintenance costs: The costs of operating a hydroelectric power station, including maintenance and personnel costs.

[0072] 6. Environmental and regulatory parameters contain dimensional values ​​including: Environmental flow requirements: minimum downstream flows required by regulations to protect aquatic ecosystems.

[0073] Water management policies: Policies and regulations that may affect reservoir operations.

[0074] 7. The dimensional values ​​contained in the historical operation data include: Past power generation records: used to train prediction models and understand the power generation efficiency under different conditions.

[0075] Historical water level and flow data: Assess the impact of long-term hydrological trends on power generation.

[0076] Step 3: Classify and integrate the dimension values ​​of each type of data, sort the dimension values ​​horizontally according to the corresponding spatial information, and then sort them vertically based on the time information corresponding to the dimension values ​​to construct a data space coordinate system.

[0077] Classify and integrate all types of dimension values ​​in the previous step, and define the data collected from various data sources as multi-source data. Each data point in the multi-source data contains one or more dimension values.

[0078] Determine the dimension value of each data in all multi-source data, deduplicate the dimension values ​​of the same type, and map the deduplicated dimension values ​​to the multi-source data to form a dimensional data relationship table.

[0079] The dimension data relationship table indicates that one or more dimension values ​​correspond to a data point. A corresponding dimension relationship identifier is generated for each dimension relationship record in the dimension data relationship table, and the dimension relationship identifier corresponding to each dimension relationship record is reversely bound to the data and corresponds to the dimension value of each data; the dimension values ​​are classified according to the dimension data relationship table.

[0080] When the data source management principle remains unchanged and data is collected again, the re-collected data is matched one by one with the dimension relationship records in the established dimension data relationship table based on the dimension relationship records extracted from the established dimension data relationship table, and the dimensional values ​​of the multi-source data to be integrated are classified. If a new type of dimension value that cannot be matched appears, it indicates that data redundancy occurs. The redundant data type is fed back to the corresponding data source, and the data source removes the redundant data type when packaging the data. If any dimension relationship record is not matched after the classification is completed, it indicates that data is missing. The missing data type is fed back to the corresponding data source, and the data source repackages the missing data and generates a third session key. After the third session key is paired, the data is supplemented and transmitted.

[0081] When the data source management principle changes and data is collected again, the newly collected data type will be matched one by one with the dimension relationship records in the established dimension data relationship table. If there is a dimension relationship record in the established dimension relationship record that matches the dimension value of the new type of data, the existing dimension relationship record will be reused.

[0082] If there is no dimension relationship record matching the dimension value of the new type of data in the established dimension relationship records, the new dimension relationship record is added to the established dimension relationship records.

[0083] After the classification is completed, the dimension relationship records that have not been matched are deleted in the dimension data relationship table, and finally the dimension data relationship that meets the new data source management principle is obtained.

[0084] The dimension value contains the corresponding spatial information, which reflects the collection location of the current dimension value. The spatial information of all dimension values ​​is obtained to construct a geographic block topology map, that is, the plane map is divided into several blocks according to the collection location on the horizontal plane, one block corresponds to one collection location, and the dimension value data is loaded into the corresponding block.

[0085] After all dimension values ​​are loaded into the corresponding blocks, the different dimension values ​​in the blocks are classified based on the dimension data relationship table, and the time information of each type of dimension value is collected.

[0086] Each type of dimension value contains multiple independent dimension value data, and the time information indicates the collection time of the current dimension value data.

[0087] Based on the collection time, the dimension value data is arranged vertically from bottom to top in chronological order. If there is a time interruption in the arrangement, that is, the corresponding time series data is missing, feedback is sent to the data source to request data supplement. If the data source replies that the data is missing due to an abnormal state and cannot be compensated, historical data is called to supplement the missing part.

[0088] The data space coordinate system constructed by the present invention makes data classification more intuitive and neat, and is convenient for sorting and searching.

[0089] Step 4: Create an index for each type of dimension value in the spatial coordinate system, and when the required application begins, find the required data based on the index.

[0090] For each type of dimension value in each block, an index mark is performed according to the results of classification and integration according to the required purposes, that is, one type of dimension value corresponds to one or several required purposes.

[0091] After finding the required data, standardize the data according to the requirements of the model you want to build.

[0092] Embodiment 2 is the second embodiment of the present invention, which is different from the first embodiment in that: a multi-source, multi-dimensional and multi-scale data fusion method also includes: in order to verify and illustrate the technical effects adopted in this method, this embodiment adopts a traditional technical solution and the method of the present invention for comparative testing, and compares the test results by means of scientific demonstration to verify the real effect of this method.

[0093] The traditional database query indexing technology and linear regression statistical model were loaded into the simulation model with our invented technology for testing. The test indicators are as follows.

[0094] 1. Data security test: Experimental setting: simulate the data transmission process and record the number and type of data leakage incidents during the transmission process of the two methods.

[0095] Metric: Number of data breaches.

[0096] 2. Information screening efficiency test: Experimental setup: Apply two methods to the same data set and compare the data volume and processing time after filtering out useless information.

[0097] Metrics: amount of data processed and processing time.

[0098] 3. Data retrieval speed test: Experimental setting: Retrieve specific information from a large-scale data set and record the time required for retrieval.

[0099] Metric: Time to complete data retrieval.

[0100] 4. Retrieval accuracy test: Experimental setup: Compare the accuracy and completeness of data retrieved by the two methods.

[0101] Indicators: accuracy and completeness of retrieval results.

[0102] Finally, the test data are summarized to obtain a comparison table as shown in Table 1.

[0103] Table 1: Test data comparison table

[0104] ,

[0105] Data security: The present invention significantly reduces data leakage incidents through double verification, thereby improving the security of the data transmission process.

[0106] Information screening efficiency: The present invention reduces the entry of useless information, reduces the amount of processed data and reduces processing time through efficient data source management principles and preliminary screening.

[0107] Data retrieval speed: The constructed dimensional data relationship table and data space coordinate system make data classification clearer, thereby improving the retrieval speed.

[0108] Retrieval accuracy: The index system constructed through the scale information in the dimension value enables the present invention to have higher accuracy and completeness when retrieving data.

[0109] Example 3, reference Figure 2 , which is the third embodiment of the present invention, and is different from the previous two embodiments in that: a system of a multi-source, multi-dimensional and multi-scale data fusion method includes a data acquisition module, a preliminary integration module, a secondary integration module and an index establishment module; the data acquisition module determines the data source according to the data source management principle, sends a session key to the data source, and collects data after pairing; the preliminary integration module classifies and integrates the collected data according to the required purpose, separates the corresponding dimension value for each type of data, and separates the scale information from the dimension value, and the scale information includes spatial information and time information; the secondary integration module classifies and integrates the dimension value of each type of data again, sorts the various dimensional values ​​horizontally according to the corresponding spatial information, and on the basis of the horizontal sorting, sorts vertically based on the time information corresponding to the various dimensional values ​​to construct a data space coordinate system; the index establishment module establishes an index for each type of dimensional value on the spatial coordinate system, and when the required purpose begins, the required data is found according to the index.

[0110] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0111] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0112] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0113] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0114] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A multi-source, multi-dimensional and multi-scale data fusion method, characterized by: include, Determine the data source according to the data source management principle, send the session key to the data source, and collect data after pairing; Classify and integrate the collected data according to the required purpose, separate the corresponding dimension value for each type of data, and separate the scale information from the dimension value, wherein the scale information includes spatial information and temporal information; The required uses include data for constructing a hydrological model, data for constructing a hydrodynamic model, and data for constructing a power generation prediction model; The data used to construct the hydrological model include meteorological data, topographic data, watershed boundaries, hydrological and hydraulic data, soil data, and anthropogenic activity data; The dimensional values ​​contained in the meteorological data include rainfall, temperature, evaporation, wind speed and sunshine hours; The dimensional values ​​of the terrain data include watershed boundaries, land cover types, elevations and slopes; The hydrological and hydraulic data may include river flow, water level data, and groundwater depth and flow characteristics; The dimension values ​​contained in the soil data include soil type and soil moisture; The dimension values ​​contained in the human activity data include land use change, construction and operation data of reservoirs, dams and channels; The data used to construct the hydrodynamic model include fluid dynamics data, topographic and geomorphic data, boundary and initial conditions, water quality data, and meteorological data; The dimensional values ​​contained in the fluid dynamics data include water level data, flow velocity and flow direction; The dimensional values ​​contained in the terrain and landform data include river cross-section data, riverbed and lake bottom terrain; The dimensional values ​​contained in the boundary conditions and initial conditions include initial water flow state, flow rate or water level at the inlet and outlet; The dimensional values ​​contained in the water quality data include pollutant concentration, biochemical oxygen demand and chemical oxygen demand; The dimensional values ​​contained in the meteorological data also include atmospheric pressure, wind direction and wind speed; The data used to construct the power generation forecast model include hydrological data, meteorological data, mechanical performance parameters, operating parameters, economic and dispatch parameters, environmental and regulatory parameters and historical operating data; The dimensional values ​​contained in the hydrological data include data on the water flow into the reservoir, the water level of the reservoir and downstream; The mechanical performance parameters contain dimensional values ​​including turbine efficiency, generator efficiency and maximum output capacity; The operating parameters contain dimensional values ​​including: the vertical distance between the reservoir water level and the turbine and the water flow through the generator; The economic and dispatch parameters contain dimensional values ​​including power demand and market price, operation and maintenance costs; The environmental and regulatory parameters contain dimensional values ​​including minimum downstream flows and water resource management policies required by regulations; The dimensional values ​​contained in the historical operation data include past power generation records, historical water level and flow data; Classifying and integrating the dimension values ​​of each type of data further includes defining the data collected from each of the data sources as multi-source data, wherein each data point in the multi-source data includes one or more dimension values; Determine the dimension value of each data in all multi-source data, remove duplicate dimension values ​​of the same type, and map the removed duplicate dimension values ​​to the multi-source data to form a dimension data relationship table; The dimension data relationship table indicates that one or more dimension values ​​correspond to a data point; Generate a corresponding dimension relationship identifier for each dimension relationship record in the dimension data relationship table, and reversely bind the dimension relationship identifier corresponding to each dimension relationship record to the data and make it correspond to the dimension value of each data; Classify dimension values ​​according to the dimension data relationship table; When the data source management principle remains unchanged and data is collected again, based on the dimension relationship records extracted from the established dimension data relationship table, the re-collected data is matched one by one with the dimension relationship records in the established dimension data relationship table, and the multi-source data to be fused is classified for dimension values. If a new type of dimension value that cannot be matched appears, it indicates that data redundancy occurs, and the redundant data type is fed back to the corresponding data source. The data source removes the redundant data type when packaging the data. If any dimension relationship record is not matched after the classification is completed, it indicates that data is missing, and the missing data type is fed back to the corresponding data source. The data source repackages the missing data and generates a third session key. After the third session key is paired, the data is supplemented and transmitted; When the data source management principle changes and data is collected again, the newly collected data will be matched one by one with the dimension relationship records in the established dimension data relationship table. If there is a dimension relationship record in the established dimension relationship record that matches the dimension value of the new data, the existing dimension relationship record will be reused. If there is no dimension relationship record matching the dimension value of the new type of data in the established dimension relationship records, the new dimension relationship record is added to the established dimension relationship records; After the classification is completed, the dimension relationship records that have not been matched are deleted from the dimension data relationship table, and finally a dimension data relationship table that meets the new data source management principle is obtained; The values ​​of various dimensions are sorted horizontally according to the corresponding spatial information. On the basis of the horizontal sorting, the values ​​of various dimensions are sorted vertically based on the time information corresponding to the values ​​of various dimensions to construct a data space coordinate system. An index is created for each type of dimension value in the spatial coordinate system, and when the required application begins, the required data is found according to the index.

2. The multi-source, multi-dimensional, and multi-scale data fusion method according to claim 1, characterized in that: The data source management principles include the type of data to be collected and the source of data collection; The data source management principles are updated in real time as needed.

3. The multi-source, multi-dimensional, and multi-scale data fusion method according to claim 2, characterized in that: The session key generation rule includes that after determining the data source, the data source negotiates with the acquisition end to determine the first session key, and after pairing based on the first session key, the data source and the acquisition end are connected through a transmission channel; According to the type of data to be collected in the data source management principle, the data of the corresponding type is packaged and a second session key is generated. After the second session key is paired, data transmission is performed.

4. The multi-source, multi-dimensional and multi-scale data fusion method according to claim 3, characterized in that: The horizontal sorting of various dimension values ​​according to the corresponding spatial information includes: the dimension values ​​contain corresponding spatial information, the spatial information reflects the collection location of the current dimension value, and the spatial information of all dimension values ​​is obtained to construct a geographical block topology map, that is, the plane map is divided into a plurality of blocks according to the collection location on the horizontal plane, one block corresponds to one collection location, and the dimension value data is loaded into the corresponding block; The vertical sorting based on the time information corresponding to each type of dimension value includes: after all the dimension values ​​are loaded into the corresponding block, different dimension values ​​in the block are classified based on the dimension data relationship table, and the time information of each type of dimension value is collected; Each type of dimension value includes multiple independent dimension value data, and the time information indicates the collection time of the current dimension value data; Based on the collection time, the dimension value data is arranged vertically from bottom to top in chronological order. If there is a time interruption in the arrangement, that is, the corresponding time series data is missing, feedback is sent to the data source to request data supplement. If the data source replies that the data is missing due to an abnormal state and cannot be compensated, historical data is called to supplement the missing part.

5. The multi-source, multi-dimensional, multi-scale data fusion method according to claim 4, characterized in that: The indexing of each type of dimension value in the spatial coordinate system includes, for each type of dimension value in each block, indexing the result of classification and integration according to the required use, that is, one type of dimension value corresponds to one or several required uses; The searching for the required data according to the index also includes, after searching for the required data, standardizing the data according to the requirements of building the model.

6. A system using a multi-source, multi-dimensional, multi-scale data fusion method as claimed in any one of claims 1 to 5, characterized in that: It includes data collection module, preliminary integration module, secondary integration module and index building module; The data acquisition module determines the data source according to the data source management principle, sends a session key to the data source, and collects data after pairing; The preliminary integration module classifies and integrates the collected data according to the required uses, separates the corresponding dimension value for each type of data, and separates the scale information from the dimension value, wherein the scale information includes spatial information and temporal information; the required uses include data for building a hydrological model, data for building a hydrodynamic model, and data for building a power generation prediction model; The data used to construct the hydrological model include meteorological data, topographic data, watershed boundaries, hydrological and hydraulic data, soil data, and anthropogenic activity data; The dimensional values ​​contained in the meteorological data include rainfall, temperature, evaporation, wind speed and sunshine hours; The dimensional values ​​of the terrain data include watershed boundaries, land cover types, elevations and slopes; The hydrological and hydraulic data may include river flow, water level data, and groundwater depth and flow characteristics; The dimension values ​​contained in the soil data include soil type and soil moisture; The dimension values ​​contained in the human activity data include land use change, construction and operation data of reservoirs, dams and channels; The data used to construct the hydrodynamic model include fluid dynamics data, topographic and geomorphic data, boundary and initial conditions, water quality data, and meteorological data; The dimensional values ​​contained in the fluid dynamics data include water level data, flow velocity and flow direction; The dimensional values ​​contained in the terrain and landform data include river cross-section data, riverbed and lake bottom terrain; The dimensional values ​​contained in the boundary conditions and initial conditions include initial water flow state, flow rate or water level at the inlet and outlet; The dimensional values ​​contained in the water quality data include pollutant concentration, biochemical oxygen demand and chemical oxygen demand; The dimensional values ​​contained in the meteorological data also include atmospheric pressure, wind direction and wind speed; The data used to construct the power generation forecast model include hydrological data, meteorological data, mechanical performance parameters, operating parameters, economic and dispatch parameters, environmental and regulatory parameters and historical operating data; The dimensional values ​​contained in the hydrological data include data on the water flow into the reservoir, the water level of the reservoir and downstream; The mechanical performance parameters contain dimensional values ​​including turbine efficiency, generator efficiency and maximum output capacity; The operating parameters contain dimensional values ​​including: the vertical distance between the reservoir water level and the turbine and the water flow through the generator; The economic and dispatch parameters contain dimensional values ​​including power demand and market price, operation and maintenance costs; The environmental and regulatory parameters contain dimensional values ​​including minimum downstream flows and water resource management policies required by regulations; The dimensional values ​​contained in the historical operation data include past power generation records, historical water level and flow data; The secondary integration module classifies and integrates the dimension values ​​of each type of data again, including defining the data collected from each of the data sources as multi-source data, and each data point in the multi-source data contains one or more dimension values; Determine the dimension value of each data in all multi-source data, remove duplicate dimension values ​​of the same type, and map the removed duplicate dimension values ​​to the multi-source data to form a dimension data relationship table; the dimension data relationship table indicates that one or more dimension values ​​correspond to a data point; Generate a corresponding dimension relationship identifier for each dimension relationship record in the dimension data relationship table, reversely bind the dimension relationship identifier corresponding to each dimension relationship record to the data, and correspond to the dimension value of each data; classify the dimension value according to the dimension data relationship table; When the data source management principle remains unchanged and data is collected again, based on the dimension relationship records extracted from the established dimension data relationship table, the re-collected data is matched one by one with the dimension relationship records in the established dimension data relationship table, and the multi-source data to be fused is classified for dimension values. If a new type of dimension value that cannot be matched appears, it indicates that data redundancy occurs, and the redundant data type is fed back to the corresponding data source. The data source removes the redundant data type when packaging the data. If any dimension relationship record is not matched after the classification is completed, it indicates that data is missing, and the missing data type is fed back to the corresponding data source. The data source repackages the missing data and generates a third session key. After the third session key is paired, the data is supplemented and transmitted; When the data source management principle changes and data is collected again, the newly collected data will be matched one by one with the dimension relationship records in the established dimension data relationship table. If there is a dimension relationship record in the established dimension relationship record that matches the dimension value of the new data, the existing dimension relationship record will be reused. If there is no dimension relationship record matching the dimension value of the new type of data in the established dimension relationship records, the new dimension relationship record is added to the established dimension relationship records; After the classification is completed, the dimension relationship records that have not been matched are deleted from the dimension data relationship table, and finally a dimension data relationship table that meets the new data source management principle is obtained; The values ​​of various dimensions are sorted horizontally according to the corresponding spatial information. On the basis of the horizontal sorting, the values ​​of various dimensions are sorted vertically based on the time information corresponding to the values ​​of various dimensions to construct a data space coordinate system. The index building module builds an index for each type of dimension value in the spatial coordinate system, and when the required application begins, the required data is searched according to the index.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a multi-source, multi-dimensional, and multi-scale data fusion method described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a multi-source, multi-dimensional, and multi-scale data fusion method described in any one of claims 1 to 5 are implemented.

Images