Multi-source heterogeneous data quality improvement method and system

Abstract

The present invention relates to the field of data processing technology, and specifically to a method and system for improving the quality of multi-source heterogeneous data. The method comprises: collecting geographic information data, establishing a related area screening model, analyzing to obtain a correlation degree evaluation value of each related area, and screening to obtain each related area; collecting climate remote sensing data of a target area and each related area, extracting a correlation degree evaluation value of each related area, and comprehensively analyzing to obtain a spatial dynamic evaluation value of the target area; collecting climate condition data, establishing a climate change rate model, obtaining a climate change rate evaluation value, comprehensively analyzing to obtain a temporal dynamic evaluation value of the target area, and matching to obtain the climate remote sensing data update frequency of the target area. The present invention provides a method and system for improving the quality of multi-source heterogeneous data, compares and verifies multi-source data, can reduce data errors and deviations, ensure the timeliness and effectiveness of data, and improve the overall quality of data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for improving the quality of multi-source heterogeneous data. Background Art

[0002] With the rapid development of information technology, the Internet of Things, cloud computing and big data technology, the speed, variety and scale of data generation have reached unprecedented heights. Data not only comes from traditional databases and information systems, but is also widely distributed in multiple fields and channels such as social media, mobile applications, sensor networks, and video surveillance. The explosive growth and diversity of this data has led to the emergence of multi-source heterogeneous data, which contains a large amount of uncertainty, redundancy, and noise, making it difficult for the data to play its due value in subsequent analysis and mining.

[0003] For example, the invention patent with announcement number: CN116703228B is a big data quality evaluation method and system. The method includes building a fine-grained quality evaluation index model, building a common quality constraint rule description template, building a basic quality evaluation object model, building a basic quality evaluation object index model, building a multi-task index calculation mechanism, building a multi-strategy task scheduling execution mechanism, building an index calculation task execution result dimensional model, and building an automatic aggregation mechanism for data quality reports.

[0004] For example, the invention patent with announcement number: CN114782027B is a multi-source heterogeneous video data interactive processing method and system for a flexible production line, which involves the field of data processing. The method includes: determining the production line supply chain information; obtaining the supply chain nodes and dividing the supply chain data stages; setting up a container engine for each supply chain data stage; performing multi-source data collection for each supply chain data stage; obtaining data interaction requests and determining data requirements and interaction requirements; determining matching supply chain data according to the data requirements, and determining the container engine corresponding to the supply chain data stage based on the data requirements; performing data format standardization preprocessing on the container engine according to the interaction requirements, and interacting with the preprocessed production line data according to the interaction requirements.

[0005] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application discovered that the above-mentioned technology has at least the following technical problems: the current data quality improvement methods often lack flexibility and scalability, and cannot adapt to the ever-changing data sources and data requirements. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a method and system for improving the quality of multi-source heterogeneous data, which can effectively solve the problems involved in the above-mentioned background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: The first aspect of the present invention provides a method for improving the quality of multi-source heterogeneous data, including: obtaining a target area and related areas of the target area, collecting geographic information data, establishing a related area screening model, analyzing to obtain a correlation degree evaluation value of each related area, and screening to obtain each related area according to the correlation degree evaluation value of each related area.

[0008] The climate remote sensing data of the target area and each related area are collected, and the correlation evaluation value of each related area is extracted, and the spatial dynamic evaluation value of the target area is obtained through comprehensive analysis.

[0009] Collect climate condition data, establish a climate change rate model, and obtain a climate change rate assessment value. Based on the climate change rate assessment value and the spatial dynamic assessment value of the target area, a comprehensive analysis is performed to obtain the temporal dynamic assessment value of the target area. The climate remote sensing data update frequency of the target area is then matched based on the temporal dynamic assessment value of the target area.

[0010] As a further method, the geographic information data specifically includes: the relative distance between the target area and each related area, the average altitude of the target area and each related area, and the proportion of each land use type in the target area and each related area.

[0011] As a further method, the analysis obtains a correlation degree evaluation value for each related area. The specific analysis process is: based on the relative distance between the target area and each related area, the average altitude and the proportion of each land use type, the critical distance, the allowable deviation average altitude and the allowable deviation land use type proportion are extracted from the system database, and a related area screening model is established to obtain a correlation degree evaluation value for each related area. The correlation degree evaluation value for each related area is used to quantitatively evaluate the correlation degree between each related area and the target area, providing a basis for screening related areas.

[0012] As a further method, the screening obtains each associated area, and the specific screening process is: extracting the regional correlation evaluation threshold from the system database, comparing the correlation evaluation value of each associated area with the regional correlation evaluation threshold, if the correlation evaluation value of a certain associated area is higher than or equal to the regional correlation evaluation threshold, then the associated area is marked as an associated area, if the correlation evaluation value of a certain associated area is lower than the regional correlation evaluation threshold, then the associated area is not marked, and the associated areas are obtained by statistics.

[0013] As a further method, the climate remote sensing data of the target area and each associated area specifically include: the precipitation of the target area and each associated area within a preset monitoring period, the average temperature of the target area and each associated area within the preset monitoring period, and the surface evaporation of the target area and each associated area within the preset monitoring period.

[0014] As a further method, the comprehensive analysis obtains the spatial dynamic evaluation value of the target area. The specific analysis process is: extracting the allowable deviation rainfall, the allowable deviation average temperature and the allowable deviation surface evaporation from the system database, and based on the climate remote sensing data of the target area and each related area and the correlation evaluation value of each related area, a comprehensive analysis is performed to obtain the spatial dynamic evaluation value of the target area. The spatial dynamic evaluation value of the target area is used to quantitatively evaluate the comprehensive impact of the climate remote sensing data of each related area on the climate remote sensing data of the target area.

[0015] As a further method, the climate change rate model is established to obtain a climate change rate assessment value. The specific process is: the climate condition data includes: the average temperature of the target area and each related area in each historical monitoring time period and the surface evaporation of the target area and each related area in each historical monitoring time period. The climate change rate model is established based on the climate condition data to obtain a climate change rate assessment value.

[0016] As a further method, the comprehensive analysis obtains the temporal dynamic evaluation value of the target area. The specific analysis process is: based on the climate change rate evaluation value and the spatial dynamic evaluation value of the target area, a comprehensive analysis is performed to obtain the temporal dynamic evaluation value of the target area. The temporal dynamic evaluation value of the target area is used to quantitatively evaluate the impact of the historical climate remote sensing data of each associated area on the climate remote sensing data of the target area.

[0017] As a further method, the matching obtains the update frequency of the climate remote sensing data of the target area. The specific matching process is: extracting a mapping set between the time dynamic evaluation value interval of the target area and the update frequency of the climate remote sensing data of the target area from the system database, inputting the time dynamic evaluation value of the target area, matching to obtain the interval in which the time dynamic evaluation value of the target area is located, and obtaining the update frequency of the climate remote sensing data of the target area through the mapping set.

[0018] The second aspect of the present invention provides a multi-source heterogeneous data quality improvement system, including: a regional correlation evaluation module, a regional spatial dynamic evaluation module, a data update frequency evaluation module and a system database.

[0019] The regional correlation evaluation module is used to obtain the target area and the related areas of the target area, collect geographic information data, establish a related area screening model, analyze and obtain the correlation evaluation value of each related area, and screen each related area according to the correlation evaluation value of each related area.

[0020] The regional spatial dynamic assessment module is used to collect climate remote sensing data of the target area and each related area, extract the correlation evaluation value of each related area, and obtain the spatial dynamic assessment value of the target area through comprehensive analysis.

[0021] The data update frequency assessment module is used to collect climate condition data, establish a climate change rate model, and obtain a climate change rate assessment value. Based on the climate change rate assessment value and the spatial dynamic assessment value of the target area, a comprehensive analysis is performed to obtain a temporal dynamic assessment value of the target area, and the climate remote sensing data update frequency of the target area is obtained based on the temporal dynamic assessment value of the target area.

[0022] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0023] (1) The present invention provides a method and system for improving the quality of multi-source heterogeneous data, comprehensively analyzes the change trends of the data in the target area and the associated areas, and thus predicts the change of the data in the target area. According to the prediction results, the update frequency of the data in the target area can be dynamically adjusted to ensure the timeliness and effectiveness of the data. A reasonable update frequency helps to optimize the use of data collection and processing resources and improve resource utilization efficiency.

[0024] (2) By analyzing the long-term climate information of the associated regions, the present invention can understand the long-term climate change trend of the target region and enhance the consistency and coherence of the data. The data of the associated regions can reflect the climate characteristics of different regions. By comparing the data of different regions, the spatial distribution characteristics of the climate of the target region can be understood, providing a basis for data interpolation and spatial analysis.

[0025] (3) The present invention can screen out associated regions that are significantly correlated with the target region by analyzing the correlation between regions. There is a certain connection or regularity between the data of these associated regions and the data of the target region, thereby improving the correlation of the data. In the process of screening the associated regions, the accuracy and consistency of the data can also be ensured by further verifying and calibrating the data.

[0026] (4) The present invention can reduce data errors and deviations and improve the overall quality of data by comparing and verifying multi-source data. The screened associated area data can fill in the blanks or missing data in the target area, thereby enhancing the integrity of the data. The complete data set helps to understand the situation in the target area more comprehensively, provide more comprehensive support for decision-making, and improve the efficiency of data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0028] Figure 1 It is a schematic diagram of the method flow of the present invention.

[0029] Figure 2 It is a schematic diagram of system module connection of the present invention.

[0030] Figure 3 The figure is a schematic diagram of the functional relationship between the spatial dynamic evaluation value of the target area and the temporal dynamic evaluation value of the target area involved in the embodiment of the present invention. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0032] Reference Figure 1 As shown, the first aspect of the present invention provides a method for improving the quality of multi-source heterogeneous data, including: acquiring a target area and related areas of the target area, collecting geographic information data, establishing a related area screening model, analyzing to obtain a correlation degree evaluation value of each related area, and screening to obtain each related area according to the correlation degree evaluation value of each related area.

[0033] Specifically, the geographic information data specifically includes: the relative distance between the target area and each related area, the average altitude of the target area and each related area, and the proportion of each land use type in the target area and each related area.

[0034] It should be explained that each relevant area is an area whose geographic information data similarity with the target area is higher than 90%, and the relative distance between the target area and each relevant area refers to the shortest straight-line distance between the target area and each relevant area.

[0035] It needs to be explained that land use types specifically include cultivated land, forest land, grassland and construction land. Land use types can be further subdivided into gardens, commercial and service land, industrial and mining storage land, residential land, public administration and public service land, special land, transportation land, water area and water conservancy facilities land.

[0036] It needs to be explained that geographic information data can be easily acquired, stored, managed, analyzed and displayed using software platforms such as GIS.

[0037] Furthermore, the analysis obtains a correlation degree evaluation value for each related area. The specific analysis process is: based on the relative distance between the target area and each related area, the average altitude and the proportion of each land use type, the critical distance, the allowable deviation average altitude and the allowable deviation land use type proportion are extracted from the system database, and a related area screening model is established to obtain a correlation degree evaluation value for each related area. The correlation degree evaluation value for each related area is used to quantitatively evaluate the correlation degree between each related area and the target area, providing a basis for screening related areas.

[0038] In a specific embodiment, the numerical expression of the correlation degree evaluation value of each related area is:

[0039] ;

[0040] in, represents the correlation evaluation value of the nth correlation area, e represents a natural constant, and n represents the number of each correlation area. , m represents the total number of relevant regions, represents the relative distance between the nth related area and the target area, represents the average altitude of the nth related area, Indicates the average altitude of the target area. represents the proportion of the rth land use type in the nth related area, r represents the number of each land use type, , h represents the total number of land use types, represents the proportion of the rth land use type in the target area, represents the critical distance, Indicates the allowable deviation from the mean altitude. Indicates the percentage of land use types with allowable deviation, Indicates the regional correlation influence factor corresponding to the preset relative distance, Indicates the regional correlation factor corresponding to the preset average altitude. Indicates the regional correlation influencing factor corresponding to the preset land use type proportion.

[0041] It needs to be explained that when the relative distance between the target area and the related area is closer, the difference in average altitude is smaller, and the difference in land use type proportion is smaller, the corresponding correlation evaluation value of the related area will be greater, indicating that the correlation between each related area and the target area is higher.

[0042] It should be explained that in this embodiment is the regional correlation factor corresponding to the preset relative distance, is the regional correlation factor corresponding to the preset average altitude, The regional correlation influence factors corresponding to the preset land use type ratio respectively represent the influence of the relative distance, average altitude and land use type ratio unit values ​​on the regional correlation. When used, the regional correlation influence factors corresponding to the relative distance, the regional correlation influence factors corresponding to the average altitude and the regional correlation influence factors corresponding to the land use type ratio can be directly obtained from the system database. The corresponding relationship can be a preset mapping relationship. For example, the relative distance, average altitude and land use type ratio of the region are respectively mapped with the relative distance, average altitude and land use type ratio preset in the system database to form a mapping set. The real-time relative distance, average altitude and land use type ratio of the region are input into the mapping set to obtain the regional correlation influence factors corresponding to the relative distance, the regional correlation influence factors corresponding to the average altitude and the regional correlation influence factors corresponding to the land use type ratio. The mapping relationship can be one-to-one or many-to-one. The above-mentioned influence factors are all extracted from the system database, and the value range is between 0 and 1.

[0043] Table 1 Data example of correlation evaluation values ​​for each relevant area

[0044] serial number Relative distance (meters) Average altitude (m) The proportion of each land use type Relevance evaluation value 1 800 550 0.5、0.25、0.25 0.45 2 1200 600 0.45、0.35、0.2 0.40 3 500 500 0.5、0.3、0.2 0.68 4 1500 450 0.4、0.3、0.3 0.34 5 700 550 0.4、0.3、0.3 0.39

[0045] It should be explained that, in this embodiment, the average altitude of the target area is set to 500 meters, the critical distance is set to 1000 meters, the allowable deviation average altitude is set to 200 meters, and the allowable deviation land use type ratio is set to 0.1. Assuming that the unit of the land use type ratio is a proportion, that is, between 0 and 1, the land use type ratio of the target area is, for example, 0.5 for agricultural land, 0.3 for forest, and 0.2 for construction land. The regional correlation influence factor corresponding to the relative distance is set to 0.3, the regional correlation influence factor corresponding to the average altitude is set to 0.4, and the regional correlation influence factor corresponding to the land use type ratio is set to 0.3.

[0046] It needs to be explained that, usually, areas that are relatively close to each other may have similar terrain and climatic conditions, including average altitude. Adjacent areas often have similar land use types due to the similarity of natural conditions such as climate, soil, and hydrology. For example, two adjacent agricultural areas may both be dominated by crop farming. Average altitude is one of the important factors affecting land use types. For example, in high-altitude areas, due to the cold climate and poor soil, it may be more suitable for the development of animal husbandry rather than crop farming. In low-altitude areas, due to the warm and humid climate and fertile soil, it may be more suitable for the development of crop farming and forestry. Adjacent areas often have similar land use types due to the similarity of natural conditions. Different altitudes may lead to different land use methods and types.

[0047] It needs to be explained that comprehensive analysis of relative distance, average altitude and proportion of each land use type can help identify the complementarity and coordinated development potential between regions. This will help to formulate more scientific and reasonable regional development plans and policy measures, and improve the efficiency of coordinated development between regions. Through comprehensive analysis, we can have a more comprehensive understanding of the ecological environment, resource conditions and development needs between regions. This will help to formulate regional development strategies that are more in line with the requirements of sustainable development and promote sustainable development between regions. Furthermore, the results of comprehensive analysis can provide scientific basis and decision-making support for regional cooperation. For example, in terms of cross-regional infrastructure construction, ecological environmental protection, resource development and utilization, more reasonable cooperation plans and implementation plans can be formulated based on the analysis results.

[0048] Furthermore, the screening obtains the associated areas, and the specific screening process is: extracting the regional correlation evaluation threshold from the system database, comparing the correlation evaluation value of each associated area with the regional correlation evaluation threshold, if the correlation evaluation value of a certain associated area is higher than or equal to the regional correlation evaluation threshold, then marking the associated area as an associated area, if the correlation evaluation value of a certain associated area is lower than the regional correlation evaluation threshold, then not marking the associated area, and obtaining the associated areas by statistics.

[0049] It should be explained that by analyzing the correlation between regions, the associated regions with significant correlation with the target region can be screened out. There are certain connections or regularities between the data of these associated regions and the data of the target region, which improves the correlation of the data. Data with high correlation can better reflect the real situation of the target region and help improve the accuracy and reliability of data analysis. In the process of screening associated regions, the data can be further verified and calibrated to ensure the accuracy and consistency of the data. By comparing and verifying multi-source data, data errors and deviations can be reduced and the overall quality of the data can be improved. The screened associated region data can fill the gaps or missing data in the target region, thereby enhancing the integrity of the data. A complete data set helps to understand the situation in the target region more comprehensively and provide more comprehensive support for decision-making. The data in the associated region often contains more information and details, which can enrich the data set of the target region. Rich data helps to discover more regularities and trends and provide a basis for in-depth analysis. By screening associated regions, the scope of data analysis can be narrowed and the efficiency of data analysis can be improved. Targeted data analysis helps to find and solve problems faster and improve the efficiency of decision-making. Data analysis based on associated regions can more accurately predict the development trends and potential risks of the target region. Optimized data analysis results help to make more scientific decisions and strategies, and improve the effectiveness and success rate of decisions. The process of screening related areas can promote data sharing and communication between different areas. Data sharing helps to break information silos and improve the efficiency and value of data utilization.

[0050] The climate remote sensing data of the target area and each related area are collected, and the correlation evaluation value of each related area is extracted, and the spatial dynamic evaluation value of the target area is obtained through comprehensive analysis.

[0051] Specifically, the climate remote sensing data of the target area and each associated area include: the precipitation of the target area and each associated area within the preset monitoring period, the average temperature of the target area and each associated area within the preset monitoring period, and the surface evaporation of the target area and each associated area within the preset monitoring period.

[0052] It needs to be explained that precipitation data can be collected in real time and continuously using automated equipment such as rain gauges, and large-scale surface temperature data and surface evaporation data can be obtained through satellite remote sensing technology. Precipitation, surface temperature and evaporation can also be obtained by visiting the official website of the meteorological department or related databases.

[0053] Furthermore, the comprehensive analysis obtains a spatial dynamic evaluation value of the target area. The specific analysis process is: extracting the allowable deviation rainfall, the allowable deviation average temperature and the allowable deviation surface evaporation from the system database, and based on the climate remote sensing data of the target area and each related area and the correlation evaluation value of each related area, a comprehensive analysis is performed to obtain the spatial dynamic evaluation value of the target area. The spatial dynamic evaluation value of the target area is used to quantitatively evaluate the comprehensive impact of the climate remote sensing data of each related area on the climate remote sensing data of the target area.

[0054] In a specific embodiment, the numerical expression of the target area spatial dynamic evaluation value is:

[0055] ;

[0056] in, Represents the spatial dynamic evaluation value of the target area, represents the precipitation in the nth associated area, represents the average temperature of the nth associated region, represents the surface evaporation of the nth associated area, represents the precipitation in the target area, represents the average temperature of the target area, represents the surface evaporation of the target area, Indicates the allowable deviation rainfall, Indicates the allowable deviation of the mean temperature, represents the allowable deviation of surface evaporation, Represents the correlation evaluation value of the nth correlation area, Indicates the regional spatial dynamic impact factor corresponding to the preset precipitation, Indicates the regional spatial dynamic impact factor corresponding to the preset average temperature, Indicates the regional spatial dynamic impact factor corresponding to the preset surface evaporation.

[0057] It needs to be explained that when the precipitation difference between the target area and the associated area is smaller, the average temperature difference is smaller, and the land evaporation difference is smaller, the corresponding spatial dynamic evaluation value of the target area will be larger, indicating that the comprehensive influence of the climate remote sensing data of the associated area on the climate remote sensing data of the target area is higher.

[0058] It should be explained that in this embodiment is the regional spatial dynamic impact factor corresponding to the preset precipitation, is the regional spatial dynamic impact factor corresponding to the preset average temperature, The regional spatial dynamic impact factors corresponding to the preset surface evaporation respectively represent the numerical values ​​of the influence of the unit values ​​of precipitation, average temperature and surface evaporation on the regional spatial dynamic change. When used, the regional spatial dynamic impact factors corresponding to precipitation, the regional spatial dynamic impact factors corresponding to average temperature and the regional spatial dynamic impact factors corresponding to surface evaporation can be directly obtained from the system database. The corresponding relationship can be a pre-set mapping relationship. For example, the regional precipitation, average temperature and surface evaporation are respectively mapped with the regional spatial dynamic impact factors corresponding to the preset precipitation, average temperature and surface evaporation in the system database to form a mapping set. The real-time regional precipitation, average temperature and surface evaporation are input into the mapping set to obtain the regional spatial dynamic impact factors corresponding to precipitation, the regional spatial dynamic impact factors corresponding to average temperature and the regional spatial dynamic impact factors corresponding to surface evaporation. The mapping relationship can be one-to-one or many-to-one. The above impact factors are all extracted from the system database, and the value range is between 0 and 1.

[0059] It needs to be explained that precipitation essentially comes from changes in temperature. It is the decrease in temperature that causes water vapor in the air to condense and cause rain. The amount of water vapor that the air can hold is related to the temperature. The higher the temperature, the more water vapor the atmosphere can hold. Therefore, in general, as the temperature rises, precipitation may increase. This is because high temperatures allow the atmosphere to hold more water vapor, and when these water vapor condenses, precipitation will form. Precipitation and surface evaporation are important links in the water cycle, and there is a close relationship between them. Evaporation is the process in which surface water enters the atmosphere in gaseous form, while precipitation is the process in which water vapor in the atmosphere falls to the ground in liquid or solid form. The balance between evaporation and precipitation in a region determines the hydrological characteristics of the region. Average temperature is one of the important factors affecting surface evaporation. Generally speaking, as the temperature rises, surface evaporation will also increase. This is because high temperatures can accelerate the movement of water molecules, making it easier for them to escape from the surface into the atmosphere. At the same time, high temperatures can also reduce the relative humidity of the air, thereby increasing the water vapor holding capacity of the atmosphere and further promoting the evaporation process. In summary, there is a complex and close relationship between regional precipitation, average temperature and surface evaporation. These relationships not only affect the hydrological characteristics and climate types of the region, but also have a profound impact on human production and life. Therefore, we need to conduct in-depth research on the changing laws of these relationships in order to better respond to natural disasters, protect water resources and promote sustainable development.

[0060] It should be explained that by analyzing the correlation between the data of precipitation, average temperature and surface evaporation in the associated area and the target area, the degree of mutual influence between them can be understood. For example, if the precipitation in the associated area increases, it may lead to an increase in the amount of river water in the target area, which in turn affects the water resources in the target area. By analyzing the changing trends of data such as precipitation, average temperature and surface evaporation in the associated area and the target area, their future changing trends can be predicted, and corresponding response measures can be formulated accordingly. For example, if it is predicted that the precipitation in the associated area will decrease, measures may need to be taken in advance to ensure the water resource security of the target area. By analyzing the spatial distribution characteristics of data such as precipitation, average temperature and surface evaporation in the associated area and the target area, their differences and similarities in different regions can be understood. This helps to formulate targeted regional development strategies and resource management measures.

[0061] It should be explained that when the difference between the associated area and the target area in terms of precipitation, average temperature and surface evaporation is small, this usually means that the two areas have a high similarity in these environmental indicators. This similarity may make the target area more susceptible to environmental changes in the associated area, because similar environmental conditions may make certain ecological processes or human activities more correlated between the two areas. For example, if the vegetation type, species distribution, etc. in the associated area change, these changes may be more likely to spread to the target area, thereby affecting the stability of its ecosystem. For human activities, when the environmental indicators of the associated area and the target area are similar, it may be easier for human activities to migrate and adapt between the two areas. This may make the target area more susceptible to changes in human activities in the associated area, such as population migration, resource development, etc.

[0062] Collect climate condition data, establish a climate change rate model, and obtain a climate change rate assessment value. Based on the climate change rate assessment value and the spatial dynamic assessment value of the target area, a comprehensive analysis is performed to obtain the temporal dynamic assessment value of the target area. The climate remote sensing data update frequency of the target area is then matched based on the temporal dynamic assessment value of the target area.

[0063] Specifically, the climate change rate model is established to obtain the climate change rate assessment value. The specific process is as follows: the climate condition data includes: the average temperature of the target area and each related area in each historical monitoring time period and the surface evaporation of the target area and each related area in each historical monitoring time period. The climate change rate model is established based on the climate condition data to obtain the climate change rate assessment value.

[0064] It should be understood that the climate change rate assessment value is a quantitative assessment data obtained through a comprehensive analysis of the average temperature and surface evaporation, which is used to quantitatively assess the climate change rate in each region and provide a basis for assessing the temporal dynamic changes in the target area.

[0065] In a specific embodiment, the climate change rate model is:

[0066] ;

[0067] in, represents the estimated value of climate change rate, represents the average temperature of the dth region in the i-th historical monitoring period, represents the surface evaporation of the dth region in the i-th historical monitoring period, d represents the number of each region, , p represents the total number of regions, i represents the number of each historical monitoring time period, , t represents the total number of historical monitoring time periods, Indicates the climate change rate impact factor corresponding to the preset average temperature, Represents the climate change rate impact factor corresponding to the preset surface evaporation.

[0068] It should be explained that the area includes the target area and various associated areas.

[0069] It needs to be explained that the greater the fluctuations in the average temperature and land evaporation in the region during the historical monitoring period, the greater the corresponding climate change rate assessment value, indicating that the climate change rate in the region is faster.

[0070] It should be explained that in this embodiment Indicates the climate change rate impact factor corresponding to the preset average temperature, Indicates the climate change rate impact factor corresponding to the preset surface evaporation, and respectively indicates the numerical value of the degree of influence of the average temperature and the surface evaporation unit value on the regional climate change rate. When used, the climate change rate impact factor corresponding to the average temperature and the climate change rate impact factor corresponding to the surface evaporation can be directly obtained from the system database. The corresponding relationship can be a pre-set mapping relationship. For example, the average temperature and surface evaporation of the region in the historical monitoring period are respectively mapped with the climate change rate impact factors corresponding to the average temperature and surface evaporation preset in the system database to form a mapping set. The real-time average temperature and surface evaporation of the region in the historical monitoring period are input into the mapping set to obtain the climate change rate impact factor corresponding to the average temperature and the climate change rate impact factor corresponding to the surface evaporation. The mapping relationship can be one-to-one or many-to-one. The above impact factors are all extracted from the system database, and the value range is between 0 and 1.

[0071] It should be explained that by comprehensively analyzing multiple climate indicators, the rate of change and trend of regional climate can be assessed more comprehensively and accurately. The policies and measures to address climate change formulated based on the comprehensive assessment results are more scientific and reasonable, and help reduce the impact of climate change on regional economic and social development. By strengthening the monitoring and assessment of regional climate change, we can promote cooperation and exchanges among regions, jointly respond to climate change challenges, and promote regional sustainable development.

[0072] Furthermore, the comprehensive analysis obtains a temporal dynamic assessment value of the target area. The specific analysis process is: based on the climate change rate assessment value and the spatial dynamic assessment value of the target area, a comprehensive analysis is performed to obtain a temporal dynamic assessment value of the target area. The temporal dynamic assessment value of the target area is used to quantitatively assess the degree of influence of the historical climate remote sensing data of each associated area on the climate remote sensing data of the target area.

[0073] In a specific embodiment, the numerical expression of the time dynamic evaluation value of the target area is:

[0074] ;

[0075] in, Represents the time dynamic evaluation value of the target area, represents the estimated value of climate change rate, Represents the spatial dynamic evaluation value of the target area, Indicates the regional temporal dynamic change impact factor corresponding to the preset climate change rate assessment value, Indicates the regional temporal dynamic change impact factor corresponding to the preset target area spatial dynamic evaluation value.

[0076] It needs to be explained that when the climate change rate assessment value is larger and the spatial dynamic assessment value of the target area is larger, the corresponding temporal dynamic assessment value of the target area will be larger, indicating that the historical climate remote sensing data of the associated area has a higher degree of influence on the climate remote sensing data of the target area.

[0077] It needs to be explained that the rate of climate change will indirectly affect spatial dynamics by affecting the stability of the ecosystem, species distribution or other environmental factors. The greater the assessed value of the climate change rate, the greater the assessed value of the spatial dynamics of the target area will also be.

[0078] It needs to be explained that the climate change rate reflects the magnitude and speed of climate change. When the climate change rate is fast, the climate characteristics of the target area may change significantly in a short period of time, which requires remote sensing data to be updated in a timely manner to accurately reflect these changes. Therefore, when the climate change rate assessment value is high, the update frequency of remote sensing data needs to be increased accordingly to ensure the accuracy and timeliness of the data. The spatial dynamic assessment value of the target area can reflect the impact of the associated regional climate remote sensing data on the target area climate remote sensing data. When the associated regional climate remote sensing data changes rapidly, it means that the climate environment of the target area is also changing rapidly, and the associated regional climate remote sensing data, therefore, needs to be updated more frequently to capture these changes, so as to reduce errors and improve data quality.

[0079] It should be explained that in this embodiment Indicates the regional temporal dynamic change impact factor corresponding to the preset climate change rate assessment value, The regional temporal dynamic change impact factor corresponding to the preset target regional spatial dynamic assessment value is represented, and the numerical value of the influence degree of the unit value of the climate change rate assessment value and the target regional spatial dynamic assessment value on the regional temporal dynamic change is respectively represented. When used, the regional temporal dynamic change impact factor corresponding to the climate change rate assessment value and the regional temporal dynamic change impact factor corresponding to the target regional spatial dynamic assessment value can be directly obtained from the system database. The corresponding relationship can be a preset mapping relationship. For example, the climate change rate assessment value and the target regional spatial dynamic assessment value are respectively mapped with the climate change rate assessment value and the regional temporal dynamic change impact factor corresponding to the target regional spatial dynamic assessment value preset in the system database to form a mapping set. The real-time climate change rate assessment value and the target regional spatial dynamic assessment value are input into the mapping set to obtain the regional temporal dynamic change impact factor corresponding to the climate change rate assessment value and the regional temporal dynamic change impact factor corresponding to the target regional spatial dynamic assessment value. The mapping relationship can be one-to-one or many-to-one. The above impact factors are all extracted from the system database, and the value range is between 0 and 1.

[0080] It should be understood that if Figure 3 As shown, curve a represents the relationship between the spatial dynamic assessment value and the temporal dynamic assessment value of the target area when the climate change rate assessment value is 0.21, curve b represents the relationship between the spatial dynamic assessment value and the temporal dynamic assessment value of the target area when the climate change rate assessment value is 0.48, and curve c represents the relationship between the spatial dynamic assessment value and the temporal dynamic assessment value of the target area when the climate change rate assessment value is 0.96.

[0081] It should be explained that, in this embodiment, the regional temporal dynamic change impact factor corresponding to the climate change rate assessment value is set to 0.5, and the regional temporal dynamic change impact factor corresponding to the target area spatial dynamic assessment value is set to 0.8.

[0082] In a specific embodiment, the matching obtains the update frequency of the climate remote sensing data of the target area, and the specific matching process is: extracting a mapping set between the target area time dynamic evaluation value interval and the target area climate remote sensing data update frequency from the system database, inputting the target area time dynamic evaluation value, matching to obtain the interval in which the target area time dynamic evaluation value is located, and obtaining the target area climate remote sensing data update frequency through the mapping set.

[0083] It needs to be explained that the climate remote sensing data of the target area are updated regularly according to the updating frequency of the climate remote sensing data of the target area.

[0084] It needs to be explained that the historical climate remote sensing data of each associated region can provide similar climate conditions and change trends as those of the target region, thus complementing the data of the target region. By analyzing the differences and similarities of these data, the climate characteristics of the target region can be understood more accurately and the accuracy of the data can be improved. The data of the associated region can be used as a reference for calibrating the data of the target region. In the process of data calibration, errors in the data of the target region can be identified and corrected to improve the accuracy of the data. By analyzing the change trend of the data of the associated region, the change of the data of the target region can also be predicted. According to the prediction results, the update frequency of the data of the target region can be dynamically adjusted to ensure the timeliness and effectiveness of the data. A reasonable update frequency helps to optimize the use of data collection and processing resources. In the case of limited resources, by matching the degree of influence of the data of the associated region, the data with greater impact on the target region can be updated first, thereby improving the efficiency of resource utilization. At the same time, the data of the associated region can provide long-term climate information. By analyzing these time series data, the long-term change trend of the climate of the target region can be understood and the consistency and coherence of the data can be enhanced. The data of the associated region can reflect the climate characteristics of different regions. By comparing the data of different regions, the spatial distribution characteristics of the climate of the target region can be understood, providing a basis for data interpolation and spatial analysis.

[0085] In a specific embodiment, since the remote sensing image of the target area is blocked by clouds and the remote sensing data of the target area is incomplete, the change of the remote sensing data of the target area can be predicted by analyzing the change trend of the remote sensing data of the associated area.

[0086] It should be explained that, on the other hand, for the problem of cloud occlusion in remote sensing images of target areas and associated areas, a semi-supervised method based on generative adversarial network and cloud distortion model can also be proposed to remove thin clouds from unpaired images of different regions, so as to obtain real remote sensing data of each region. First, a cloudy image with cloud-free background and cloud distortion layer decomposed based on generative adversarial network and image decomposition principle is input; then, the cloudy image is put into the redefined cloud distortion physical model for reconstruction. The decomposition process ensures that the decomposed background layer is cloud-free, and the reconstruction process ensures that the generated background layer is related to the input cloudy image. After experiments, the results show that the above method is effective and stable for removing thin clouds in different frequency bands. Among them, the cloudy image with cloud-free background and cloud distortion layer is obtained by extracting the feature difference between cloud images and cloud-free images through a semi-supervised algorithm (Self-Attentive Generative Adversarial Network) based on training self-attention, using a generative adversarial network model, and combining attention mechanism. The main idea of ​​this method is to introduce visual attention in the process to generate "real" cloud-free images.

[0087] It is important to explain that the training process of this method can expand the attention to replace the cloud area map interpretation with cloud-free images, or reduce the attention to match the cloud boundaries and optimize the extreme cases handled by the attention network. The training input of this method is images and image-level labels, which are easier, cheaper, and more time-saving than existing neural network-based methods.

[0088] Reference Figure 2 As shown, the second aspect of the present invention provides a multi-source heterogeneous data quality improvement system, including: a regional correlation evaluation module, a regional spatial dynamic evaluation module, a data update frequency evaluation module and a system database.

[0089] The regional correlation evaluation module is used to obtain the target area and the related areas of the target area, collect geographic information data, establish a related area screening model, analyze and obtain the correlation evaluation value of each related area, and screen each related area according to the correlation evaluation value of each related area.

[0090] The regional spatial dynamic assessment module is used to collect climate remote sensing data of the target area and each related area, extract the correlation evaluation value of each related area, and obtain the spatial dynamic assessment value of the target area through comprehensive analysis.

[0091] The data update frequency assessment module is used to collect climate condition data, establish a climate change rate model, and obtain a climate change rate assessment value. Based on the climate change rate assessment value and the spatial dynamic assessment value of the target area, a comprehensive analysis is performed to obtain a temporal dynamic assessment value of the target area, and the climate remote sensing data update frequency of the target area is obtained based on the temporal dynamic assessment value of the target area.

[0092] In a specific embodiment, the system database is used to store relevant data in the process of updating the climate remote sensing data of the target area for evaluation, including the critical distance, the allowable deviation from the average altitude and the allowable deviation from the land use type ratio, the regional correlation evaluation threshold, the regional correlation influencing factor corresponding to the relative distance, the regional correlation influencing factor corresponding to the average altitude and the regional correlation influencing factor corresponding to the land use type ratio, as well as the data extracted from the system database in the above embodiments. Historical climate data can be obtained by querying the National Meteorological Administration, surface coverage data, national topographic maps and other data can be obtained by querying the geographic information professional knowledge service system, and data in the database can also be obtained through a geographic information system.

[0093] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A method for improving the quality of multi-source heterogeneous data, characterized in that: include: Obtain the target area and the related areas of the target area, collect geographic information data, establish a related area screening model, analyze and obtain the correlation evaluation value of each related area, and screen and obtain each related area according to the correlation evaluation value of each related area; Collect climate remote sensing data of the target area and related areas, extract the correlation evaluation value of each related area, and conduct comprehensive analysis to obtain the spatial dynamic evaluation value of the target area; Collect climate condition data, establish a climate change rate model, obtain a climate change rate assessment value, and obtain a temporal dynamic assessment value of the target area through comprehensive analysis based on the climate change rate assessment value and the spatial dynamic assessment value of the target area. Then, match the temporal dynamic assessment value of the target area to obtain the climate remote sensing data update frequency of the target area. The numerical expression of the correlation evaluation value of each correlation area is: ； in, represents the correlation evaluation value of the nth correlation area, e represents a natural constant, and n represents the number of each correlation area. , m represents the total number of relevant regions, represents the relative distance between the nth related area and the target area, represents the average altitude of the nth relevant area, Indicates the average altitude of the target area. represents the proportion of the rth land use type in the nth related area, r represents the number of each land use type, , h represents the total number of land use types, represents the proportion of the rth land use type in the target area, represents the critical distance, Indicates the allowable deviation from the mean altitude. Indicates the percentage of land use types with allowable deviation, Indicates the regional correlation influence factor corresponding to the preset relative distance, Indicates the regional correlation factor corresponding to the preset average altitude. Indicates the regional correlation influence factor corresponding to the preset land use type proportion; The numerical expression of the target area spatial dynamic evaluation value is: ； in, Represents the spatial dynamic evaluation value of the target area, represents the precipitation in the nth associated area, represents the average temperature of the nth associated region, represents the surface evaporation of the nth associated area, represents the precipitation in the target area, represents the average temperature of the target area, represents the surface evaporation of the target area, Indicates the allowable deviation rainfall, Indicates the allowable deviation of the mean temperature, represents the allowable deviation of surface evaporation, Represents the correlation evaluation value of the nth correlation area, Indicates the regional spatial dynamic impact factor corresponding to the preset precipitation, Indicates the regional spatial dynamic impact factor corresponding to the preset average temperature, Indicates the regional spatial dynamic impact factor corresponding to the preset surface evaporation.

2. The method for improving the quality of multi-source heterogeneous data according to claim 1, characterized in that: The geographic information data specifically include: the relative distance between the target area and each related area, the average altitude of the target area and each related area, and the proportion of each land use type in the target area and each related area.

3. The method for improving the quality of multi-source heterogeneous data according to claim 2, characterized in that: The analysis obtains the correlation evaluation value of each relevant area, and the specific analysis process is as follows: According to the relative distance between the target area and each related area, the average altitude and the proportion of each land use type, the critical distance, the allowable deviation average altitude and the allowable deviation land use type proportion are extracted from the system database, and a related area screening model is established to obtain the correlation degree evaluation value of each related area. The correlation degree evaluation value of each related area is used to quantitatively evaluate the correlation degree between each related area and the target area, providing a basis for screening related areas.

4. The method for improving the quality of multi-source heterogeneous data according to claim 3, characterized in that: The screening obtains the associated regions, and the specific screening process is as follows: The regional correlation evaluation threshold is extracted from the system database, and the correlation evaluation value of each related area is compared with the regional correlation evaluation threshold. If the correlation evaluation value of a related area is higher than or equal to the regional correlation evaluation threshold, the related area is marked as an associated area. If the correlation evaluation value of a related area is lower than the regional correlation evaluation threshold, the related area is not marked, and the associated areas are obtained by statistics.

5. The method for improving the quality of multi-source heterogeneous data according to claim 1, characterized in that: The climate remote sensing data of the target area and each associated area specifically include: the precipitation of the target area and each associated area within a preset monitoring period, the average temperature of the target area and each associated area within the preset monitoring period, and the surface evaporation of the target area and each associated area within the preset monitoring period.

6. The method for improving the quality of multi-source heterogeneous data according to claim 5, characterized in that: The comprehensive analysis obtains the spatial dynamic evaluation value of the target area, and the specific analysis process is as follows: The allowable deviation rainfall, allowable deviation average temperature and allowable deviation surface evaporation are extracted from the system database, and based on the climate remote sensing data of the target area and each associated area and the correlation evaluation value of each associated area, a comprehensive analysis is performed to obtain the spatial dynamic evaluation value of the target area. The spatial dynamic evaluation value of the target area is used to quantitatively evaluate the comprehensive impact of the climate remote sensing data of each associated area on the climate remote sensing data of the target area.

7. The method for improving the quality of multi-source heterogeneous data according to claim 1, characterized in that: The specific process of establishing a climate change rate model and obtaining a climate change rate assessment value is as follows: The climate condition data include: the average temperature of the target area and each associated area in each historical monitoring time period and the surface evaporation of the target area and each associated area in each historical monitoring time period. A climate change rate model is established based on the climate condition data to obtain a climate change rate assessment value.

8. The method for improving the quality of multi-source heterogeneous data according to claim 7, characterized in that: The comprehensive analysis obtains the time dynamic evaluation value of the target area, and the specific analysis process is as follows: Based on the climate change rate assessment value and the spatial dynamic assessment value of the target area, a comprehensive analysis is performed to obtain the temporal dynamic assessment value of the target area, which is used to quantitatively assess the impact of the historical climate remote sensing data of each associated area on the climate remote sensing data of the target area.

9. The method for improving the quality of multi-source heterogeneous data according to claim 8, characterized in that: The matching obtains the updating frequency of the climate remote sensing data of the target area, and the specific matching process is: A mapping set between the time dynamic evaluation value interval of the target area and the climate remote sensing data update frequency of the target area is extracted from the system database, the time dynamic evaluation value of the target area is input, the time dynamic evaluation value of the target area is matched, and the climate remote sensing data update frequency of the target area is obtained through the mapping set.

10. A system for improving the quality of multi-source heterogeneous data according to any one of claims 1 to 9, characterized in that: include: Regional correlation assessment module, regional spatial dynamic assessment module, data update frequency assessment module and system database; The regional correlation evaluation module is used to obtain the target area and the relevant areas of the target area, collect geographic information data, establish a related area screening model, analyze and obtain the correlation evaluation value of each related area, and screen each related area according to the correlation evaluation value of each related area; The regional spatial dynamic assessment module is used to collect climate remote sensing data of the target area and each related area, extract the correlation evaluation value of each related area, and obtain the spatial dynamic assessment value of the target area through comprehensive analysis; The data update frequency assessment module is used to collect climate condition data, establish a climate change rate model, and obtain a climate change rate assessment value. Based on the climate change rate assessment value and the spatial dynamic assessment value of the target area, a comprehensive analysis is performed to obtain a temporal dynamic assessment value of the target area, and the climate remote sensing data update frequency of the target area is obtained based on the temporal dynamic assessment value of the target area.

Images