Ambient temperature data collection and management method and system based on big data

Abstract

The present application relates to the field of temperature measurement technology, and specifically to an environmental temperature data collection and management method and system based on big data. The method comprises: obtaining a first temperature data set of a preset time period of a target area; mapping the first temperature data set into a two-dimensional data table; obtaining a growth algorithm for the data points of the two-dimensional data table, and performing regional growth on the data points of the two-dimensional data table; using Huffman coding to compress the data after the growth is completed, and obtaining the meteorological temperature storage data of the preset time period of the target area. The embodiment of the present application can integrate similar data into multiple connected domain data sets, generate a second data set, and then perform Huffman coding compression on the second data set, thereby improving data compression efficiency, reducing the storage space of meteorological temperature storage data, and alleviating the data storage pressure of the meteorological department.

Description

Technical Field

[0001] The present application belongs to the field of temperature measurement technology, and specifically relates to an environmental temperature data collection and management method and system based on big data. Background Art

[0002] Ambient temperature data refers to the temperature data recorded in the target area over a period of time in the past. It is usually based on date and time and is a time series data. By analyzing the historical temperature data of the target area, we can obtain the climate change trend of the area and formulate corresponding countermeasures based on the climate change trend. However, due to the huge amount of historical temperature data obtained by monitoring and the large amount of redundant data, directly storing the original data will put pressure on the storage system of the meteorological department, resulting in low data compression efficiency. Summary of the invention

[0003] In order to solve the above problem, according to a first aspect of an embodiment of the present application, a method for collecting and managing ambient temperature data based on big data is provided, the method comprising:

[0004] Acquire a first temperature data set of a preset time period of a target area, wherein the first temperature data set includes ambient temperature data collected by different temperature collection sites of the target area at preset time intervals within the preset time period;

[0005] Mapping the first temperature data set into a two-dimensional data table, wherein the row direction of the two-dimensional data table is the collection time point, the column direction is the position of different collection sites, and the data point of the two-dimensional data table is the ambient temperature data collected at the corresponding position at the corresponding time;

[0006] Perform regional growth on the data points of the two-dimensional data table, determine the first growth criterion compliance of the corresponding point to be grown according to the data distribution of the initial seed point and the points to be grown in its neighborhood, and perform regional growth based on the first growth criterion compliance; calculate the adjustment parameters of the difference degree of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table during the regional growth process; obtain the second growth criterion compliance of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table during the regional growth process based on the adjustment parameters; perform regional growth on the data points to be grown in the two-dimensional data table based on the second growth criterion compliance, and obtain the corresponding temperature correction parameters according to the distribution of the data points around the target seed point during the regional growth process;

[0007] The compliance degree of the third growth criterion is determined according to the temperature correction parameter; based on the compliance degree of the third growth criterion, regional growth is performed on the data points to be grown in the two-dimensional data table, and Huffman coding is used to compress the data based on the regional growing results to obtain the meteorological temperature storage data of the preset time period of the target area.

[0008] Optionally, determining the first growth criterion compliance of the corresponding point to be grown according to the data distribution of the initial seed point and the point to be grown in its neighborhood, and performing regional growth based on the first growth criterion compliance includes:

[0009] Obtaining a seed temperature value of the initial seed data point;

[0010] Obtaining a temperature value of a point to be grown in a neighborhood of the initial seed point;

[0011] Obtaining the first growth criterion compliance degree according to the seed temperature value, the to-be-grown temperature value and a preset degree of difference;

[0012] When the compliance degree of the first growth criterion is greater than or equal to zero, determining the to-be-grown point as a new growing point;

[0013] When the compliance degree of the first growth criterion is less than zero, it is determined that the to-be-grown point is not a new growth point.

[0014] Optionally, the adjustment parameters of the difference degree of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table during the calculation region growing process include:

[0015] Obtaining a first temperature change trend of data points in a target time window corresponding to a first seed data point, wherein the first temperature change trend is an absolute value of a maximum temperature difference of surrounding data points of the first seed data point in the target time window;

[0016] Obtaining a second temperature change trend of peripheral data points within the target time window corresponding to the second seed data point, where the second temperature change trend is an absolute value of a maximum temperature difference of peripheral data points of the second seed data point within the target time window;

[0017] The adjustment parameter of the preset degree of difference is obtained according to the ratio of the first temperature change trend to the second temperature change trend, wherein the first seed data point is a seed data point and the second seed data point is an adjacent seed data point of the seed data point.

[0018] Optionally, obtaining the target time window corresponding to the first seed data point includes:

[0019] Performing straight-line fitting on the temperature data of all data points within a first preset time window centered on the first seed data point to obtain an initial fitting error;

[0020] Performing straight line fitting on the temperature data of all data points within a second preset time window with the first seed data point as the left boundary to obtain a right-side fitting error;

[0021] Performing straight line fitting on the temperature data of all data points within a second preset time window with the first seed data point as the right boundary to obtain a left fitting error;

[0022] If the initial fitting error is greater than the right fitting error, and the initial fitting error is greater than the left fitting error, the first seed data point is used as a segmentation point;

[0023] The data points are divided based on all segmentation points, and the data segment where the first seed data point is located is used as the target time window corresponding to the first seed data point.

[0024] Optionally, the following formula is used to calculate the compliance degree of the second growth criterion of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table:

[0025]

[0026] in, For the The first The conformity of the second growth criterion of the data points to be grown, For the The difference degree adjustment parameter of the seed data points is For the acceptable degree of temperature difference, For the The temperature value of the seed data point, For the The first The temperature value of the data point to be grown.

[0027] Optionally, obtaining corresponding temperature correction parameters according to the distribution of data points around the target seed point during the region growing process includes:

[0028] Acquire abnormal data points, where the abnormal data points are data points that cannot be connected to the target seed point in the eight neighborhoods of the target seed point when regional growing is performed according to the first growth criterion compliance;

[0029] The normalized result of the product of the number of data points that have not grown out in the eight neighborhoods of the abnormal data point and the total number of data points in the eight neighborhoods of the abnormal data point is used as the temperature correction parameter corresponding to the abnormal data point.

[0030] Optionally, determining the compliance degree of the third growth criterion according to the temperature correction parameter includes:

[0031] Obtaining the abnormality degree of the abnormal data point;

[0032] The third growth criterion compliance of the abnormal data point is obtained according to the temperature correction parameter and the abnormal degree.

[0033] Optionally, obtaining the abnormality degree of the abnormal data point includes:

[0034] Obtaining a third temperature change trend of surrounding data points of the seed data point within a first preset time window, wherein the third temperature change trend is an absolute value of a maximum temperature difference of surrounding data points of the seed data point within the first preset time window;

[0035] Acquire a fourth temperature change trend of surrounding data points of the abnormal data point within the first preset time window, wherein the fourth temperature change trend is an absolute value of a maximum temperature difference of surrounding data points of the abnormal data point within the first preset time window;

[0036] determining the similarity between the abnormal data point and the seed data point according to the absolute value of the difference between the third temperature change trend and the fourth temperature change trend;

[0037] The average of the similarities between all the abnormal data points and the seed data point is used as the abnormality degree of the abnormal data point.

[0038] Optionally, the third growth criterion compliance of the abnormal data point is calculated using the following formula:

[0039]

[0040] in, For the The g-th abnormal data point in the eight-neighborhood of the target seed point is relative to the g-th abnormal data point. The third growth criterion compliance of the target seed point, For the The adjustment parameter of the difference degree of target seed points, is the preset difference degree, is the temperature correction parameter of the gth abnormal data point in the eight-neighborhood of the i-th target seed point, For the The temperature value of the target seed point, For the The abnormality degree of the abnormal area of ​​the target seed point, For the The temperature value of the gth abnormal data point in the eight-neighborhood of the target seed point, Indicates the absolute value sign.

[0041] According to a second aspect of an embodiment of the present application, there is provided an ambient temperature data acquisition and management system based on big data, the system comprising a server, the server comprising:

[0042] a memory having a computer program stored thereon;

[0043] A processor is used to execute the computer program in the memory to implement the steps of any method described in the first aspect.

[0044] In summary, the embodiment of the present application provides an environmental temperature data collection and management method based on big data, the method comprising: obtaining a first temperature data set of a preset time period of a target area, the first temperature data set comprising environmental temperature data collected by different temperature collection sites of the target area at preset time intervals within the preset time period; mapping the first temperature data set into a two-dimensional data table; obtaining a growth algorithm for the data points of the two-dimensional data table, integrating the data points of the two-dimensional data table according to the growth algorithm, and obtaining a plurality of connected domain data sets; and then using Huffman coding to perform data compression to obtain the meteorological temperature storage data of the preset time period of the target area. The embodiment of the present application can integrate similar data into a plurality of connected domain data sets, and then perform Huffman coding compression, thereby improving data compression efficiency, reducing the storage space of meteorological temperature storage data, and alleviating the data storage pressure of the meteorological department. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the implementation scheme of the present application, the drawings required for use in the implementation scheme will be briefly introduced below. It should be understood that the drawings only show certain implementation schemes of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on the drawings without paying creative work.

[0046] Figure 1 is a flow chart of an environmental temperature data collection and management method based on big data according to an exemplary embodiment;

[0047] Figure 2 is a structural block diagram of an environmental temperature data acquisition and management system based on big data according to an exemplary embodiment;

[0048] Figure 3 It is a block diagram of a server according to an exemplary embodiment. DETAILED DESCRIPTION

[0049] In order to clearly illustrate the technical features of the present solution, the present application is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0050] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0051] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0052] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0053] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0054] It should be noted that the modifications of "one" and "multiple" mentioned in this application are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more". In the description of this application, unless otherwise specified, "multiple" means two or more than two, and other quantifiers are similar; "at least one item", "one or more items" or similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one item a can represent any number of a; for another example, one or more items of a, b and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple; "and / or" is a kind of association relationship that describes the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural.

[0055] Although the operations or steps are described in a specific order in the drawings in the embodiments of the present application, it should not be understood that it is required to perform these operations or steps in the specific order shown or in a serial order, or to perform all the operations or steps shown to obtain the desired results. In the embodiments of the present application, these operations or steps can be performed in series; these operations or steps can also be performed in parallel; or some of these operations or steps can be performed.

[0056] First, the application scenario of the present application is described. The applicant found that when losslessly compressing the temperature data obtained from multiple temperature monitoring stations in a region, the compression efficiency of the data using Huffman coding is related to the amount of redundant data in the data to be compressed. Huffman coding has a higher compression efficiency for data containing more repeated values. Therefore, a growth algorithm is constructed to merge temperature data with similar values ​​into multiple connected areas with similar attributes through a growth criterion. A larger connected domain containing more data is grown for seed points located where the data change trend is relatively gentle; a smaller connected domain containing less data is grown for seed points located where the data change trend is relatively drastic. Huffman coding compression is performed on larger connected domains to avoid compressing smaller connected domains or data that is not in the connected domain, thereby improving the efficiency of data compression and reducing the space occupied by data storage. The present application is described below in conjunction with specific embodiments.

[0057] Figure 1 FIG. 1 is a flow chart of a method for collecting and managing ambient temperature data based on big data according to an exemplary embodiment. Figure 1 As shown, the embodiment of the present application provides an environmental temperature data collection and management method based on big data, and the method may include the following steps:

[0058] In step S10, a first temperature data set of a preset time period of a target area is obtained, wherein the first temperature data set includes ambient temperature data collected by different temperature collection sites of the target area at preset time intervals within the preset time period.

[0059] In this step, the first temperature data set of the target area in a preset time period is obtained. , the first temperature data set The data includes the ambient temperature data collected by different temperature collection sites in the target area at preset time intervals within a preset time period. For example, the preset time period can be one month, one quarter, half a year or one year. The time interval can be days or hours.

[0060] In step S20, the first temperature data set is mapped into a two-dimensional data table, wherein the row direction of the two-dimensional data table is the collection time point, the column direction is the position of different collection sites, and the data points of the two-dimensional data table are the ambient temperature data collected at the corresponding position at the corresponding time.

[0061] In this step, the first temperature data set is mapped into a two-dimensional data table, wherein the row direction of the two-dimensional data table is the collection time point, the column direction is the location of different collection sites, and the data points of the two-dimensional data table are the ambient temperature data collected at the corresponding location at the corresponding time.

[0062] In step S30, regional growth is performed on the data points of the two-dimensional data table, and the first growth criterion compliance of the corresponding points to be grown is determined according to the data distribution of the initial seed point and the points to be grown in its neighborhood, and regional growth is performed based on the first growth criterion compliance; adjustment parameters for the degree of difference of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table during the regional growing process are calculated; based on the adjustment parameters, the second growth criterion compliance of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table during the regional growing process is obtained; based on the second growth criterion compliance, regional growth is performed on the data points to be grown in the two-dimensional data table, and corresponding temperature correction parameters are obtained according to the distribution of data points around the target seed point during the regional growing process.

[0063] In this step, a growth algorithm for the data points of the two-dimensional data table is obtained, and the growth algorithm is used to grow and connect the data points whose temperature changes are within a preset difference degree into a connected domain data set. Exemplarily, an initial seed point can be randomly selected from the first temperature data set, and then the first growth criterion compliance of the points to be grown in the neighborhood of the initial seed point in the two-dimensional data table is obtained, and then the growth algorithm of the points to be grown in the two-dimensional data table is obtained according to the first growth criterion compliance. Among them, the preset difference degree is the preset temperature change amplitude.

[0064] In step S301, a seed data set of the first temperature data set is obtained, where the seed data set includes typical ambient temperature data of the target area in a preset time period.

[0065] In this step, the first temperature data set is obtained The seed dataset , seed dataset Includes typical ambient temperature data for a target area over a preset time period.

[0066] In step S302, the first growth criterion compliance of the surrounding data points to be grown of the seed data point in the two-dimensional data table is obtained.

[0067] In this step, the first growth criterion conformity between the initial seed point in the two-dimensional data table and the points to be grown in its neighborhood is obtained. Exemplarily, the seed temperature value of the initial seed point can be obtained first, and then the temperature values ​​of the points to be grown in the neighborhood of the initial seed point can be obtained, and then the first growth criterion conformity can be obtained according to the seed temperature value, the temperature value to be grown and the preset difference degree.

[0068] In step S303, a growth algorithm for the data points to be grown in the two-dimensional data table is obtained according to the compliance degree of the first growth criterion.

[0069] In this step, a growth algorithm for the points to be grown in the two-dimensional data table is obtained according to the first growth criterion compliance.

[0070] The specific steps include:

[0071] In step S3021, the seed temperature value of the initial seed point is obtained.

[0072] In this step, the initial seed point is obtained The seed temperature value .

[0073] In step S3022, the temperature values ​​of the points to be grown in the neighborhood of the initial seed point are obtained.

[0074] In this step, get the initial seed point The growth temperature value of the growth point j in the neighborhood of .

[0075] In step S3023, the first growth criterion compliance is obtained according to the seed temperature value, the to-be-grown temperature value and the preset difference degree.

[0076] In this step, the temperature value of the initial seed point is , the growth temperature value The degree of difference from the preset , and obtain the first growth criterion compliance . Exemplarily, the first growth criterion conformity It can be obtained by the following formula:

[0077] Formula 1

[0078] in, For the The first The first growth criterion compliance of the growth points to be grown, For the The temperature value of the initial seed point, For the The first The temperature value of the growth point, is the preset temperature difference, indicating the The acceptable temperature difference of the initial seed point is Can be 1.

[0079] The specific steps include:

[0080] In step S3031, when the compliance degree of the first growth criterion is greater than or equal to zero, the data point to be grown is determined to be a new growth point.

[0081] In this step, the first growth criterion meets the When it is greater than or equal to zero, the data point j to be grown is determined to be relative to the initial seed point The new growth point. That is, the growth point j can be the same as the initial seed point merged into one connected domain.

[0082] In step S3032, when the compliance degree of the first growth criterion is less than zero, it is determined that the data point to be grown is not a new growth point.

[0083] In this step, the first growth criterion meets the When it is less than zero, it is determined that the data point j to be grown is not relative to the initial seed point The new growth point. That is, the data point j to be grown cannot be the same as the initial seed point merged into one connected domain.

[0084] In step S304, the second growth criterion compliance of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table.

[0085] In this step, the second growth criterion compliance of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table is obtained. Exemplarily, a preset adjustment parameter of the degree of difference can be obtained first, and then the second growth criterion compliance of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table is obtained based on the adjustment parameter.

[0086] In step S305, a growth algorithm for the data points to be grown in the two-dimensional data table is obtained according to the compliance degree of the second growth criterion.

[0087] In this step, a growth algorithm for the data point to be grown in the two-dimensional data table is obtained according to the second growth criterion conformity. Exemplarily, when the second growth criterion conformity is greater than or equal to zero, the data point to be grown is determined to be a new growth point; when the second growth criterion conformity is less than zero, the data point to be grown is determined not to be a new growth point.

[0088] In step S3041, the adjustment parameter of the preset difference degree is obtained.

[0089] In this step, the adjustment parameter of the preset difference degree is obtained. Exemplarily, the first temperature change trend of the data points in the target time window corresponding to the first seed data point can be obtained first, and the first temperature change trend is the absolute value of the maximum temperature difference of the surrounding data points of the first seed data point in the target time window.

[0090] Obtain a second temperature change trend of the surrounding data points within the target time window corresponding to the second seed data point, wherein the second temperature change trend is the absolute value of the maximum temperature difference of the surrounding data points of the second seed data point within the target time window; obtain an adjustment parameter of the preset degree of difference according to the ratio of the first temperature change trend to the second temperature change trend, wherein the first seed data point is a seed data point, and the second seed data point is an adjacent seed data point of the seed data point.

[0091] Among them, the acquisition process of the target time window is: perform straight-line fitting on the temperature data of all data points in the first preset time window with the first seed data point as the center to obtain the initial fitting error; perform straight-line fitting on the temperature data of all data points in the second preset time window with the first seed data point as the left boundary to obtain the right fitting error; perform straight-line fitting on the temperature data of all data points in the second preset time window with the first seed data point as the right boundary to obtain the left fitting error; if the initial fitting error is greater than the right fitting error, and the initial fitting error is greater than the left fitting error, then use the first seed data point as the segmentation point. In this embodiment, the size of the first preset time window is , the size of the second preset time window is In other embodiments, it can be set according to specific circumstances.

[0092] The data points are divided based on all segmentation points, and the data segment where the first seed data point is located is used as the target time window corresponding to the first seed data point.

[0093] In step S3042, the second growth criterion compliance of the surrounding data points to be grown of the seed data point in the two-dimensional data table is obtained according to the adjustment parameter.

[0094] In this step, the parameters are adjusted according to , the following formula is used to calculate the conformity of the second growth criterion of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table:

[0095] Formula 2

[0096] in, For the The first The conformity of the second growth criterion of the data points to be grown, For the The difference degree adjustment parameter of the seed data points is For the acceptable degree of temperature difference, For the The temperature value of the seed data point, For the The first The temperature value of the data point to be grown.

[0097] Acquiring the adjustment parameter of the preset difference degree may include the following steps:

[0098] In step S30411, a first temperature change trend of peripheral data points within a first preset time window centered on the first seed data point is obtained, where the first temperature change trend is the absolute value of the maximum temperature difference of peripheral data points of the first seed data point within the first preset time window.

[0099] In this step, the first temperature change trend of the surrounding data points within the first preset time window with the first seed data point i as the center is obtained. The first temperature change trend is the absolute value of the maximum temperature difference of the surrounding data points of the first seed data point i within the first preset time window (for example, the time of 7 data points). .

[0100] In step S30412, a second temperature change trend of the peripheral data points centered on the second seed data point within the first preset time window is obtained, where the second temperature change trend is the absolute value of the maximum temperature difference of the peripheral data points of the second seed data point within the first preset time window.

[0101] In this step, the second seed data point is obtained The second temperature change trend is the second seed data point in the first preset time window (e.g., the time of 7 data points). The absolute value of the maximum temperature difference of the surrounding data points .

[0102] In step S30413, the adjustment parameter of the preset degree of difference is obtained according to the ratio of the first temperature change trend to the second temperature change trend, wherein the first seed data point is the target seed point and the second seed data point is an adjacent seed data point of the target seed point.

[0103] In this step, according to the first temperature change trend The second temperature trend ratio, and obtain the adjustment parameters of the preset difference degree , where the first seed data point is the target seed point i, and the second seed data point is the adjacent seed data point of the target seed point .

[0104] Exemplary, preset degree of difference Adjustment parameters It can be obtained by the following formula:

[0105] Formula 3

[0106] Wherein, β is a constant greater than zero and less than 1. In this embodiment, the value of β is 0.01. In specific applications, the implementer can set it according to specific circumstances.

[0107] In step S306, abnormal data points are obtained, where the abnormal data points are data points that cannot be connected to the target seed point in the eight neighborhoods of the target seed point when regional growing is performed according to the first growth criterion compliance.

[0108] In step S307, the abnormality degree of the abnormal data point is obtained.

[0109] In this step, the degree of abnormality of the abnormal data point area is obtained. Exemplarily, the third temperature change trend of the surrounding data points of the seed data point within the first preset time window can be obtained first, the third temperature change trend is the absolute value of the maximum temperature difference of the surrounding data points of the seed data point within the first preset time window, and then the fourth temperature change trend of the surrounding data points of the abnormal data point within the first preset time window is obtained, the fourth temperature change trend is the absolute value of the maximum temperature difference of the surrounding data points of the abnormal data point within the first preset time window, and then the similarity between the abnormal data point and the seed data point is determined according to the absolute value of the difference between the third temperature change trend and the fourth temperature change trend, and finally the average of the similarities between all abnormal data points in the abnormal data point area and the seed data point is used as the degree of abnormality of the abnormal data point.

[0110] Among them, in the step of obtaining the third temperature change trend, the third temperature change trend of the surrounding data points of the target seed point within the first preset time window is obtained, and the third temperature change trend is the absolute value of the maximum temperature difference of the surrounding data points of the target seed point within the first preset time window.

[0111] In the step of obtaining the fourth temperature change trend, the fourth temperature change trend of the surrounding data points of the abnormal data point within the first preset time window is obtained, and the fourth temperature change trend is the absolute value of the maximum temperature difference of the surrounding data points of the abnormal data point within the first preset time window.

[0112] In the step of obtaining similarity, the similarity between each abnormal data point and the target seed point is determined according to the absolute value of the difference between the third temperature change trend and the fourth temperature change trend.

[0113] The mean of the similarities between all abnormal data points and the target seed point is taken as the abnormal degree of the abnormal data point.

[0114] In step S308, the temperature correction parameter of the abnormal data point is obtained.

[0115] The normalized result of the product of the number of data points that have not grown in the eight neighborhoods of the abnormal data point and the total number of data points in the eight neighborhoods of the abnormal data point is used as the temperature correction parameter corresponding to the abnormal data point. The normalization can be processed by the maximum and minimum value normalization method.

[0116] In step S40, the compliance degree of the third growth criterion is determined according to the temperature correction parameter; based on the compliance degree of the third growth criterion, regional growth is performed on the data points to be grown in the two-dimensional data table, and Huffman coding is used to compress the data based on the regional growing results to obtain the meteorological temperature storage data of the preset time period of the target area.

[0117] In this step, the parameters and the degree of abnormality are corrected according to the temperature. , obtain the third growth criterion compliance of the abnormal data point. Exemplarily, the third growth criterion compliance can be obtained by the following formula:

[0118]

[0119]

[0120] in, is the third growth criterion compliance, indicating The g-th abnormal data point in the eight-neighborhood of the target seed point is relative to the g-th abnormal data point. The growth criterion compliance of the target seed point, For the The difference degree adjustment parameter of target seed points is is the preset difference degree, is the temperature correction parameter of the gth abnormal data point in the eight-neighborhood of the i-th target seed point, For the The temperature value of the target seed point, For the The abnormality degree of the abnormal area of ​​the target seed point, For the The temperature value of the gth abnormal data point in the eight-neighborhood of the target seed point, represents the number of data points that have not been grown in the eight-neighborhood of the g-th abnormal data point in the eight-neighborhood of the i-th target seed point, represents the total number of data points in the eight-neighborhood of the g-th abnormal data point in the eight-neighborhood of the i-th target seed point, Indicates the absolute value sign, The target seed point is a data point for completing the regional growth based on the compliance of the second growth criterion.

[0121] Next, according to the third growth criterion compliance, we get the first A growth algorithm for abnormal data points in the eight-neighborhood of a target seed point.

[0122] For example, when the third growth criterion compliance is greater than or equal to zero, the first The abnormal data points in the eight neighborhoods of the target seed point are new growth points; when the conformity of the third growth criterion is less than zero, the first The abnormal data points in the eight neighborhoods of the target seed point are not new growth points.

[0123] This way you can The abnormal data points in the eight neighborhoods of the target seed point that meet the third growth criterion compliance condition are merged into the In the connected domain of the target seed points, the amount of data in the connected domain is further increased, and the data compression efficiency is improved.

[0124] At this point, the above method is used to merge the data points of the two-dimensional data table according to the growth algorithm, and the ambient temperature data is divided into multiple connected domains.

[0125] In this step, Huffman coding is used to compress the data in each connected domain to obtain the meteorological temperature storage data of the preset time period of the target area. Huffman coding is a prior art and will not be described in detail here.

[0126] To sum up, the embodiment of the present application provides an environmental temperature data collection and management method based on big data, which can perform Huffman coding compression on the environmental temperature data after regional growth is completed, thereby improving data compression efficiency, reducing the storage space of meteorological temperature storage data, and alleviating the data storage pressure of the meteorological department.

[0127] The present application also provides a computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the environmental temperature data acquisition and management method based on big data provided by the present application are implemented.

[0128] Figure 2 FIG. 1 is a block diagram of an environmental temperature data collection and management system based on big data according to an exemplary embodiment. Figure 2 As shown, an embodiment of the present application provides an environmental temperature data acquisition and management system 1000 based on big data, including a server 1100.

[0129] Figure 3 is a block diagram of a server according to an exemplary embodiment. Figure 3 The server 1100 may include a processor 1122, which further includes one or more processors, and a memory resource represented by a memory 1132 for storing instructions executable by the processor 1122, such as an application. The application stored in the memory 1132 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 1122 is configured to execute instructions to perform the above-mentioned environmental temperature data collection and management method based on big data.

[0130] The server 1100 may also include a power component 1126 configured to perform power management of the server 1100, a communication component 1150 configured to connect the server 1100 to a network, and an input / output interface 1158. The server 1100 may operate based on an operating system stored in the memory 1132.

[0131] In another exemplary embodiment, a computer program product is also provided, which includes a computer program that can be executed by a programmable electronic device, and the computer program has a code portion for executing the above-mentioned big data-based ambient temperature data acquisition management method when executed by the programmable electronic device.

[0132] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A method for collecting and managing ambient temperature data based on big data, characterized in that: The method comprises: Acquire a first temperature data set of a preset time period of a target area, wherein the first temperature data set includes ambient temperature data collected by different temperature collection sites of the target area at preset time intervals within the preset time period; Mapping the first temperature data set into a two-dimensional data table, wherein the row direction of the two-dimensional data table is the collection time point, the column direction is the position of different collection sites, and the data point of the two-dimensional data table is the ambient temperature data collected at the corresponding position at the corresponding collection time point; Perform regional growth on the data points of the two-dimensional data table, determine the first growth criterion compliance of the corresponding point to be grown according to the data distribution of the initial seed point and the points to be grown in its neighborhood, and perform regional growth based on the first growth criterion compliance; calculate the adjustment parameters of the difference degree of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table during the regional growth process; obtain the second growth criterion compliance of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table during the regional growth process based on the adjustment parameters; perform regional growth on the data points to be grown in the two-dimensional data table based on the second growth criterion compliance, and obtain the corresponding temperature correction parameters according to the distribution of the data points around the target seed point during the regional growth process; Determine the compliance degree of the third growth criterion according to the temperature correction parameter; perform regional growth on the data points to be grown in the two-dimensional data table based on the compliance degree of the third growth criterion, and use Huffman coding to perform data compression based on the regional growth result to obtain the meteorological temperature storage data of the preset time period of the target area; The step of determining the first growth criterion compliance of the corresponding point to be grown according to the data distribution of the initial seed point and the points to be grown in its neighborhood, and performing regional growth based on the first growth criterion compliance includes: Obtaining a seed temperature value of the initial seed point; Obtaining a temperature value of a point to be grown in a neighborhood of the initial seed point; Obtaining the first growth criterion compliance degree according to the seed temperature value, the to-be-grown temperature value and a preset difference degree; When the compliance degree of the first growth criterion is greater than or equal to zero, determining the to-be-grown point as a new growing point; When the compliance degree of the first growth criterion is less than zero, determining that the to-be-grown point is not a new growth point; The adjustment parameters for the difference degree of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table during the calculation region growing process include: Obtaining a first temperature change trend of data points in a target time window corresponding to a first seed data point, wherein the first temperature change trend is an absolute value of a maximum temperature difference of surrounding data points of the first seed data point in the target time window; Obtaining a second temperature change trend of peripheral data points within the target time window corresponding to the second seed data point, where the second temperature change trend is an absolute value of a maximum temperature difference of peripheral data points of the second seed data point within the target time window; According to the ratio of the first temperature change trend to the second temperature change trend, obtaining the adjustment parameter of the preset difference degree, wherein the first seed data point is a seed data point and the second seed data point is an adjacent seed data point of the seed data point; The following formula is used to calculate the conformity of the second growth criterion of the data points to be grown in the neighborhood of the seed data point in the two-dimensional data table: in, For the The first The conformity of the second growth criterion of the data points to be grown, For the The difference degree adjustment parameter of the seed data points is For the acceptable degree of temperature difference, For the The temperature value of the seed data point, For the The first The temperature value of the data point to be grown; The step of obtaining corresponding temperature correction parameters according to the distribution of data points around the target seed point during the region growth process includes: Acquire abnormal data points, where the abnormal data points are data points that cannot be connected to the target seed point in the eight neighborhoods of the target seed point when regional growing is performed according to the first growth criterion compliance; The normalized result of the product of the number of data points that have not grown out in the eight neighborhoods of the abnormal data point and the total number of data points in the eight neighborhoods of the abnormal data point is used as the temperature correction parameter corresponding to the abnormal data point; The step of determining the compliance degree of the third growth criterion according to the temperature correction parameter comprises: Obtaining the abnormality degree of the abnormal data point; According to the temperature correction parameter and the abnormality degree, obtaining the third growth criterion compliance of the abnormal data point; The third growth criterion compliance of abnormal data points is calculated using the following formula: in, For the The g-th abnormal data point in the eight-neighborhood of the target seed point is relative to the g-th abnormal data point. The third growth criterion compliance of the target seed point, For the The adjustment parameter of the difference degree of target seed points, is the preset difference degree, is the temperature correction parameter of the gth abnormal data point in the eight-neighborhood of the i-th target seed point, For the The temperature value of the target seed point, For the The abnormality degree of the abnormal area of ​​the target seed point, For the The temperature value of the gth abnormal data point in the eight-neighborhood of the target seed point, Indicates the absolute value sign; The temperature correction parameter is calculated using the following formula: in, represents the number of data points that have not been grown in the eight-neighborhood of the g-th abnormal data point in the eight-neighborhood of the i-th target seed point, represents the total number of data points in the eight-neighborhood of the g-th abnormal data point in the eight-neighborhood of the i-th target seed point, Represents the normalization function.

2. The method for collecting and managing ambient temperature data based on big data according to claim 1 is characterized in that: The acquisition of the target time window corresponding to the first seed data point includes: Performing straight-line fitting on the temperature data of all data points within a first preset time window centered on the first seed data point to obtain an initial fitting error; Performing straight line fitting on the temperature data of all data points within a second preset time window with the first seed data point as the left boundary to obtain a right-side fitting error; Performing straight line fitting on the temperature data of all data points within a second preset time window with the first seed data point as the right boundary to obtain a left fitting error; If the initial fitting error is greater than the right fitting error, and the initial fitting error is greater than the left fitting error, the first seed data point is used as a segmentation point; The data points are divided based on all segmentation points, and the data segment where the first seed data point is located is used as the target time window corresponding to the first seed data point.

3. The method for collecting and managing ambient temperature data based on big data according to claim 1 is characterized in that: Obtaining the abnormality degree of the abnormal data point includes: Obtaining a third temperature change trend of surrounding data points of the seed data point within a first preset time window, wherein the third temperature change trend is an absolute value of a maximum temperature difference of surrounding data points of the seed data point within the first preset time window; Acquire a fourth temperature change trend of surrounding data points of the abnormal data point within the first preset time window, wherein the fourth temperature change trend is an absolute value of a maximum temperature difference of surrounding data points of the abnormal data point within the first preset time window; determining the similarity between the abnormal data point and the seed data point according to the absolute value of the difference between the third temperature change trend and the fourth temperature change trend; The average of the similarities between all the abnormal data points and the seed data point is used as the abnormality degree of the abnormal data point.

4. An environmental temperature data acquisition and management system based on big data, characterized in that: The system comprises a server, wherein the server comprises: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 3.

Images