A method and system for secure transmission of engineering cost data
By recording the number and time of data access in the engineering cost system, determining the access priority, and establishing a storage copy, the problems of low data transmission efficiency and large load caused by centralized storage are solved, and safe and effective data transmission is achieved.
Patent Information
- Application Number
- CN202411505092.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2025-02-07
- Estimated Expiration
- 2044-10-28
AI Technical Summary
In existing engineering cost systems, centralized storage methods lead to low data transmission efficiency and large access and transmission loads, which easily lead to data loss and system crashes.
By recording the number and time of data accesses of each node of the engineering cost system within a specified period, determining the access priority of each indicator, and establishing a storage copy of the target storage indicator based on the priority, balancing the access load, and ultimately achieving secure transmission.
It improves the effect of data storage, balances the access load, reduces the transmission load, and ensures the secure transmission of engineering cost data.
Smart Images

Figure CN119025302B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of digital information transmission, and in particular to a method and system for securely transmitting engineering cost data. Background Art
[0002] Project cost is based on the relevant data of each project in the project (such as bidding control price, material cost, labor cost, equipment rental fee, etc.) to conduct investment estimation, feasibility study, comprehensive evaluation and other analyses on the project, so as to improve project efficiency and achieve cost control. With the rapid development of digitalization, enterprises have gradually combined project cost with information management to establish a project cost system.
[0003] Existing engineering cost systems usually adopt a centralized storage method, that is, a large amount of data obtained by various departments is stored in the same storage unit. On the one hand, the centralized storage of a large amount of data leads to low data transmission efficiency. On the other hand, as the project progresses, the amount of data generated gradually increases. For example, when the processing equipment needs to process the relevant data, if the amount of data to be processed is too large, the centralized storage method will lead to a large access and transmission load, which will cause the data transmission system to crash and even data loss. Summary of the invention
[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a method and system for secure transmission of engineering cost data. The technical solutions adopted are as follows:
[0005] In a first aspect, an embodiment of the present application provides a method for securely transmitting construction cost data, including:
[0006] Recording the engineering cost data of each node in the engineering cost system, the number of accesses and the access time of each indicator included in each of the engineering cost data within a specified period;
[0007] Determining the access priority of each of the indicators according to the number of accesses and the access time, and determining the target storage indicator according to the access priority;
[0008] According to the target storage index, determining the effective coefficient for each node to establish a storage copy of the target storage index, and according to the effective coefficient, determining a target storage node, and establishing a storage copy of the target storage index at the target storage node;
[0009] The target storage indicator is securely transmitted according to the storage copy.
[0010] In one implementation, determining the access priority of each of the indicators according to the number of accesses and the access time includes:
[0011] According to the number of visits and the visit time, establish the access volume time series curve of each of the indicators, and segment each of the access volume time series curves through an adaptive segmentation algorithm, determine a number of curve segments corresponding to each of the access time series curves, and determine the access fluctuation index of each of the curve segments;
[0012] Determine a first indicator from the indicators, respectively determine the approximate curve segments that are closest in time sequence to the respective curve segments corresponding to the first indicator, and respectively determine the similarity coefficients between the respective curve segments of the first indicator and the corresponding approximate curve segments according to the access fluctuation index;
[0013] Determining, from the approximate curve segments, similar curve segments to each curve segment corresponding to the first indicator according to the similarity coefficient;
[0014] Determine a correlation coefficient of the first indicator according to the number of curve segments, similar curve segments and the similarity coefficient of the first indicator, and determine an access priority of the first indicator according to the correlation coefficient;
[0015] Return to the step of determining the first indicator from the indicators until the access priority of each of the indicators is obtained.
[0016] In one embodiment, determining the access fluctuation index of each curve segment includes:
[0017] Determine a first access count mean value of each of the pageview time series curves according to the access counts of the data points of each of the pageview time series curves;
[0018] Determine the number of data points in each of the curve segments, and determine the variance of the number of accesses and the second mean number of accesses according to the number of accesses of each data point in each of the curve segments;
[0019] respectively determining a first absolute value of a difference between each of the second access times average and the first access times average of the corresponding access time series curve;
[0020] A first product of the variance of the number of visits of each curve segment and the first absolute value is determined respectively, and an access fluctuation index of each curve segment is determined respectively according to a first ratio of the first product to the number of data points of each curve segment.
[0021] In one embodiment, determining, according to the access fluctuation index, the similarity coefficient between each curve segment of the first indicator and the corresponding approximate curve segment includes:
[0022] Determine the similarity of the access fluctuation index according to the access fluctuation index of the curve segment and the access fluctuation index of the corresponding close curve segment, and determine a second ratio of the second average of the number of accesses corresponding to the curve segment to the similarity;
[0023] Determine a third mean of the number of visits according to the number of visits of each data point in each of the approximate curve segments, and respectively determine a third ratio of the mean of the third number of visits of the approximate curve segments corresponding to the curve segments to the maximum mean of the third number of visits;
[0024] According to the second product of the second ratio and the third ratio, a similarity coefficient between each curve segment of the first indicator and the corresponding approximate curve segment is obtained.
[0025] In one implementation, determining the correlation coefficient of the first indicator according to the number of curve segments, similar curve segments, and the similarity coefficient of the first indicator, and determining the access priority of the first indicator according to the correlation coefficient includes:
[0026] respectively determining the number of similar curve segments of the similar curve segments corresponding to each of the curve segments, and determining the average value of similarity coefficients of the similar curve segments corresponding to each of the curve segments;
[0027] respectively determining the third product of the number of similar curve segments and the corresponding similarity coefficient means;
[0028] respectively determining the maximum similarity coefficient of the similar curve segments corresponding to each of the curve segments, and the second absolute value of the difference between the maximum similarity coefficient and the corresponding mean value of the similarity coefficient;
[0029] respectively determining fourth ratios of the third product and the corresponding second absolute value, summing the fourth ratios and the number of the curve segments to obtain a correlation index, and normalizing the correlation index to obtain a correlation coefficient of the first indicator;
[0030] Taking the node where the first indicator is located as the first original storage node, determining the number of neighboring nodes included in the neighborhood of the first original storage node and a fourth access count average of the access counts of all the neighboring nodes;
[0031] Determine the fourth product of the correlation coefficient, the second average of the number of visits, the number of neighborhood nodes and the fourth average of the number of visits, and determine the fifth ratio of the fourth product to the first average of the number of visits to obtain the access priority of the first indicator.
[0032] In one implementation, determining the target storage index according to the access priority includes:
[0033] Normalizing the access priority of each of the indicators to obtain a storage priority index of each of the indicators;
[0034] An index whose storage priority index is greater than an index threshold is determined as a target storage index.
[0035] In one embodiment, determining, according to the target storage indicator, the effective coefficient for each node to establish a storage copy of the target storage indicator, and determining, according to the effective coefficient, the target storage node comprises:
[0036] Determine a second index from the target storage index, use a node where the second index is located as a second original storage node, and determine a candidate node from nodes other than the second original storage node;
[0037] Determine, according to the second original storage node and the candidate node, a validity coefficient for the candidate node to establish a storage copy of the second indicator;
[0038] Returning to the step of determining candidate nodes from nodes other than the second original storage node, until valid coefficients corresponding to all nodes other than the second original storage node are determined;
[0039] Marking the node corresponding to the largest effective coefficient as a candidate storage node, and marking all nodes in the neighborhood of the candidate storage node as invalid nodes for the second indicator, after removing the current candidate storage node, returning to the step of marking the node corresponding to the largest effective coefficient as a candidate storage node, until all nodes are marked, and all marked candidate storage nodes are used as target storage nodes corresponding to the second indicator; wherein the invalid nodes are not used as candidate storage nodes;
[0040] Return to the step of determining the second indicator from the target storage indicators until the target storage node corresponding to each of the target storage indicators is determined.
[0041] In one embodiment, the specified period includes several time periods; and determining, based on the second original storage node and the candidate node, the effective coefficient of the candidate node establishing the storage copy of the second indicator includes:
[0042] Determine the sum of the first access times of all nodes accessing the second index and the sum of the second access times of the candidate nodes accessing the second index in the current period;
[0043] Determining the distance between the second original storage node and the candidate node and the remaining capacity of the candidate node;
[0044] Determine a fifth product of the second total number of accesses, the distance, and the remaining capacity, and determine an effective coefficient for the candidate node to establish a storage copy of the second indicator based on a sixth ratio of the fifth product to the first total number of accesses.
[0045] In one embodiment, the method further comprises:
[0046] Determine a candidate storage replica from each of the storage replicas;
[0047] In a new designated period next to the designated period, determining a new storage priority index of the indicator corresponding to the candidate storage copy in the new designated period, and determining a storage priority index difference between the new storage priority index and the storage priority index of the indicator;
[0048] Determine a target storage node where the candidate storage copy is established, determine a first effective coefficient mean of effective coefficients of all nodes in the neighborhood of the target storage node for establishing the candidate storage copy within a new specified period, and determine a second effective coefficient mean of effective coefficients of all nodes in the neighborhood of the target storage node for establishing the candidate storage copy within the specified period;
[0049] Determine a seventh ratio of the second effective coefficient mean to the first effective coefficient mean, and obtain an index to be updated of the candidate storage copy according to a sixth product of the storage priority index difference and the seventh ratio;
[0050] Returning to the step of determining the candidate storage replica from each of the storage replicas, until the to-be-updated index of the candidate storage replica in each target storage node is determined;
[0051] The to-be-updated index is normalized to obtain a to-be-updated normalized value, and the candidate storage copies whose to-be-updated normalized value is greater than a normalization threshold are updated.
[0052] In a second aspect, an embodiment of the present application provides a system for securely transmitting engineering cost data, including:
[0053] A recording module, used to record the engineering cost data of each node in the engineering cost system, the number of accesses and the access time of each indicator included in each of the engineering cost data within a specified period;
[0054] A determination module, used to determine the access priority of each of the indicators according to the access number and the access time, and determine the target storage indicator according to the access priority;
[0055] An establishment module, used to determine, according to the target storage indicator, an effective coefficient for each of the nodes to establish a storage copy of the target storage indicator, and determine a target storage node according to the effective coefficient, and establish a storage copy of the target storage indicator at the target storage node;
[0056] A transmission module is used to securely transmit the target storage indicator according to the storage copy.
[0057] In one embodiment, the establishing module is further used to:
[0058] Determine a candidate storage replica from each of the storage replicas;
[0059] In a new designated period next to the designated period, determining a new storage priority index of the indicator corresponding to the candidate storage copy in the new designated period, and determining a storage priority index difference between the new storage priority index and the storage priority index of the indicator;
[0060] Determine a target storage node where the candidate storage copy is established, determine a first effective coefficient mean of effective coefficients of all nodes in the neighborhood of the target storage node for establishing the candidate storage copy within a new specified period, and determine a second effective coefficient mean of effective coefficients of all nodes in the neighborhood of the target storage node for establishing the candidate storage copy within the specified period;
[0061] Determine a seventh ratio of the second effective coefficient mean to the first effective coefficient mean, and obtain an index to be updated of the candidate storage copy according to a sixth product of the storage priority index difference and the seventh ratio;
[0062] Returning to the step of determining the candidate storage replica from each of the storage replicas, until the to-be-updated index of the candidate storage replica in each target storage node is determined;
[0063] The to-be-updated index is normalized to obtain a to-be-updated normalized value, and the candidate storage copies whose to-be-updated normalized value is greater than a normalization threshold are updated.
[0064] The present invention has the following beneficial effects:
[0065] By recording the engineering cost data of each node in the engineering cost system, the number of accesses and the access time of each indicator contained in each engineering cost data within a specified period, determining the access priority of each indicator according to the number of accesses and the access time, and determining the target storage indicator according to the access priority, determining the target storage indicator that needs to be stored with priority, which is conducive to improving the effect of subsequent storage; according to the target storage indicator, determining the effectiveness coefficient of each node to establish a storage copy of the target storage indicator, and according to the effectiveness coefficient, determining the target storage node, and establishing a storage copy of the target storage indicator at the target storage node, which is conducive to balancing the access load; based on the storage copy, the target storage indicator is securely transmitted, which is conducive to reducing the transmission load and ensuring the secure transmission of engineering cost data. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0067] Figure 1 A schematic flow chart of the steps of a method for securely transmitting construction cost data provided by an embodiment of the present invention;
[0068] Figure 2 A node topology diagram in a construction cost system provided by an embodiment of the present invention;
[0069] Figure 3 A schematic diagram of a segmented access time series curve provided by an embodiment of the present invention;
[0070] Figure 4 A schematic diagram of marking based on a second original storage node provided by an embodiment of the present invention;
[0071] Figure 5 A structural block diagram of a system for securely transmitting engineering cost data provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0072] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of a method and system for secure transmission of engineering cost data proposed by the present invention, its specific implementation method, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0073] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0074] It should be noted that the term “exemplary” in the embodiments of the present application refers to examples listed for the convenience of explanation, and other embodiments are not limited to the examples listed.
[0075] The specific scheme of a method and system for securely transmitting engineering cost data provided by the present invention is described in detail below in conjunction with the accompanying drawings.
[0076] See also Figure 1 , which shows a flow chart of a method for securely transmitting engineering cost data provided by an embodiment of the present invention. The method for securely transmitting engineering cost data may at least include steps S100-S400:
[0077] S100. Recording the engineering cost data of each node in the engineering cost system, the number of accesses and the access time of each indicator included in each engineering cost data within a specified period.
[0078] S200: Determine the access priority of each indicator according to the number of accesses and the access time, and determine the target storage indicator according to the access priority.
[0079] S300. According to the target storage index, determine the effective coefficient of each node to establish a storage copy of the target storage index, and according to the effective coefficient, determine the target storage node, and establish the storage copy of the target storage index at the target storage node.
[0080] S400: Perform secure transmission of target storage indicators according to the storage replica.
[0081] The technical solution of the embodiment of the present application records the construction cost data of each node in the construction cost system, the number of accesses and the access time of each indicator contained in each construction cost data within a specified period, determines the access priority of each indicator according to the number of accesses and the access time, and determines the target storage indicator according to the access priority, and determines the target storage indicator that needs to be stored with priority, which is conducive to improving the effect of subsequent storage; determines the effective coefficient of each node to establish a storage copy of the target storage indicator according to the target storage indicator, and determines the target storage node according to the effective coefficient, and establishes a storage copy of the target storage indicator at the target storage node, which is conducive to balancing the access load; and performs secure transmission of the target storage indicator based on the storage copy, which is conducive to reducing the transmission load and ensuring the secure transmission of the construction cost data.
[0082] In one embodiment, the length of the designated period is adjusted based on actual conditions, and 24 hours is taken as an example as a designated period; the project cost system has multiple nodes, for example, a department can be used as a node, and each node contains the project cost data stored therein, and the project cost data in each node contains a number of indicators, including but not limited to the quantity of materials, the price of materials, etc., which are not specifically limited. In the embodiment of the present application, the staff of each department accesses the indicators in the node through operations such as adding, deleting, modifying and checking, so the number of visits to each indicator in the node by the staff (that is, the number of times the indicator is visited) and the access time corresponding to each visit can be recorded, thereby determining the number of visits and access time of each indicator contained in each project cost data. Figure 2 As shown in the figure, it is a node topology diagram in the engineering cost system. 1 represents the node and 2 is the connecting line, which indicates the data transmission direction between nodes, that is, each node can access and broadcast to each other.
[0083] In one implementation, step S200 determines the access priority of each indicator according to the number of accesses and the access time, including steps S201-S205:
[0084] S201. According to the number of visits and the visit time, establish the visit volume time series curve of each indicator, and segment each visit volume time series curve through an adaptive segmentation algorithm to determine a number of curve segments corresponding to each visit time series curve and determine the visit fluctuation index of each curve segment.
[0085] Optionally, the specified period is divided into several time periods based on a specified time interval, and the specified time interval is exemplarily 1 minute. Then, based on the number of visits and the visit time, a visit volume timing curve of each indicator is established. The unit of the horizontal axis in the visit volume timing curve is time: 1 minute, and the vertical axis is the visit volume corresponding to each indicator within the time corresponding to the horizontal axis.
[0086] It should be noted that in the process of formulating and transmitting engineering quotations, a large amount of data is involved. As the project progresses, the number of times the data in the engineering cost system is accessed fluctuates. For example, the data corresponding to indicators such as the unit price of labor and materials and the amount of labor and materials consumed during the construction period are accessed more frequently. During the bidding period, the relevant data corresponding to indicators such as project investment estimates are accessed more frequently. However, after the bidding is completed, as the project progresses, the number of times the relevant data is accessed will gradually decrease. Therefore, there are certain differences in the frequency of access to each indicator in different time periods, and the corresponding access volume time series curve also fluctuates accordingly. In order to obtain the fluctuation of different indicators over time, segmentation is required.
[0087] Specifically, each access time series curve is segmented based on an adaptive segmentation algorithm (APCA), thereby determining a number of curve segments corresponding to each access time series curve, such as Figure 3 As shown, there are several curve segments corresponding to any access timing curve.
[0088] In addition, if a curve segment contains fewer data points, then there are fewer similar data points in the period, and the greater the degree of fluctuation, the greater the difference between the number of visits at each moment in the current period and the average number of visits of the indicator, the greater the degree of fluctuation. Based on this principle, the visit fluctuation index of each curve segment is determined, including:
[0089] First, according to the number of visits of the data points of each visit time series curve, the first visit number mean of each visit time series curve is determined. , that is, the sum of the visits of all data points in each visit time series curve is divided by the number of data points to obtain the mean of the first visit times of each visit time series curve .
[0090] Secondly, determine the number of data points for each curve segment (i.e. The number of data points in each curve segment) is used to determine the variance of the number of visits according to the number of visits to each data point in each curve segment. (i.e. The variance of the number of visits corresponding to the curve segment) and the mean of the second number of visits (i.e. The average number of second visits corresponding to each curve segment).
[0091] Furthermore, the first absolute value of the difference between each second access count mean and the first access count mean of the corresponding access time series curve is determined respectively. , which indicates the difference between the number of visits of the curve segment in the current period and the number of visits of the overall time series curve.
[0092] Then, determine the first product of the variance of the number of visits and the first absolute value of each curve segment respectively. , and respectively according to the first product and the number of data points of each curve segment The first ratio of is used to determine the access fluctuation index of each curve segment. The specific formula is:
[0093] ;
[0094] in, For the The access fluctuation index of the curve segment, Different values correspond to different curve segments.
[0095] S202, determining a first indicator from the indicators, respectively determining the approximate curve segments that are closest in time sequence to the respective curve segments corresponding to the first indicator, and respectively determining similarity coefficients between the respective curve segments of the first indicator and the corresponding approximate curve segments according to the access fluctuation index.
[0096] It should be noted that in the engineering cost system, each indicator needs to be shared and transmitted between different departments and personnel, and there is a certain correlation between the indicators, so that the number of accesses also has a certain relationship. For example, when the pre-settlement department determines the budget amount of a project, it needs to call the project's construction cost (estimated amount of materials, labor, machinery, etc.) to control and supervise the project implementation; the actual construction cost during the project will affect the budget amount, and the budget amount range needs to be adjusted according to the actual construction cost. Based on this logic, the correlation between the indicators is analyzed, that is, if a certain indicator is accessed, it causes the other multiple indicators to be accessed, then it is considered that it has a strong correlation with the other indicators. If the number of times it is accessed does not cause the number of times the other indicator data is accessed, or the degree of influence is small, then the correlation of the indicator is small.
[0097] Optionally, any indicator is determined from the indicators as the first indicator, and the approximate curve segments that are closest in time sequence to each curve segment corresponding to the first indicator are determined respectively, that is, from each curve segment of other indicators other than the first indicator (recorded as the remaining curve segments), the center point horizontal coordinate A of the remaining curve segments is determined, and the center point horizontal coordinate A of the remaining curve segments is compared with the center point horizontal coordinate B of each curve segment corresponding to the first indicator, and the center point horizontal coordinate A that is closest to each center point horizontal coordinate B is determined, and the remaining curve segments corresponding to the closest center point horizontal coordinate A are respectively used as the approximate curve segments of each curve segment corresponding to the first indicator. Then, according to the access volatility index, the similarity coefficients of each curve segment of the first indicator and the corresponding approximate curve segment are determined respectively, specifically including:
[0098] First, according to the access fluctuation index of the curve segment (Indicates the first indicator The visit fluctuation index of the curve segment) and the visit fluctuation index of the corresponding close curve segment (Indicates the same as The other curve segments corresponding to The visit volume time series curve of the indicator The access fluctuation index of the curve segment is close to the curve segment, and the similarity of the access fluctuation index is determined. , and determine the mean of the second access times corresponding to the curve segment The second ratio of similarity .
[0099] Secondly, according to the number of visits to each data point in each approximate curve segment, the third mean of the number of visits is determined. , respectively determine the mean of the third visit times of the curve segments close to the curve segments corresponding to the curve segments The third largest visit average The third ratio .
[0100] Then, according to the second product of the second ratio and the third ratio, the similarity coefficient between each curve segment of the first indicator and the corresponding close curve segment is obtained. , the specific formula is:
[0101] ;
[0102] In the formula, Indicates the first indicator corresponding to the current indicator The curve segment and the other The visit volume time series curve of the indicator The similarity coefficient of the approximate curve segment is Indicates The corresponding The fluctuation degree of the approximate curve segment is related to the current The difference in the maximum visits corresponding to the indicators is The larger the The larger the number, the greater the increase in the number of visits to the current indicator (i.e. the first indicator). The greater the increase in the number of visits to an indicator, the greater the increase.
[0103] S203. Determine, from the approximate curve segments, similar curve segments to the curve segments corresponding to the first indicator according to the similarity coefficient.
[0104] Optionally, the similarity coefficient is normalized to (0, 1) to obtain a normalized result of the similarity coefficient, and the close curve segment whose normalized result is greater than a similarity threshold such as 0.5 is taken as a similar curve segment similar to the curve segment corresponding to the first indicator, so that the similar curve segments of each curve segment corresponding to the first indicator can be determined.
[0105] It should be noted that, based on the above analysis, if the access of a certain indicator can lead to the access of multiple other indicators, then the stronger the correlation between it and the other indicators, that is, for a single indicator, the more curve segments in the other access time series curves that are similar to the current access time series curve, and the higher the degree of similarity, the stronger the correlation between the current indicator and the other indicators.
[0106] S204: Determine a correlation coefficient of the first indicator according to the number of curve segments, similar curve segments, and similarity coefficient of the first indicator, and determine an access priority of the first indicator according to the correlation coefficient.
[0107] It should be noted that, in the actual operation of the engineering cost system, there are one or more indicators that are frequently accessed, such as during the bidding period, the frequency of each department accessing the project-related forecast amount is high, which will cause the access load of some nodes to be too large, resulting in a large difference in the access load between them and the rest of the nodes, which is not conducive to the system to transmit data stably and safely. Therefore, in the embodiment of the present application, the index with higher storage priority is screened, and a storage copy of the index is established at the node that accesses the index more times, so that the node where the copy is located and its neighboring nodes directly access the copy when they access the index next time, thereby saving network bandwidth and balancing network load. Based on the above analysis, it can be seen that by analyzing the possibility of a certain indicator being accessed, its storage priority is obtained. The greater the possibility of being accessed, the greater the ability of the storage copy established for it to balance the network load. Correspondingly, its storage priority should be higher, that is, the possibility of being accessed by the index with a strong correlation with the rest of the indicators is higher, and the index with a higher frequency of historical access is more likely to be accessed again in the next period, and if the node with a closer spatial distance to the corresponding node is accessed more times in the current period, then the possibility of each index in the node being accessed in the next period is greater, and its storage priority is higher. Based on this principle, the following processing is performed:
[0108] First, determine the number of similar curve segments corresponding to each curve segment , and determine the mean similarity coefficient of the similarity coefficients of similar curve segments corresponding to each curve segment , for example, the sum of the similarity coefficients of similar curve segments divided by the number of similar curve segments gives the mean similarity coefficient , the larger its value is, the more similar the visits of other indicators are to the current indicator during this period.
[0109] Secondly, determine the third product of the number of similar curve segments and the corresponding similarity coefficient mean .
[0110] Next, determine the maximum similarity coefficient of the similar curve segments corresponding to each curve segment (i.e. the largest ), as well as the maximum similarity coefficient and the corresponding similarity coefficient mean The second absolute value of the difference , which indicates the degree of fluctuation of similar indicators in all similar curve segments.
[0111] Furthermore, a fourth ratio of the third product to the corresponding second absolute value is determined respectively According to the fourth ratio and the number of curve segments Sum and get the correlation index corresponding to the current indicator (i.e. the first indicator) :
[0112] ;
[0113] And, the correlation index Normalize to (0,1) to get the correlation coefficient of the first indicator .
[0114] Then, the node where the first indicator is located is used as the first original storage node, that is, the first original storage node stores the first indicator, and the number of neighboring nodes included in the neighborhood of the first original storage node is determined. , and the fourth mean of the number of visits to all neighboring nodes , that is, the sum of the number of visits to neighboring nodes divided by the number of neighboring nodes. The limited range of the neighborhood is based on actual settings and is not specifically limited.
[0115] Finally, determine the correlation coefficient , the average number of second visits , the number of neighboring nodes And the fourth visit number mean The fourth product of the first visit number and the fourth product are determined The fifth ratio of the first indicator is obtained by the formula:
[0116] ;
[0117] In the formula, Indicates The priority index (i.e., access priority) of the indicator in the next period of time. Assuming The first indicator is Indicates the access priority of the first indicator, Indicates that The correlation coefficient of the first indicator is Indicates the fourth average of the number of visits to all neighboring nodes in the node neighborhood of the current first original storage node, that is, the more nodes are adjacent to the current first original storage node, the more the number of visits to the neighboring node index is, and the greater the possibility of the current first original storage node being visited. Indicates The difference between the number of visits to a metric in the current period and the total number of visits.
[0118] S205. Return to the step of determining the first indicator from the indicators until the access priority of each indicator is obtained.
[0119] In the embodiment of the present application, the step of determining the first indicator from the indicators is returned until the access priority of each indicator is obtained, that is, the access priority of each indicator can be obtained based on The formula principle is used to determine the similarity coefficient between each curve segment of each indicator and the corresponding close curve segment, and based on the formula Formula principle to determine the correlation index corresponding to each indicator And the corresponding correlation coefficient , and finally based on the formula Determine the access priority of each indicator, such as Different values represent different indicators. Can indicate the The priority index (i.e. access priority) of an indicator in the next period of time.
[0120] In one implementation, determining the target storage index according to the access priority in step S200 includes:
[0121] First, prioritize the access to each indicator. Normalize to (0,1) to get the storage priority index of each indicator , that is, The storage priority index of the metric.
[0122] Then, the index whose storage priority index is greater than the index threshold, such as 0.5, is determined as the target storage index, that is, the index for which a storage copy needs to be established.
[0123] It should be noted that since too many storage copies for the same indicator will lead to a waste of storage space, which is not conducive to saving network bandwidth and ensuring data security transmission, too few storage copies will not be able to effectively balance the access load of each node. Therefore, it is necessary to determine the nodes where storage copies need to be created for the target storage indicators that need to be stored based on the preliminary screening obtained in the above steps. In order to avoid establishing multiple storage copies for the same indicator in a close range, taking the data corresponding to an indicator as an example, a storage copy should be established on a node with a high access frequency to the data, a long distance from the original storage node of the data, and a large amount of remaining storage capacity.
[0124] In one embodiment, in step S300, according to the target storage index, determining the effective coefficient of each node to establish a storage copy of the target storage index, and determining the target storage node according to the effective coefficient, includes steps S301-S305:
[0125] S301. Determine a second index from the target storage index, take the node where the second index is located as the second original storage node, and determine a candidate node from nodes other than the second original storage node.
[0126] Optionally, any one of the target storage indicators is determined as the second indicator, assuming that The node where the second indicator is located is used as the second original storage node, the second original storage node stores the second indicator, and any node is determined from all nodes except the second original storage node as a candidate node (denoted as the first nodes).
[0127] S302. Determine, based on the second original storage node and the candidate node, the effective coefficient of the candidate node for establishing a storage copy of the second indicator.
[0128] First, determine the sum of the first access times of all nodes to the second index in the current period And the sum of the second access times of the candidate node to the second index .
[0129] Secondly, determine the distance between the second original storage node and the candidate node and the remaining capacity of the candidate nodes .
[0130] Then, determine the fifth product of the second total number of visits, the distance, and the remaining capacity , according to the sixth ratio of the fifth product to the first access count sum, determine the effective coefficient of the candidate node to establish a storage copy of the second index , the formula is:
[0131] ;
[0132] In the formula, Represents the candidate node ( nodes) for the The index establishes the effective coefficient of the storage copy, assuming that The second indicator is That is, the candidate node ( Nodes) establish the effective coefficient of the storage copy of the second indicator; The larger the The node is relative to the rest of the nodes The number of visits to each indicator is relatively large.
[0133] S303 , returning to the step of determining candidate nodes from nodes other than the second original storage node, until valid coefficients corresponding to all nodes other than the second original storage node are determined.
[0134] Optionally, returning to the step of determining a candidate node from nodes other than the second original storage node, that is, returning to the step of determining a candidate node from nodes other than the second original storage node in step S301, until the candidate node is determined based on the above The formula is used to determine the effective coefficients corresponding to all nodes except the second original storage node, that is, the effective coefficients of all nodes except the second original storage node to establish storage copies of the second indicator.
[0135] S304. Mark the node corresponding to the largest effective coefficient as a candidate storage node, and mark all nodes in the neighborhood of the target storage node as invalid nodes for the second indicator. After removing the current candidate storage node, return to the step of marking the node corresponding to the largest effective coefficient as a candidate storage node until all nodes are marked, and use all marked candidate storage nodes as target storage nodes corresponding to the second indicator; wherein, invalid nodes are not used as candidate storage nodes.
[0136] Optionally, the node corresponding to the largest effective coefficient is marked as a candidate storage node, and all nodes in the neighborhood of the candidate storage node are marked as invalid nodes for the second indicator, and the invalid nodes are not used as candidate storage nodes, that is, in the subsequent processing process, these invalid nodes continue to be maintained as invalid nodes and will not be used as candidate storage nodes. Figure 4 As shown, a part of the nodes is illustrated, 1 is the second original storage node, assuming that the node corresponding to the largest effective coefficient is node 2, node 2 is marked as a candidate storage node, and all nodes in the neighborhood of the candidate storage node, such as nodes 4, 5, and 6, are marked as invalid nodes for the second indicator. Then, after removing the current candidate storage node, that is, excluding node 2, node 2 is not considered, and the step of marking the node corresponding to the largest effective coefficient as a candidate storage node is returned. At this time, it is assumed that the node corresponding to the largest effective coefficient other than node 2 is node 3, and node 3 is marked as a candidate storage node. Then, all nodes in the neighborhood of node 3 are marked as invalid nodes for the second indicator until all nodes are marked. Marking includes being marked as candidate storage nodes or invalid nodes. Therefore, it is assumed that all nodes are marked at this time, and finally all marked candidate storage nodes are used as target storage nodes corresponding to the second indicator, that is, as shown Figure 4 As shown, the target storage nodes include node 2 and node 3.
[0137] S305. Return to the step of determining the second indicator from the target storage indicators until the target storage node corresponding to each target storage indicator is determined.
[0138] Optionally, the step of determining the second indicator from the target storage indicator is returned, that is, returning to step S301, so as to determine the target storage node corresponding to each target storage indicator based on the above principle.
[0139] In the embodiment of the present application, after the target storage node corresponding to each target storage indicator is determined in S300, a storage copy of the corresponding target storage indicator is established in the target storage node.
[0140] In the embodiment of the present application, after determining the storage copy, the corresponding target storage node will broadcast the relevant information of its own storage copy to all nodes, that is, all nodes can obtain the relevant information such as the access parameters of other nodes, so that in S400, the target storage index can be securely transmitted based on the storage copy. Specifically, when a certain node C accesses the target storage index, the target storage index can be securely transmitted based on the storage copy in the node closest to the node C, so that the node C can access the target storage index quickly and with low load.
[0141] In one embodiment, since the access frequency of each indicator in the engineering cost system has the characteristic of periodic changes, the created storage copy should be updated according to the change in the number of accesses of each indicator in each stage to avoid wasting storage space, that is, deleting the corresponding data of the indicator with reduced access frequency, and re-establishing storage copies for the indicators with more accesses. For example, during the bidding and tendering period, the frequency of each department accessing the indicators related to the forecast amount of the project is relatively high. At this time, establishing storage copies for them can effectively balance the access load; after the bidding and tendering are completed, as the project progresses steadily, the number of accesses to the indicators related to the forecast amount will gradually decrease, and the number of accesses to the indicators related to the project implementation is relatively large. At this time, there is no need to retain the storage copies of the indicators related to the forecast amount, and they should be replaced with the storage copies of the indicators related to the project implementation, and the original storage copies of the indicators whose accesses are gradually decreasing should be deleted. Based on this, the method of the embodiment of the present application may also include the following update steps:
[0142] First, determine the candidate storage replica from each storage replica, for example, The node A stored copy of each metric.
[0143] Secondly, in the specified period The next new designated period When the new specified period is determined The new storage priority index of the indicator corresponding to the candidate storage replica (Based on the above The new storage priority index and the storage priority index of the indicator are determined based on the principle of (Based on the above The storage priority index difference is determined by the principle of .
[0144] Next, determine the target storage node where the candidate storage copy is established ( nodes), in the new specified period Within, determine the effective coefficients of all nodes in the neighborhood of the target storage node for establishing candidate storage copies (based on the above The first effective coefficient mean value is determined by the principle of , which is obtained by dividing the sum of these effective coefficients by the number of nodes in the neighborhood of the target storage node; and in the specified period Within, determine the effective coefficients of all nodes in the neighborhood of the target storage node for establishing candidate storage copies (based on the above The second effective coefficient mean value is determined by the principle of .
[0145] Furthermore, a seventh ratio of the second effective coefficient mean to the first effective coefficient mean is determined , and according to the sixth product of the storage priority index difference and the seventh ratio, the to-be-updated index of the candidate storage copy is obtained :
[0146] ;
[0147] In the formula, Indicates The node The index to be updated of the storage copy of the index (i.e. the current candidate storage copy), Characterizes the degree to which the new specified period and the specified period storage priority index decrease, Indicates the number of nodes in the neighborhood of the target storage node. This metric establishes the degree of difference in the effectiveness coefficient of storage replicas.
[0148] Then, the process returns to the step of determining candidate storage copies from each storage copy until the to-be-updated index of the candidate storage copy in each target storage node is determined based on the principle of the above steps.
[0149] Finally, the index to be updated is normalized to [-1,1] to obtain the normalized value to be updated, and the candidate storage replicas whose normalized value to be updated is greater than the normalization threshold such as 0.5 are determined as the storage replicas to be updated, and then the storage replicas to be updated are updated. Therefore, based on different new specified periods, the existing storage replicas can be regularly updated, and the storage replicas of the corresponding indicators can be deleted or newly created, so as to avoid the storage replicas that are not frequently accessed from occupying the memory, and after deleting the storage replicas that are not frequently accessed, the deletion will also be broadcasted, and the deletion log of the node will be recorded, so as to achieve the purpose of effectively balancing the access load, saving network bandwidth, improving data transmission efficiency, and ensuring the secure transmission of data.
[0150] Through the method of the embodiment of the present application, based on the difference in the number of accesses to the indicators of each node in the engineering cost system, the access priority (storage priority) of each indicator is analyzed in combination with the correlation between the nodes, and the suitable nodes are selected in combination with the spatial distance between the nodes and the difference in the remaining storage capacity of each node to establish storage copies of the corresponding indicators, and the storage copies are updated regularly, thereby optimizing the situation in the related technology where centralized storage of data leads to excessive data transmission in a certain period of time resulting in data loss, shortening the data transmission distance, reducing the transmission network load, improving the transmission efficiency, and reducing the transmission cost, thereby ensuring the safe transmission of engineering cost data.
[0151] Reference Figure 5 , shows a structural block diagram of a secure transmission system for construction cost data according to an embodiment of the present application, the system may include:
[0152] The recording module is used to record the engineering cost data of each node in the engineering cost system, the number of accesses and the access time of each indicator contained in each engineering cost data within a specified period;
[0153] A determination module, used to determine the access priority of each indicator according to the number of accesses and the access time, and determine the target storage indicator according to the access priority;
[0154] An establishment module is used to determine the effective coefficient of each node to establish a storage copy of the target storage indicator according to the target storage indicator, and determine the target storage node according to the effective coefficient, and establish the storage copy of the target storage indicator at the target storage node;
[0155] The transmission module is used to securely transmit the target storage indicators based on the storage replicas.
[0156] In one embodiment, the establishment module is further configured to:
[0157] Determine a candidate storage replica from each storage replica;
[0158] In a new designated period next to the designated period, determining a new storage priority index of the indicator corresponding to the candidate storage replica in the new designated period, and determining a storage priority index difference between the new storage priority index and the storage priority index of the indicator;
[0159] Determine a target storage node where a candidate storage copy is established, determine a first effective coefficient mean of effective coefficients of all nodes in the neighborhood of the target storage node for establishing the candidate storage copy within a new specified period, and determine a second effective coefficient mean of effective coefficients of all nodes in the neighborhood of the target storage node for establishing the candidate storage copy within the specified period;
[0160] Determine a seventh ratio of the second effective coefficient mean to the first effective coefficient mean, and obtain an index to be updated of the candidate storage copy according to a sixth product of the storage priority index difference and the seventh ratio;
[0161] Return to the step of determining the candidate storage replica from each storage replica until the to-be-updated index of the candidate storage replica in each target storage node is determined;
[0162] The to-be-updated index is normalized to obtain a to-be-updated normalized value, and the candidate storage replicas whose to-be-updated normalized value is greater than a normalization threshold are updated.
[0163] In the embodiment of the present application, the functions of each module in the system can be found in the corresponding description in the above method and will not be repeated here.
[0164] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0165] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
Claims
1. A method for secure transmission of engineering cost data, characterized in that: The method comprises: Recording the engineering cost data of each node in the engineering cost system, the number of accesses and the access time of each indicator included in each of the engineering cost data within a specified period; Determining the access priority of each of the indicators according to the number of accesses and the access time, and determining the target storage indicator according to the access priority; According to the target storage index, determining the effective coefficient for each node to establish a storage copy of the target storage index, and according to the effective coefficient, determining a target storage node, and establishing a storage copy of the target storage index at the target storage node; According to the storage copy, securely transmit the target storage indicator; Among them, according to the target storage indicator, determining the effective coefficient of each node to establish a storage copy of the target storage indicator, and determining the target storage node according to the effective coefficient, including: determining a second indicator from the target storage indicator, taking the node where the second indicator is located as the second original storage node, and determining a candidate node from nodes other than the second original storage node; determining the effective coefficient of the candidate node to establish a storage copy of the second indicator according to the second original storage node and the candidate node; returning to the step of determining the candidate node from nodes other than the second original storage node until the effective coefficients corresponding to all nodes other than the second original storage node are determined; marking the node corresponding to the largest effective coefficient as a candidate storage node, and marking all nodes in the neighborhood of the candidate storage node as invalid nodes for the second indicator, removing the current candidate storage node, and returning to the step of marking the node corresponding to the largest effective coefficient as a candidate storage node until all nodes are marked, and taking all marked candidate storage nodes as the target storage nodes corresponding to the second indicator; wherein the invalid nodes are not used as candidate storage nodes; returning to the step of determining the second indicator from the target storage indicator until the target storage node corresponding to each target storage indicator is determined.
2. The method for secure transmission of construction cost data according to claim 1, characterized in that: Determining the access priority of each of the indicators according to the number of accesses and the access time includes: According to the number of visits and the visit time, establish the visit volume time series curve of each of the indicators, and segment each of the visit volume time series curves through an adaptive segmentation algorithm, determine a number of curve segments corresponding to each of the visit volume time series curves, and determine the visit fluctuation index of each of the curve segments; Determine a first indicator from the indicators, respectively determine the approximate curve segments that are closest in time sequence to the respective curve segments corresponding to the first indicator, and respectively determine the similarity coefficients between the respective curve segments of the first indicator and the corresponding approximate curve segments according to the access fluctuation index; Determining, from the approximate curve segments, similar curve segments of each curve segment corresponding to the first indicator according to the similarity coefficient; Determine a correlation coefficient of the first indicator according to the number of curve segments, similar curve segments and the similarity coefficient of the first indicator, and determine an access priority of the first indicator according to the correlation coefficient; Return to the step of determining the first indicator from the indicators until the access priority of each of the indicators is obtained.
3. The method for secure transmission of construction cost data according to claim 2 is characterized in that: Determining the access fluctuation index of each curve segment comprises: Determine a first access count mean value of each of the pageview time series curves according to the access counts of the data points of each of the pageview time series curves; Determine the number of data points in each of the curve segments, and determine the variance of the number of accesses of each curve segment and the second mean number of accesses of each curve segment according to the number of accesses of each data point in each of the curve segments; respectively determining a first absolute value of a difference between each of the second access times average and the first access times average of the corresponding access time series curve; A first product of the variance of the number of visits of each curve segment and the first absolute value is determined respectively, and an access fluctuation index of each curve segment is determined respectively according to a first ratio of the first product to the number of data points of each curve segment.
4. The method for secure transmission of construction cost data according to claim 3 is characterized by: Determining the similarity coefficients of each curve segment of the first indicator and the corresponding approximate curve segment according to the access fluctuation index includes: Determine the similarity of the access fluctuation index according to the access fluctuation index of the curve segment and the access fluctuation index of the corresponding close curve segment, and determine a second ratio of the second average of the number of accesses corresponding to the curve segment to the similarity; Determine a third mean of the number of visits according to the number of visits of each data point in each of the approximate curve segments, and respectively determine a third ratio of the mean of the third number of visits of the approximate curve segments corresponding to the curve segments to the maximum mean of the third number of visits; According to the second product of the second ratio and the third ratio, a similarity coefficient between each curve segment of the first indicator and the corresponding approximate curve segment is obtained.
5. The method for secure transmission of construction cost data according to claim 3 is characterized by: Determining the correlation coefficient of the first indicator according to the number of curve segments, similar curve segments, and the similarity coefficient of the first indicator, and determining the access priority of the first indicator according to the correlation coefficient includes: respectively determining the number of similar curve segments of the similar curve segments corresponding to each of the curve segments, and determining the average value of similarity coefficients of the similar curve segments corresponding to each of the curve segments; respectively determining the third product of the number of similar curve segments and the corresponding similarity coefficient means; respectively determining the maximum similarity coefficient of the similar curve segments corresponding to each of the curve segments, and the second absolute value of the difference between the maximum similarity coefficient and the corresponding mean value of the similarity coefficient; respectively determining fourth ratios of the third product and the corresponding second absolute value, summing the fourth ratios and the number of the curve segments to obtain a correlation index, and normalizing the correlation index to obtain a correlation coefficient of the first indicator; Taking the node where the first indicator is located as the first original storage node, determining the number of neighboring nodes included in the neighborhood of the first original storage node and a fourth access count average of the access counts of all the neighboring nodes; Determine the fourth product of the correlation coefficient, the second average of the number of visits, the number of neighborhood nodes and the fourth average of the number of visits, and determine the fifth ratio of the fourth product to the first average of the number of visits to obtain the access priority of the first indicator.
6. The method for secure transmission of construction cost data according to any one of claims 1 to 5, characterized in that: Determining the target storage index according to the access priority includes: Normalizing the access priority of each of the indicators to obtain a storage priority index of each of the indicators; An index whose storage priority index is greater than an index threshold is determined as a target storage index.
7. The method for secure transmission of construction cost data according to claim 6 is characterized by: The specified period includes a plurality of time periods; the determining, based on the second original storage node and the candidate node, the effective coefficient of the candidate node establishing the storage copy of the second indicator includes: Determine the sum of the fifth number of visits made by all nodes to the second index and the sum of the sixth number of visits made by the candidate nodes to the second index in the current period; Determining the distance between the second original storage node and the candidate node and the remaining capacity of the candidate node; Determine a fifth product of the sixth total of access times, the distance, and the remaining capacity, and determine an effective coefficient for the candidate node to establish a storage copy of the second indicator based on a sixth ratio of the fifth product to the fifth total of access times.
8. The method for secure transmission of construction cost data according to claim 7 is characterized by: The method further comprises: Determine a candidate storage replica from each of the storage replicas; In a new designated period next to the designated period, determining a new storage priority index of the indicator corresponding to the candidate storage copy in the new designated period, and determining a storage priority index difference between the new storage priority index and the storage priority index of the indicator; Determine a target storage node where the candidate storage copy is established, determine a first effective coefficient mean of effective coefficients of all nodes in the neighborhood of the target storage node for establishing the candidate storage copy within a new specified period, and determine a second effective coefficient mean of effective coefficients of all nodes in the neighborhood of the target storage node for establishing the candidate storage copy within the specified period; Determine a seventh ratio of the second effective coefficient mean to the first effective coefficient mean, and obtain an index to be updated of the candidate storage copy according to a sixth product of the storage priority index difference and the seventh ratio; Returning to the step of determining the candidate storage replica from each of the storage replicas, until the to-be-updated index of the candidate storage replica in each target storage node is determined; The to-be-updated index is normalized to obtain a to-be-updated normalized value, and the candidate storage copies whose to-be-updated normalized value is greater than a normalization threshold are updated.
9. A secure transmission system for engineering cost data, characterized in that: include: A recording module, used to record the engineering cost data of each node in the engineering cost system, the number of accesses and the access time of each indicator included in each of the engineering cost data within a specified period; A determination module, used to determine the access priority of each of the indicators according to the access number and the access time, and determine the target storage indicator according to the access priority; An establishment module, used to determine, according to the target storage indicator, an effective coefficient for each of the nodes to establish a storage copy of the target storage indicator, and determine a target storage node according to the effective coefficient, and establish a storage copy of the target storage indicator at the target storage node; Wherein, according to the target storage index, determining the effective coefficient of each node to establish a storage copy of the target storage index, and determining the target storage node according to the effective coefficient, including: determining a second index from the target storage index, taking the node where the second index is located as the second original storage node, and determining a candidate node from nodes other than the second original storage node; determining the effective coefficient of the candidate node to establish a storage copy of the second index according to the second original storage node and the candidate node; returning to the step of determining the candidate node from nodes other than the second original storage node until the effective coefficients corresponding to all nodes other than the second original storage node are determined; marking the node corresponding to the largest effective coefficient as a candidate storage node, and marking all nodes in the neighborhood of the candidate storage node as invalid nodes for the second index, removing the current candidate storage node, and returning to the step of marking the node corresponding to the largest effective coefficient as a candidate storage node until all nodes are marked, and taking all marked candidate storage nodes as the target storage nodes corresponding to the second index; wherein the invalid nodes are not used as candidate storage nodes; returning to the step of determining the second index from the target storage index until the target storage node corresponding to each target storage index is determined; A transmission module is used to securely transmit the target storage indicator according to the storage copy.
Citation Information
Patent Citations
Dynamic save file access method based on use page view and prediction mechanism
CN106648456A
Dynamic file placement method based on combined prediction in cloud storage system
CN111258980A