Construction engineering cost data inspection method and system based on big data, and medium

Abstract

The invention discloses a construction engineering cost data inspection method and system based on big data and a medium, and the method comprises the steps: converting an obtained construction engineering cost table into a standardized table, and splitting the standardized table into column table data and row table data; null value checking and numerical value attribute checking are carried out on each column of table data, and a first checking result is generated; for each row of table data, determining a list header corresponding to each content cell in the table data, and performing numerical value anomaly detection on the content cells according to a historical numerical value interval corresponding to the list header to generate a second detection result; and generating a final inspection result from each first inspection result and each second inspection result, and outputting the final inspection result. According to the method, various types of cost tables are converted into standardized tables and then split, null value checking and numerical value attribute checking are carried out, meanwhile, numerical value anomaly checking is carried out based on a large amount of historical cost data, and efficient and accurate checking of the integrity, the numerical value attribute and the numerical value anomaly of the cost data is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of big data processing, and in particular to a method, system and medium for verifying construction engineering cost data based on big data. Background Art

[0002] There are many types of construction projects, including the construction of various public infrastructure such as roads, bridges, stations, and rails, as well as the construction of various industrial buildings and civil buildings. Regardless of the type of construction project, it must go through several stages such as design, construction, and acceptance, and is a complex system project. In addition, in order to better control the overall cost of the construction project, the cost will be calculated in each of the above stages of the construction project. Among them, the cost of the design stage is mainly a preliminary budget for the costs of building materials, labor costs, machinery and equipment, engineering design, etc.; the cost of the construction stage is based on the specific construction plan and construction plan. The relatively accurate assessment and tracking control of labor costs, machinery and equipment rental, material procurement and other costs; the cost of the acceptance stage is based on the actual project volume and costs for audit and accounting.

[0003] It can be seen that the entire construction project involves various types of cost documents compiled by different personnel. Each cost document is usually compiled in the form of a table, involving cost data in multiple cells, and anomalies are prone to occur during the compilation process. For example, a cell lacks a value, or the cell data type is wrong, or the value size of the cell is abnormal. In order to avoid such anomalies, it is usually necessary to check the cost data in each cell after the cost document is compiled. The inspection is usually carried out by the compiler and the relevant auditors. Two-level manual verification, but because there are many cost data involved in the cost documents, the inspection efficiency is low and it is easy to miss, resulting in low inspection accuracy, especially the anomalies of the value size are difficult to accurately inspect. Therefore, how to accurately and efficiently inspect the cost data in various cost documents of construction projects is one of the technical problems that need to be solved at present. Summary of the invention

[0004] The main purpose of the present invention is to provide a construction project cost data verification method, system and storage medium based on big data, aiming to solve the technical problem of how to accurately and efficiently verify the cost data in various cost documents of construction projects.

[0005] To achieve the above object, the present invention provides a construction engineering cost data verification method based on big data, and the construction engineering cost data verification based on big data includes:

[0006] When a cost table of a construction project is obtained, the cost table is converted into a standardized table, and the standardized table is split into columns and rows to generate a plurality of column table data and a plurality of row table data;

[0007] Performing a null value check and a numerical attribute check on each of the table data in the columns, and generating a first check result corresponding to each of the table data in the columns;

[0008] For each row of table data, determine a column header corresponding to each content cell in the row of table data, and perform a value abnormality check on the content cell according to a historical value interval corresponding to the column header to generate a second test result corresponding to the row of table data;

[0009] Each of the first inspection results and each of the second inspection results is generated as the final inspection result output of the cost table, thereby completing the inspection of each cost data in the cost table.

[0010] Preferably, the step of performing a value abnormality check on the content cell according to the historical value interval corresponding to the column header to generate a second test result corresponding to the row table data includes:

[0011] Determine whether there is a target column header having a logical relationship with the column header in the standardized table, and if the target column header exists, search for target data corresponding to the target column header in the row table data;

[0012] Performing a logic check on the content cell according to the logical relationship and the target data, and determining whether the logic check passes;

[0013] If the logic check passes, the value of the content cell is checked for abnormality according to the historical value interval, a value check result corresponding to the content cell is generated, and the value check result corresponding to each content cell is formed as the second check result.

[0014] Preferably, the step of performing a null value check and a numerical attribute check on each of the table data in the columns, and generating a first check result corresponding to each of the table data in the columns comprises:

[0015] For each of the table data, determine whether there is a null value column cell in each column cell included in the table data, and generate a null value test result;

[0016] Determine the column attribute corresponding to the column table data, and judge whether there is a target value that does not match the column attribute in the value of each column cell, and generate an attribute inspection result;

[0017] The null value check result and the attribute check result are jointly generated as the first check result.

[0018] Preferably, the step of converting the cost table into a standardized table, splitting the standardized table into columns and rows, and generating a plurality of column table data and a plurality of row table data comprises:

[0019] Identify the merged cells in the cost table, and split the merged cells to obtain split cells;

[0020] Filling the values ​​in the merged cells into each of the split cells to form the cost table into a standardized table;

[0021] Splitting the standardized table into a row header part and a row content part, and performing row splitting on the row content part to generate a plurality of row table data;

[0022] The standardized table is split into a column header part and a column content part, and the column content part is split into columns to generate a plurality of column table data.

[0023] Preferably, the step of splitting the standardized table into a row header part and a row content part comprises:

[0024] For each cell in the standardized table, based on a preset word segmentation model, the text in the cell is segmented to obtain cell keywords;

[0025] Generate a word meaning weight matrix corresponding to each of the cell keywords, and modify the keyword vector corresponding to each of the cell keywords according to the word meaning weight matrix, perform mean processing on each of the modified keyword vectors, and obtain a text vector corresponding to the cell;

[0026] Generate the text vectors corresponding to each of the cells into a vector matrix corresponding to the standardized table, and determine the row vector similarity values ​​between adjacent vector rows in the vector matrix;

[0027] Compare the row vector similarity values ​​to determine the target row vector similarity value with the smallest value among the row vector similarity values;

[0028] A target row in the standardized table corresponding to the target row vector similarity value is determined, and the standardized table is split into a row header part and a row content part according to the target row.

[0029] Preferably, before the step of segmenting the text in the cell based on a preset segmentation model to obtain cell keywords, the step includes:

[0030] Acquire a corpus data set, and iteratively train the preset word segmentation model based on the corpus data set, and update each model parameter according to the learning rate of each model parameter in the preset word segmentation model during the iterative training process:

[0031] When the number of iterative training reaches the preset number, the objective function of the preset word segmentation model is calculated to obtain a calculation result, and the calculation formula is:

[0032]

[0033] Among them, LM represents the calculation result, Q represents the number of corpora in the corpus dataset, and m p Indicates the number of words that the preset word segmentation model divides the pth corpus in the corpus dataset, m p0 represents the number of reference words corresponding to the p-th corpus, S p S represents the word vector of the p-th corpus divided by the preset word segmentation model, p0 represents the reference word vector corresponding to the p-th corpus, γ represents the restriction parameter, b represents the bias value of the preset word segmentation model, and w represents the weight value of the preset word segmentation model;

[0034] According to the calculation result, judging whether the preset word segmentation model has converged, and if the preset word segmentation model has converged, completing the iterative training of the preset word segmentation model;

[0035] If the preset word segmentation model has not converged, a step of iteratively training the preset word segmentation model based on the corpus data set is executed.

[0036] Preferably, the step of performing a value anomaly check on the content cell according to the historical value interval corresponding to the column header includes:

[0037] Reading a plurality of historical data corresponding to the column header, and selecting any target historical data from each of the historical data, and calculating distance parameters between the target historical data and each of the historical data;

[0038] Arrange the distance parameters in ascending order, and generate the difference between any two adjacent distance parameters according to the arrangement order;

[0039] Determine a clustering radius according to each of the differences, and cluster each of the historical data according to the clustering radius to form a historical data group;

[0040] Sorting each group of data in the historical data group class in ascending order to generate a historical sequence;

[0041] Quartile values ​​in the historical sequence are determined, and the historical value interval is determined based on the quartile values.

[0042] Preferably, the step of clustering the historical data according to the clustering radius to form historical data groups includes:

[0043] Obtaining a preset clustering density, and clustering the historical data according to the clustering radius and the preset clustering density to form an initial data group;

[0044] The density index and proximity index of the initial data group class are calculated according to a preset evaluation formula, and the preset evaluation formula is:

[0045]

[0046] Among them, ρ represents the density index, C represents the number of initial data group classes, represents the variance of the i-th initial data group, N represents the number of data contained in the historical data, x k represents the kth data in the historical data, μ represents the proximity index, exp represents the natural exponential function, Xci represents the center of the i-th initial data group class, Xcj represents the center of the j-th initial data group class, represents the Gaussian constant.

[0047] Obtaining density weights and proximity weights corresponding to the density index and proximity index, respectively, and respectively modifying the density index and proximity index according to the density weight and the proximity weight, to obtain an evaluation index corresponding to the initial data group class;

[0048] Compare the evaluation index with a preset index threshold to determine whether the evaluation index is greater than the preset index threshold, and if so, generate the initial data group class as a historical data group class;

[0049] If the evaluation index is less than or equal to the preset index threshold, the preset clustering density is updated, and based on the updated preset clustering density, the step of clustering the historical data according to the clustering radius and the preset clustering density is performed until the evaluation index is less than the preset index threshold.

[0050] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a construction engineering cost data verification system based on big data, the construction engineering cost data verification system based on big data includes a storage, a processor, a communication bus and a control program stored on the storage:

[0051] The communication bus is used to realize the connection and communication between the processor and the storage;

[0052] The processor is used to execute the control program to implement the steps of the construction project cost data verification method based on big data as described above.

[0053] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a medium, which is a readable storage medium, on which a control program is stored, and when the control program is executed by a processor, the steps of the construction project cost data verification method based on big data as described above are implemented.

[0054] The invention discloses a method, system and medium for checking construction project cost data based on big data. When obtaining the cost table of the construction project, it is first converted into a standardized table, and then the converted standardized table is split into columns and rows to generate multiple column table data and multiple row table data. The column table data is data obtained by splitting according to the content column of the standardized table, and the column table data is subjected to null value test and numerical attribute test to ensure that each cell of the cost table has data and the numerical attribute of the data is correct. The row table data is data obtained by splitting according to the content row of the standardized table, and the column header corresponding to each content cell is determined, and then the numerical abnormality test is performed on it according to the historical numerical interval corresponding to the column header. The historical data interval is generated by a large amount of historical cost data on the column header of multiple other construction projects with similar characteristics to the current construction project. As the cost interval of the construction project reference represented by the column header, the abnormality of the cost data in the cell can be tested by the size relationship between the cost data in the cell and the cost interval. Then, the first test result obtained by the null value test and the numerical attribute test and the second test result obtained by the numerical abnormality test are jointly generated as the final test result output of the cost table. In this way, by converting various types of cost tables into standardized tables for column splitting and row splitting, the column table data and row table data are obtained for inspection in the form of columns and rows, which improves the inspection efficiency and avoids omissions. In addition, numerical anomaly inspection is performed through historical numerical ranges generated by a large amount of historical cost data, which ensures the accuracy of numerical size inspection and realizes efficient and accurate inspection of the integrity, numerical attributes and numerical anomalies of each cost data in the cost table. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of the first embodiment of the construction engineering cost data verification method based on big data of the present invention;

[0056] Figure 2 It is a flow chart of the second embodiment of the construction engineering cost data verification method based on big data of the present invention;

[0057] Figure 3It is a flow chart of the third embodiment of the construction engineering cost data verification method based on big data of the present invention;

[0058] Figure 4 It is a structural schematic diagram of the hardware operating environment involved in an embodiment of the construction project cost data verification system based on big data of the present invention.

[0059] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0061] The present invention provides a construction engineering cost data verification method based on big data, please refer to Figure 1 , Figure 1 It is a flow chart of the first embodiment of the method for verifying construction project cost data based on big data of the present invention.

[0062] The embodiment of the present invention provides an embodiment of a construction project cost data verification method based on big data. It should be noted that although the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than here. Specifically, the construction project cost data verification method based on big data in this embodiment includes:

[0063] Step S10, when a cost table of a construction project is obtained, the cost table is converted into a standardized table, and the standardized table is split into columns and rows to generate a plurality of column table data and a plurality of row table data.

[0064] The method for verifying construction project cost data based on big data in this embodiment is applied to the backend server, and the cost data in each cost table of the construction project is verified by the backend server to ensure the integrity, correctness of numerical attributes and numerical values ​​of each cost data in the cost table. Among them, the verification of cost data can be the verification of cost data in a single cost document, or it can be the unified verification of all cost documents of the construction project, and there is no restriction on this. Each cost table can be stored in a database that is connected to the backend server in communication. When it is necessary to verify the cost data of the construction project, the backend server first obtains the corresponding cost table from the database.

[0065] Furthermore, the cells in the cost table may be varied, such as cells merged in the form of rows, cells merged in the form of columns, multi-level headers, etc. In order to uniformly process the cost table, the background server converts the cost table after obtaining it and converts it into a standardized table. A standardized table is a table in which cells exist in a single form, that is, there are no merged cells. Then, the content part in the standardized table is split into columns and rows in the form of columns and rows. The data with the same column identifier in the content part of the standardized table is represented as data in the same column, and such data is identified and extracted from the standardized table to form column table data, and the data with the same row identifier in the content part of the standardized table is represented as data in the same row, and such data is identified and extracted from the standardized table to form row table data. After each column and each row in the standardized table are identified and extracted, multiple column table data and multiple row table data are obtained, and each column table data and each row table data each contain all the data in the standardized table.

[0066] Step S20 , performing a null value check and a numerical attribute check on each of the column table data, and generating a first check result corresponding to each of the column table data.

[0067] Furthermore, each column of tabular data contains a plurality of cells with the same column identifier and sequentially arranged row identifiers, such as 10 cells corresponding to the 2nd column and the 1st to 10th rows. For each column of tabular data of this type, a null value check and a numerical attribute check are performed, and a first test result is generated through the test to indicate whether each cell contained in each column of tabular data has data and whether the numerical attribute of each data is correct. Specifically, the steps of performing a null value check and a numerical attribute check on each of the columns of tabular data and generating a first test result corresponding to each of the columns of tabular data include:

[0068] Step S21, for each column of table data, determining whether there is a null value column cell in each column of cells contained in the column of table data, and generating a null value test result;

[0069] Step S22, determining the column attribute corresponding to the column table data, and judging whether there is a target value that does not match the column attribute in the value of each column cell, and generating an attribute check result;

[0070] Step S23: Generate the null value test result and the attribute test result together as the first test result.

[0071] Furthermore, each cell constituting the list table data is used as a column cell included in the list table data. For each list table data, it is detected whether there is a null value column cell in each column cell included therein, and a corresponding null value test result is generated. The null value column cell is a cell whose value is a null value, that is, it is determined whether there is a null value column cell by detecting whether the value in each column cell is a null value. If there is a column cell whose value is a null value, it is determined that there is a null value column cell, and the row identifier and column identifier of the null value column cell are generated together as a null value test result of the existence of the null value column cell. If there is no column cell whose value is a null value, it is determined that there is no null value column cell, and a null value test result of the non-existence of the null value column cell is generated.

[0072] Furthermore, the data meanings corresponding to different table data are different, for example, table data representing the unit price of various materials used in a certain engineering section of a construction project, or table data representing the quantity of various materials used, or table data representing the unit of the quantity of various materials used. The data meaning corresponding to the table data is used as the column attribute corresponding to the table data. This type of column attribute can be determined by the header of the cost table. The header cell with the column identifier in the header of the cost table is searched through the column identifier corresponding to the table data, and then the column attribute corresponding to the table data is determined by the header cell. The data meanings represented by different column attributes are different and have different numerical characteristics. For example, for the column attribute representing the meaning of the amount and the column attribute representing the meaning of the unit, the corresponding numerical characteristics between the two are that the former is a number and the latter is non-number. Therefore, after determining the column attribute corresponding to the table data, the numerical characteristics corresponding to the column attribute can be identified, and then it can be determined whether the values ​​in each column cell contained in the table data have the numerical characteristics, and thereby it can be determined whether there is a target value that does not match the column attribute in the values ​​of each column cell. If the value of a column cell does not have the numerical characteristic corresponding to the column attribute, for example, the numerical characteristic is non-numeric, and the value of a column cell is numeric, then it is determined that there is a target value that does not match the column attribute in the values ​​of the cells in each column, and the row identifier and column identifier of the column cell where the target value is located are generated as the attribute test result of the mismatch of the numerical attribute. On the contrary, if the values ​​of the cells in each column have the numerical characteristic corresponding to the column attribute, it is determined that there is no target value in the values ​​of the cells in each column, and the attribute test result of the matching of the numerical attribute is generated.

[0073] Furthermore, after the null value check and the numerical attribute check of each column of table data are completed and the corresponding null value check results and attribute check results are generated, the null value check results and attribute check results of each column of table data are generated as the first check result of the table data of the column. The first check result formed by the null value check result and the attribute check result includes four combinations, which respectively represent the normal column cells without null value column cells and without target numerical values ​​in the table data of the column, and the abnormal column cells with null value column cells and without target numerical values, the null value column cells and with target numerical values, and the null value column cells and with target numerical values. In addition, for the abnormal column cells, their row identifiers and column identifiers are formed as null value check results and / or attribute check results, so that the first check result also carries the row identifier and the column identifier, thereby facilitating the rapid determination of the abnormal cells in the cost table.

[0074] Step S30, for each row of table data, determine the column header corresponding to each content cell in the row of table data, and perform a value anomaly check on the content cell according to the historical value interval corresponding to the column header to generate a second test result corresponding to the row of table data.

[0075] Furthermore, the standardized table before the row table data is split into rows usually contains header data and content data. Each cell in the header data is a content that represents the specific meaning of the data, such as unit, quantity, unit price, etc. Each cell in the content data is a content that represents the specific value of the data, such as quantity "50", "100", "200", unit price "10", "20", "30", etc. The set of row table data forms the overall content data, and the inspection of the row table data is a numerical abnormality inspection of each content cell. Specifically, for each row of table data, determine the column header corresponding to each content cell. The column header is the header cell in the column where the content cell is located, indicating the specific meaning of the data in the content cell. For example, the value of the content cell is p, and after searching the header cell in the column where it is located is the unit price of concrete, it means that the value of the content cell is the unit price of concrete.

[0076] Further, other completed construction projects with similar characteristics to the construction project are searched. The first similarity is that they are of the same type, for example, they are all construction projects for rail transit. Secondly, they are similar in terms of project scale, construction time, geological factors, material prices, labor prices, equipment rental prices and other factors. This type of similarity indicates that the construction project is similar to other construction projects in terms of material usage, material prices, labor prices, and equipment rental prices, so that the material usage, material prices, labor prices and other data of other construction projects can be formed into corresponding historical numerical intervals, which are used as a reference for evaluating whether the data in the content cell is reasonable. Specifically, after determining the column header corresponding to the content cell, the historical numerical interval corresponding to the column header is determined, and the corresponding historical numerical interval is the reasonable numerical change interval formed based on a large amount of data on the column headers of other construction projects with similar characteristics. Based on the historical numerical interval, the content cell is subjected to a numerical abnormality test to determine whether the value in the content cell is within the historical numerical interval, and a numerical test result is generated to determine whether the value in the content cell is abnormal or not, and then the numerical test results of each content cell are used to jointly generate the second test result corresponding to the row table data. Specifically, the step of performing a value abnormality check on the content cell according to the historical value interval corresponding to the column header to generate a second test result corresponding to the row table data includes:

[0077] Step S31, determining whether there is a target column header having a logical relationship with the column header in the standardized table, and if the target column header exists, searching for target data corresponding to the target column header in the row table data;

[0078] Step S32, performing a logic check on the content cell according to the logic relationship and the target data, and determining whether the logic check passes;

[0079] Step S33, if the logic check passes, then perform an abnormality check on the value of the content cell according to the historical value interval, generate a value check result corresponding to the content cell, and form the value check result corresponding to each content cell as the second check result.

[0080] It is understandable that, in addition to the content data directly corresponding to the column header, the column table data can also be content data with a logical relationship with other column table data. For example, the table header cell corresponding to the column table data A is the quantity of material w, and the content data of the column table data A is the specific quantity of material w; the table header cell corresponding to the column table data C is the total price of material w, and the column table data C is obtained by multiplying the column table data B representing the unit price of material w and the column table data A representing the amount of material w, then the content data in the column table data C has a logical relationship with the column table data A and B. Based on this, in order to more accurately check the data values ​​in the content cells, this embodiment distinguishes and checks the values ​​in the content cells according to whether they have a logical relationship with other content cells.

[0081] Specifically, for the column header corresponding to the content cell, first determine whether there is a target column header that has a logical relationship with the column header in the standardized table. If there is a target column header, it means that the value of the content cell is generated based on the logical relationship with other content cells. If there is no target column header, it means that the value in the content cell is a single data that has no logical relationship with other content cells. For the single data without a target column header, directly compare the value in the content cell with the historical numerical interval to determine whether the value in the content cell is within the numerical range of the historical numerical interval. If it is within its numerical range, it is determined that the value of the content cell belongs to reasonable normal data, and a normal numerical test result of the numerical range test is generated. On the contrary, if the comparison determines that it is not within the numerical range corresponding to the historical numerical interval, it is determined that the value of the content cell does not belong to reasonable normal data, that is, there is an abnormality, and a numerical test result of the abnormal numerical range test is generated.

[0082] Furthermore, for the content cells with target column headers, the target data corresponding to the target column headers are searched from the row table data, and then the content cells are logically checked according to the searched target data and the logical relationship. The specific logical check method varies with the logical relationship. If the logical relationship is a product relationship, the logical check of the product relationship is performed; if the logical relationship is an addition relationship, the logical check of the addition relationship is performed. For example, for the standardized table corresponding to the above-mentioned column table data A, B, and C, it contains row table data k1, and k1 contains content cells a, b, and c, and the content cells a, b, and c correspond to the column header cells A', B', and C' of the column table data A, B, and C, respectively. That is, there are target column headers A' and B' corresponding to the column header C', and the logical relationship is a product relationship. Therefore, for the content cell c, a logical check is performed according to the product between the target data in the content cells a and b. Then, based on the test result generated by the logical test, determine whether the logical test passes, compare the generated test result with the value in the content cell to determine whether the two are consistent. If they are consistent, the logical test passes, and if they are inconsistent, the logical test fails.

[0083] Furthermore, for the content cells that pass the logic test, it is necessary to continue to check whether the value size is reasonable. That is, the value in the content cell is compared with the historical value interval to determine whether the value in the content cell is within the value range of the historical value interval. If it is within its value range, it is determined that the value of the content cell belongs to reasonable normal data, and a value test result of normal value range test is generated at this time. On the contrary, if the comparison determines that it is not within the value range corresponding to the historical value interval, it is determined that the value of the content cell does not belong to reasonable normal data, that is, there is an abnormality, and a value test result of abnormal value range test is generated. For the content cells that fail the logic test, it means that the value in the content cell is not generated by the logical relationship, and there is an abnormality, so a value test result of abnormal logic test is generated. After the values ​​of each content cell in the row table data are all generated by the abnormality test and their respective value test results are generated, each value test result is combined according to the arrangement position of each content cell in the row table data, that is, a second test result corresponding to the row table data is formed. In this way, by performing a logic test and a value range rationality test on each content cell in the row table data, the accuracy of the data in each content cell in terms of logical relationship and value size is ensured.

[0084] Step S40, generating each of the first inspection results and each of the second inspection results as the final inspection result output of the cost table, thereby completing the inspection of each cost data in the cost table.

[0085] Furthermore, after performing a null value check and a numerical attribute check on each column of table data to generate the first test results corresponding to each, and performing a numerical anomaly check on each content cell in each row of table data to generate the second test result corresponding to each row of table data, the first test results and the second test results jointly represent the test results of the cost table, so the two are jointly generated as the final test results corresponding to the cost table. Among them, the first test results exist in the form of data columns, and the second test results exist in the form of data rows. According to the position of the table data in the column corresponding to each first test result in the cost table, each first test result is formed into a first matrix, and according to the position of the table data in the row corresponding to each second test result in the cost table, each second test result is formed into a second matrix. The final test result can be generated by adding the first matrix and the second matrix, and the elements in the generated sum matrix reflect the test results of the integrity, numerical attributes and numerical anomalies of the cost data in the cost table. Then, it is determined whether there are abnormal elements in the sum matrix representing the final inspection result. If there are abnormal elements, it means that there are abnormal cost data in the initial cost table. The cell information corresponding to the abnormal element is generated as prompt information, which is output together with the final inspection result to prompt the abnormal cost data in the cost table. If it is determined that there are no abnormal cells, it means that there is no abnormal cost data in the initial cost table, and the final inspection result is directly output. In this way, the integrity, numerical attributes and numerical abnormality of each cost data in the cost table are inspected.

[0086] The construction project cost data inspection method based on big data implemented in this embodiment, when the cost table of the construction project is obtained, it is first converted into a standardized table, and then the converted standardized table is split into columns and rows to generate multiple column table data and multiple row table data. Among them, the column table data is the data obtained by splitting according to the content column of the standardized table, and the column table data is subjected to null value inspection and numerical attribute inspection to ensure that each cell of the cost table has data and the numerical attribute of the data is correct. The row table data is the data obtained by splitting according to the content row of the standardized table, by determining the column header corresponding to each content cell, and then performing a numerical anomaly inspection on it according to the historical numerical interval corresponding to the column header. The historical data interval is generated by a large amount of historical cost data on the column header of multiple other construction projects with similar characteristics to the current construction project. As the cost interval of the construction project reference represented by the column header, the abnormality of the cost data in the cell can be inspected by the size relationship between the cost data in the cell and the cost interval. Then, the first inspection result obtained by the null value inspection and the numerical attribute inspection and the second inspection result obtained by the numerical anomaly inspection are jointly generated as the final inspection result output of the cost table. In this way, by converting various types of cost tables into standardized tables for column splitting and row splitting, the column table data and row table data are obtained for inspection in the form of columns and rows, which improves the inspection efficiency and avoids omissions. In addition, numerical anomaly inspection is performed through historical numerical ranges generated by a large amount of historical cost data, which ensures the accuracy of numerical size inspection and realizes efficient and accurate inspection of the integrity, numerical attributes and numerical anomalies of each cost data in the cost table.

[0087] For further information, please refer to Figure 2 Based on the first embodiment of the construction project cost data verification method based on big data of the present invention, a second embodiment of the construction project cost data verification method based on big data of the present invention is proposed.

[0088] The difference between the second embodiment of the construction project cost data verification method based on big data and the first embodiment of the construction project cost data verification method based on big data is that the steps of converting the cost table into a standardized table, splitting the standardized table into columns and rows, and generating a plurality of column table data and a plurality of row table data include:

[0089] Step S11, identifying the merged cells in the cost table, and splitting the merged cells to obtain split cells;

[0090] Step S12, filling the values ​​in the merged cells into the split cells to form the cost table into a standardized table;

[0091] Step S13, splitting the standardized table into a row header part and a row content part, and performing row splitting on the row content part to generate a plurality of row table data;

[0092] Step S14, splitting the standardized table into a column header part and a column content part, and performing column splitting on the column content part to generate a plurality of columns of table data.

[0093] Furthermore, in the process of converting the cost table into a standardized table, the merged cells in the cost table are first identified, and the cells in the cost table that have undergone a merge operation are identified. Specifically, the identification can be performed by identifying the row identifier and column identifier of the cell, and it is determined whether the cell corresponds to a row identifier and a column identifier. If it corresponds to a row identifier and a column identifier, the cell is determined to be a non-merged cell. If the cell corresponds to multiple row identifiers or multiple column identifiers, the cell is determined to be a merged cell. Then, the determined merged cell is split according to its corresponding row identifier and column identifier to obtain a split cell. In addition, before the merged cell is split, the data contained in the merged cell is applicable to all the split cells, so the value of the data in the merged cell is read and filled into each split cell, and the cost table is formed into a standardized table that does not contain merged cells, and each split cell has the data before the split.

[0094] It is understandable that the cost table may include a header located on the left side of the table to indicate another meaning of the data in the table, in addition to the header located at the top of the table to indicate the meaning of the data in the table. In this embodiment, the header located at the top of the table is used as the row header, and the header located on the left side of the table is used as the column header. Then, the standardized table is split into a row header part and a row content part according to the row header; wherein, the row header is divided into a set consisting of all header rows in the standardized table, and the row content part is a set consisting of all data rows in the standardized table. For the row content part, the splitting is continued according to the row identifier. The data with the same row identifier comes from the same data row. The data row is split from the row content part to split the row content part into rows to obtain the corresponding row table data. One row table data corresponds to one data row. In addition, the standardized table is split into a column header part and a column content part according to the column header; wherein, the column header is divided into a set consisting of all header columns in the standardized table, and the column content part is a set consisting of all data columns in the standardized table. Then, the column content part is further split according to the column identifier. The data with the same column identifier comes from the same data column. This type of data column is split out from the column content part to perform column splitting on the column content part to obtain corresponding column table data. One column table data corresponds to one data column.

[0095] It should be noted that for the splitting of the row header part and the row content part, and the splitting of the list header part and the column content part, the same method can be used for both. In this embodiment, the splitting of the row header part and the row content part is taken as an example for illustration, and the splitting of the list header part and the column content part will not be elaborated further. Specifically, the steps of splitting the standardized table into a row header part and a row content part include:

[0096] Step S131: For each cell in the standardized table, based on a preset word segmentation model, perform word segmentation on the text in the cell to obtain cell keywords;

[0097] Step S132: Generate a semantic weight matrix corresponding to each cell keyword, and correct the keyword vectors corresponding to each cell keyword according to the semantic weight matrix. Perform an average processing on the corrected keyword vectors to obtain the text vector corresponding to the cell;

[0098] Step S133: Generate a vector matrix corresponding to the standardized table from the text vectors respectively corresponding to each cell, and determine the row vector similarity values between adjacent two vector rows in the vector matrix;

[0099] Step S134: Compare the row vector similarity values to determine the target row vector similarity value with the smallest value among the row vector similarity values;

[0100] Step S135: Determine the target row in the standardized table corresponding to the target row vector similarity value, and split the standardized table into a row header part and a row content part according to the target row.

[0101] Furthermore, the background server has previously formed a preset word segmentation model through training. For each cell in the standardized table, the text therein is segmented through this preset word segmentation model to obtain multiple segments. Then, invalid words such as "de", "he", "yu", etc. in the multiple segments are removed to obtain the cell keywords corresponding to each cell.

[0102] Furthermore, the word meaning weight matrix corresponding to each cell keyword is obtained, and the word meaning weight matrix represents the semantic relevance between related words in the construction project. Collect a large number of various documents related to construction projects, and count the frequency probability of each word related to construction projects in the sentences of various documents, and determine the relevance probability between each word. Among them, the relevance probability can be determined by the positional relationship of each word in the sentence. If the words appear in a sentence with a similar positional relationship for many times, the higher the relevance probability between the words, the lower it is. Then, based on the frequency probability of each word and the relevance probability between each word, the word meaning probability between each word is generated. Among them, for the word meaning probability between two specific words, if the frequency probability of the two words is high and the relevance probability between the two words is also high, it means that the semantic relevance between the two words is high, so that the word meaning probability is high, otherwise the word meaning probability is low. For each cell keyword, while taking each keyword as a matrix row and a matrix column, the word meaning probability between each keyword is taken as the element value of the matrix to form a word meaning weight matrix corresponding to each cell keyword.

[0103] Furthermore, the keyword vectors corresponding to each cell keyword are corrected through the word meaning weight matrix, and the semantic accuracy of each cell keyword obtained by word segmentation is reflected through the correction. If the cell keyword represented by the keyword vector after correction does not change from that before correction, it means that the cell keyword obtained by word segmentation is accurate. On the contrary, if the represented cell keyword changes after correction, it means that the cell keyword obtained by word segmentation is inaccurate, so the keyword vector before correction is replaced with the corrected keyword vector. Then, the corrected keyword vectors are averaged to obtain the average result. The average result represents the vector corresponding to the text in the cell, so it is used as the text vector corresponding to the cell.

[0104] Furthermore, after each cell is segmented and corrected to obtain the corresponding text vector, the text vectors are arranged according to the position of each cell in the standardized table to obtain a vector matrix corresponding to the standardized table. Then the similarity values ​​between the adjacent two matrix rows in the vector matrix are calculated to obtain the row vector similarity values. For example, if the vector matrix formed is 3*3, the similarity values ​​between the first row and the second row, and between the second row and the third row are calculated respectively to obtain two row vector similarity values. Then, a comparison is made between the similarity values ​​of each row vector to determine the row vector similarity value with the smallest value. The row vector similarity value with the smallest value indicates that the similarity between rows is the lowest and the data difference between rows is the largest. Generally speaking, compared with the two adjacent matrix rows corresponding to content data, the difference between the two adjacent matrix rows corresponding to the header content and the other corresponding to the content data is greater, so the standardized table can be split according to the row vector similarity value with the smallest value.

[0105] Specifically, the row vector similarity value with the smallest value is used as the target row vector similarity value, and the two-by-two vector rows that generate the target row vector similarity value are found, and then the target row corresponding to the two-by-two vector rows in the standardized table is found, and the standardized table is split into a row header part and a row content part according to the target row. Among them, the target rows corresponding to the two-by-two vector rows are two rows, and the splitting of the row header part and the row content part can be achieved by splitting the two target rows.

[0106] It should be noted that, in order to ensure the accuracy of the preset word segmentation model for the words related to construction engineering, the training of the preset word segmentation model is formed by training the corpus data set related to construction engineering. Specifically, the step of segmenting the text in the cell based on the preset word segmentation model and obtaining the cell keywords includes:

[0107] Step S136, obtaining a corpus data set, and iteratively training the preset word segmentation model based on the corpus data set, and updating each model parameter according to the learning rate of each model parameter in the preset word segmentation model during the iterative training process:

[0108] Step S137, when the number of iterative training reaches a preset number, the objective function of the preset word segmentation model is calculated to obtain a calculation result;

[0109] Step S138, judging whether the preset word segmentation model has converged according to the calculation result, and if the preset word segmentation model has converged, completing the iterative training of the preset word segmentation model;

[0110] Step S139: if the preset word segmentation model has not converged, executing the step of iteratively training the preset word segmentation model based on the corpus dataset.

[0111] Furthermore, a large number of various texts related to construction project costs are obtained, and various texts are processed into sentences and words, and the sentences and words are cleaned to form a corpus data set, and the preset word segmentation model is iteratively trained based on the preset data set. Among them, different learning rates are set for different model parameters of the preset word segmentation model, and different learning rates correspond to different parameter update frequencies. In the iterative training process, each model parameter is updated according to the learning rate corresponding to each model parameter. In this way, each model parameter is trained according to a different number of updates, avoiding the simultaneous update of each model parameter, which increases the complexity of model training.

[0112] Furthermore, a preset number of times is set in advance. When the number of iterative training reaches the preset number of times, the objective function of the preset word segmentation model is calculated to obtain a corresponding calculation result. The objective function can be a loss function of the preset word segmentation model, which is used to judge the difference between the actual word segmentation performance of the preset word segmentation model and the expected word segmentation performance. The smaller the difference, the closer the actual word segmentation performance of the preset word segmentation model is to the expected word segmentation performance, and the better the word segmentation effect. The specific calculation formula for calculating the objective function can be found in the following formula (1).

[0113]

[0114] Among them, LM represents the calculation result, Q represents the number of corpora in the corpus dataset, and m p Indicates the number of words that the preset word segmentation model divides the pth corpus in the corpus dataset, m p0 represents the number of reference words corresponding to the p-th corpus, S p S represents the word vector of the p-th corpus divided by the preset word segmentation model, p0 represents the reference word vector corresponding to the p-th corpus, γ represents the restriction parameter, b represents the bias value of the preset word segmentation model, and w represents the weight value of the preset word segmentation model.

[0115] In this way, the accuracy of the preset word segmentation model in dividing the number of words is reflected through the number of words and the number of reference words, and the accuracy of the preset word segmentation model in dividing the semantics of words is reflected through the word vector and the reference word vector, and the combination of the two ensures the accuracy of the preset word segmentation model in the number of word segmentations and semantics. In addition, by setting restriction parameters to limit the overall change range of the bias value and weight value of the preset word segmentation model, it is prevented that the overall change range of the bias value and weight value is too large, resulting in overfitting of the model, and only the word segmentation effect of the corpus data set is good, but the actual word segmentation effect is poor.

[0116] Furthermore, after the objective function is calculated to obtain the corresponding calculation result, a judgment is made on whether the preset word segmentation model has converged based on the calculation result. The calculation result obtained by the current calculation is compared with the calculation result obtained by the previous calculation to determine whether the difference between the two is greater than a preset threshold. If it is greater than the preset threshold, it means that the calculation result of the objective function has not yet stabilized, and the preset word segmentation model has not converged. It is necessary to continue to iteratively train the preset word segmentation model through the corpus data set until it is determined that the preset word segmentation model converges. On the contrary, if the difference between the two is less than the preset threshold, it means that the calculation result of the objective function has stabilized, the preset word segmentation model has converged, and the iterative training of the preset word segmentation model is completed.

[0117] Among them, in order to ensure the accuracy of the judgment, after determining that the difference between the two is less than the preset threshold, the calculation results obtained by the previous calculations can be obtained for comparison, and the difference between the calculation results of two consecutive adjacent times can be analyzed to see whether they are all less than the preset threshold. For example, the calculation results of the first three times are obtained, and a total of three differences are generated between the third and second times, the second and first times, and the first and current times, respectively, to determine whether the three differences are all less than the preset threshold. If they are all less than the preset threshold, it is determined that the preset word segmentation model has stably converged and the training of the preset word segmentation model is completed. On the contrary, if there is a time that is greater than the preset threshold, it means that the preset word segmentation model has not yet stably converged and iterative training needs to continue.

[0118] This embodiment converts various types of cost tables into standardized tables by setting up a standardized table mechanism, and then splits the standardized table into columns and rows to obtain corresponding row table data and column table data, so that no matter what type of table, it has standardized row table data and column table data, which is conducive to performing null value checks, numerical attribute checks and numerical anomaly checks in a standardized manner.

[0119] For further information, please refer to Figure 3 Based on the first and second embodiments of the construction project cost data verification method based on big data of the present invention, a third embodiment of the construction project cost data verification method based on big data of the present invention is proposed.

[0120] The third embodiment of the construction engineering cost data verification method based on big data is different from the first and second embodiments of the construction engineering cost data verification method based on big data in that the step of performing a value abnormality check on the content cell according to the historical value interval corresponding to the column header includes:

[0121] Step S50, reading a plurality of historical data corresponding to the column header, and selecting any target historical data from each of the historical data, and calculating distance parameters between the target historical data and each of the historical data;

[0122] Step S60, arranging the distance parameters in ascending order, and generating the difference between any two adjacent distance parameters according to the arrangement order;

[0123] Step S70, determining a clustering radius according to each of the difference values, and clustering each of the historical data according to the clustering radius to form a historical data group;

[0124] Furthermore, in order to make the numerical anomaly inspection of the content cell more accurate based on the historical numerical interval, it is necessary to form an accurate historical numerical interval in advance based on a large amount of historical data. Specifically, a large amount of historical data at the column header of other construction projects with similar characteristics is read, and any historical data is selected from each historical data as the target historical data. Then, based on the target historical data, the distance parameter between it and other historical data is calculated. In addition, in order to reflect the difference between other historical data and the target historical data, each distance parameter is arranged in order from small to large, and then the difference operation is performed between the adjacent two distance parameters according to the arrangement order, and the distance parameter arranged in the back is subtracted from the distance parameter arranged in the front among the adjacent two distance parameters to obtain the difference between each adjacent two distance parameters.

[0125] Understandably, for each construction project with similar characteristics, each should have similar values ​​on the items represented by the column header. The difference between each adjacent distance parameter indicates the degree of similarity between the values. The closer the differences are, the more similar the distance parameters generating the differences are, and the closer the historical data generating the distance parameters are. Such historical data can be used to represent the data range of the construction project on the items represented by the column header, and such historical data can be clustered and generate relevant historical numerical intervals. Therefore, the concentration range of each difference can be determined by analyzing each difference, and the concentration range represents the close difference, and the radius of the cluster is determined based on it. In a specific embodiment, a plane XY coordinate system is established, and each difference is added to the X-axis or Y-axis respectively, and the concentration position of each difference on the X-axis or Y-axis is determined, and the radius of the cluster is determined by the coordinate point of the concentration position. For example, after analysis, it is determined that 80% of the differences are concentrated near the point with the X-axis coordinate value p, and the cluster radius is determined as p.

[0126] Furthermore, after the clustering radius is determined, the historical data can be clustered to form historical data groups. Specifically, the steps of clustering the historical data according to the clustering radius to form historical data groups include:

[0127] Step S71, obtaining a preset clustering density, and clustering the historical data according to the clustering radius and the preset clustering density to form an initial data group;

[0128] Step S72, calculating the density index and proximity index of the initial data group class according to a preset evaluation formula;

[0129] Step S73, obtaining a density weight and a proximity weight corresponding to the density index and the proximity index respectively, and respectively modifying the density index and the proximity index according to the density weight and the proximity weight to obtain an evaluation index corresponding to the initial data group class;

[0130] Step S74, comparing the evaluation index with a preset index threshold to determine whether the evaluation index is greater than the preset index threshold; if so, generating the initial data group class as a historical data group class;

[0131] Step S75, if the evaluation index is less than or equal to the preset index threshold, the preset clustering density is updated, and based on the updated preset clustering density, the step of clustering the historical data according to the clustering radius and the preset clustering density is executed until the evaluation index is less than the preset index threshold.

[0132] Furthermore, a preset clustering density is set in advance based on experience, and the historical data is clustered in combination with the preset clustering density and clustering radius. Select any point from the historical data as the clustering starting point, identify other historical data points in the circular area formed with the point as the center and the clustering radius as the radius, and judge that the number of other historical data points is greater than the number represented by the preset clustering density. If it is greater than the number represented by the preset clustering density, it means that the selected arbitrary point is the clustering core point. Then, each point in the circular area is taken as the center, and the number of other historical data points identified within the clustering radius is compared with the number represented by the preset clustering density to determine the core point of each point. In this way, after all historical data are judged, all core points in the historical data can be determined, and points located in the circular area but not core points are regarded as edge points, and points not located in the circular area are regarded as invalid points. All core points and edge points are points that are close to each other and can be classified into one category to form an initial data group class. The number of initial data group classes can be one or more; if all core points can be associated through circular areas, that is, there are intersections between the circular areas, then the number of initial data group classes formed is one; if there are core points that cannot be associated through circular areas, that is, there are at least two circular areas that have no intersection, then the number of initial group classes formed is greater than one. It should be noted that for historical data with similar characteristics, the initial number of group classes formed by clustering is usually one. If there are more than one, it means that the initial number of group classes formed by clustering is inaccurate and needs to be re-clustered.

[0133] Furthermore, in order to evaluate the accuracy of clustering, a preset evaluation formula is pre-set, and the density index and proximity index of the initial data group class are calculated by the preset evaluation formula. The density index is used to evaluate the density of data points of the same initial data group class, and the proximity index is used to evaluate the distance between different initial data group classes. For a certain initial data group class, the higher the density of its internal data points and the farther the distance from other adjacent initial data group classes, the more accurate the clustering. Among them, the preset evaluation formula can be specifically referred to as the following formula (2).

[0134]

[0135] Among them, ρ represents the density index, C represents the number of initial data group classes, represents the variance of the i-th initial data group, N represents the number of data contained in the historical data, x k represents the kth data in the historical data, μ represents the proximity index, exp represents the natural exponential function, Xci represents the center of the i-th initial data group class, Xcj represents the center of the j-th initial data group class, represents the Gaussian constant.

[0136] Furthermore, the density index and the proximity index reflect the accuracy of clustering from two dimensions, namely, the data aggregation and the distance from other initial data groups. The two dimensions have different degrees of importance in affecting the accuracy. Therefore, the density weight and proximity weight representing the importance of the two are set in advance based on experience, and the density weight corresponding to the density index and the proximity weight corresponding to the proximity index are obtained. Then, the density index and the proximity index are modified according to the density weight and the proximity weight to obtain the evaluation index for evaluating the accuracy of the initial data group classification. Among them, the modification of the density index and the proximity index can be carried out according to the following formula (3).

[0137] δ=τ1*ρ+τ2*μ (3);

[0138] Among them, δ represents the modified evaluation index, τ1 represents the density weight, τ2 represents the proximity weight, and τ1+τ2=1.

[0139] Furthermore, in order to indicate the level of the evaluation index, a preset index threshold is set in advance, and the obtained evaluation index is compared with the preset index threshold to determine whether the evaluation index is greater than the preset index threshold. If it is greater than the preset index threshold, the accuracy of the initial data group classification is determined to be high, and the initial data group classification is generated as a historical data group classification. On the contrary, if the evaluation index is determined to be less than or equal to the preset index threshold through comparison, the accuracy of the initial data group classification is determined to be low. At this time, the preset clustering density is updated according to the preset update formula, and the updated preset clustering density is combined with the clustering radius to re-cluster the historical data until the obtained evaluation index is less than the preset index threshold, and the initial data group classification is generated as a historical data group classification.

[0140] Step S80, sorting the data of each group in the historical data group in ascending order to generate a historical sequence;

[0141] Step S90, determining the quartile values ​​in the historical sequence, and determining the historical value interval based on the quartile values.

[0142] Furthermore, after the historical data groups are generated by clustering, the data of each group of the historical data groups are sorted in order from small to large values ​​to generate a historical sequence. Then the quartile values ​​in the historical sequence are determined. The quartile values ​​are obtained by dividing the data in the historical sequence into 4 equal parts, and then finding the values ​​at the last digit of the first, second and third equal parts, respectively, as the upper quartile values, middle quartile values ​​and lower quartile values ​​in the quartile values. Then the quartile distance is determined by the difference calculation between the upper quartile value and the lower quartile value, and the lower boundary value is determined by the upper quartile value combined with the quartile distance, and the upper boundary value is determined by the lower quartile value combined with the quartile distance. The interval formed by the lower boundary value and the upper boundary value is the historical numerical interval used for numerical abnormality testing.

[0143] This embodiment collects a large amount of historical data corresponding to the column header for clustering. In the clustering process, the clustering radius is determined according to the difference in the distance parameter formed by each historical data, so that the clustering radius is closely related to each historical data, ensuring the accuracy of the clustering radius, which is conducive to accurate clustering. At the same time, for the clustering of the initial data group, the clustering accuracy is evaluated through the two dimensions of density index, density weight reflected by the density within the group and proximity index, proximity weight reflected by the proximity outside the group, which is further conducive to improving the accuracy of clustering. In addition, for the historical data group formed by clustering, the historical numerical range is determined in combination with the quartile value, and the abnormal data is further excluded, so that the formed historical numerical range is more accurate, thereby improving the accuracy of the numerical abnormality test.

[0144] In addition, the embodiment of the present invention also provides a construction engineering cost data verification system based on big data. Figure 4 , Figure 4 It is a structural schematic diagram of the equipment hardware operating environment involved in the implementation scheme of the construction project cost data verification system based on big data of the present invention.

[0145] like Figure 4 As shown, the construction engineering cost data verification system based on big data may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a storage 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The storage 1005 may be a high-speed RAM storage, or a stable storage (non-volatile memory), such as a disk storage. The storage 1005 may also be a storage device independent of the aforementioned processor 1001.

[0146] Those skilled in the art will understand that Figure 4 The hardware structure of the construction project cost data verification system based on big data shown in the figure does not constitute a limitation of the construction project cost data verification system based on big data, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0147] like Figure 4 As shown, the storage 1005 as a storage medium may include an operating system, a network communication module, a user interface module and a control program. The operating system is a program for managing and controlling the construction engineering cost data inspection system and software resources based on big data, and supports the operation of the network communication module, the user interface module, the control program and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.

[0148] exist Figure 4 In the hardware structure of the construction engineering cost data verification system based on big data shown in the figure, the network interface 1004 is mainly used to connect to the system server and communicate data with the system server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; the processor 1001 can call the control program stored in the storage 1005 and perform the following operations:

[0149] When a cost table of a construction project is obtained, the cost table is converted into a standardized table, and the standardized table is split into columns and rows to generate a plurality of column table data and a plurality of row table data;

[0150] Performing a null value check and a numerical attribute check on each of the table data in the columns, and generating a first check result corresponding to each of the table data in the columns;

[0151] For each row of table data, determine a column header corresponding to each content cell in the row of table data, and perform a value abnormality check on the content cell according to a historical value interval corresponding to the column header to generate a second test result corresponding to the row of table data;

[0152] Each of the first inspection results and each of the second inspection results is generated as the final inspection result output of the cost table, thereby completing the inspection of each cost data in the cost table.

[0153] Furthermore, the step of performing a value abnormality check on the content cell according to the historical value interval corresponding to the column header to generate a second test result corresponding to the row table data includes:

[0154] Determine whether there is a target column header having a logical relationship with the column header in the standardized table, and if the target column header exists, search for target data corresponding to the target column header in the row table data;

[0155] Performing a logic check on the content cell according to the logical relationship and the target data, and determining whether the logic check passes;

[0156] If the logic check passes, the value of the content cell is checked for abnormality according to the historical value interval, a value check result corresponding to the content cell is generated, and the value check result corresponding to each content cell is formed as the second check result.

[0157] Furthermore, the step of performing a null value check and a numerical attribute check on each of the table data in the columns to generate a first check result corresponding to each of the table data in the columns includes:

[0158] For each of the table data, determine whether there is a null value column cell in each column cell included in the table data, and generate a null value test result;

[0159] Determine the column attribute corresponding to the column table data, and judge whether there is a target value that does not match the column attribute in the value of each column cell, and generate an attribute inspection result;

[0160] The null value check result and the attribute check result are jointly generated as the first check result.

[0161] Furthermore, the steps of converting the cost table into a standardized table, splitting the standardized table into columns and rows, and generating a plurality of column table data and a plurality of row table data include:

[0162] Identify the merged cells in the cost table, and split the merged cells to obtain split cells;

[0163] Filling the values ​​in the merged cells into each of the split cells to form the cost table into a standardized table;

[0164] Splitting the standardized table into a row header part and a row content part, and performing row splitting on the row content part to generate a plurality of row table data;

[0165] The standardized table is split into a column header part and a column content part, and the column content part is split into columns to generate a plurality of column table data.

[0166] Furthermore, the step of splitting the standardized table into a row header part and a row content part includes:

[0167] For each cell in the standardized table, based on a preset word segmentation model, the text in the cell is segmented to obtain cell keywords;

[0168] Generate a word meaning weight matrix corresponding to each of the cell keywords, and modify the keyword vector corresponding to each of the cell keywords according to the word meaning weight matrix, perform mean processing on each of the modified keyword vectors, and obtain a text vector corresponding to the cell;

[0169] Generate the text vectors corresponding to each of the cells into a vector matrix corresponding to the standardized table, and determine the row vector similarity values ​​between adjacent vector rows in the vector matrix;

[0170] Compare the row vector similarity values ​​to determine the target row vector similarity value with the smallest value among the row vector similarity values;

[0171] A target row in the standardized table corresponding to the target row vector similarity value is determined, and the standardized table is split into a row header part and a row content part according to the target row.

[0172] Furthermore, before the step of segmenting the text in the cell based on the preset segmentation model to obtain the cell keyword, the processor 1001 may call the control program stored in the storage 1005 and perform the following operations:

[0173] Acquire a corpus data set, and iteratively train the preset word segmentation model based on the corpus data set, and update each model parameter according to the learning rate of each model parameter in the preset word segmentation model during the iterative training process:

[0174] When the number of iterative training reaches the preset number, the objective function of the preset word segmentation model is calculated to obtain a calculation result, and the calculation formula is:

[0175]

[0176] Among them, LM represents the calculation result, Q represents the number of corpora in the corpus dataset, and m p Indicates the number of words that the preset word segmentation model divides the pth corpus in the corpus dataset, m p0 represents the number of reference words corresponding to the p-th corpus, S p S represents the word vector of the p-th corpus divided by the preset word segmentation model, p0 represents the reference word vector corresponding to the p-th corpus, γ represents the restriction parameter, b represents the bias value of the preset word segmentation model, and w represents the weight value of the preset word segmentation model;

[0177] According to the calculation result, judging whether the preset word segmentation model has converged, and if the preset word segmentation model has converged, completing the iterative training of the preset word segmentation model;

[0178] If the preset word segmentation model has not converged, a step of iteratively training the preset word segmentation model based on the corpus data set is executed.

[0179] Furthermore, before the step of performing a value anomaly check on the content cell according to the historical value interval corresponding to the column header, the processor 1001 may call a control program stored in the storage 1005 and perform the following operations:

[0180] Reading a plurality of historical data corresponding to the column header, and selecting any target historical data from each of the historical data, and calculating distance parameters between the target historical data and each of the historical data;

[0181] Arrange the distance parameters in ascending order, and generate the difference between any two adjacent distance parameters according to the arrangement order;

[0182] Determine a clustering radius according to each of the differences, and cluster each of the historical data according to the clustering radius to form a historical data group;

[0183] Sorting each group of data in the historical data group class in ascending order to generate a historical sequence;

[0184] Quartile values ​​in the historical sequence are determined, and the historical value interval is determined based on the quartile values.

[0185] Furthermore, the step of clustering the historical data according to the clustering radius to form historical data groups includes:

[0186] Obtaining a preset clustering density, and clustering the historical data according to the clustering radius and the preset clustering density to form an initial data group;

[0187] The density index and proximity index of the initial data group class are calculated according to a preset evaluation formula, and the preset evaluation formula is:

[0188]

[0189] Among them, ρ represents the density index, C represents the number of initial data group classes, represents the variance of the i-th initial data group, N represents the number of data contained in the historical data, x k represents the kth data in the historical data, μ represents the proximity index, exp represents the natural exponential function, Xci represents the center of the i-th initial data group class, Xcj represents the center of the j-th initial data group class, represents the Gaussian constant.

[0190] Obtaining density weights and proximity weights corresponding to the density index and proximity index, respectively, and respectively modifying the density index and proximity index according to the density weight and the proximity weight, to obtain an evaluation index corresponding to the initial data group class;

[0191] Compare the evaluation index with a preset index threshold to determine whether the evaluation index is greater than the preset index threshold, and if so, generate the initial data group class as a historical data group class;

[0192] If the evaluation index is less than or equal to the preset index threshold, the preset clustering density is updated, and based on the updated preset clustering density, the step of clustering the historical data according to the clustering radius and the preset clustering density is performed until the evaluation index is less than the preset index threshold.

[0193] The specific implementation of the construction project cost data verification system based on big data of the present invention is basically the same as the various embodiments of the above-mentioned construction project cost data verification method based on big data, and will not be repeated here.

[0194] The embodiment of the present invention further provides a medium. The medium is a readable storage medium, and a control program is stored on the storage medium. When the control program is executed by a processor, the steps of the construction project cost data inspection method based on big data are implemented.

[0195] The storage medium of the present invention may be a computer-readable storage medium, and its implementation method is basically the same as the above-mentioned embodiments of the construction project cost data verification method based on big data, and will not be repeated here.

[0196] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.

Claims

1. A construction engineering cost data verification method based on big data, characterized in that: The construction engineering cost data verification method comprises: When a cost table of a construction project is obtained, the cost table is converted into a standardized table, and the standardized table is split into columns and rows to generate a plurality of column table data and a plurality of row table data; Performing a null value check and a numerical attribute check on each of the table data in the columns, and generating a first check result corresponding to each of the table data in the columns; For each row of table data, determine a column header corresponding to each content cell in the row of table data, and perform a value abnormality check on the content cell according to a historical value interval corresponding to the column header to generate a second test result corresponding to the row of table data; Each of the first inspection results and each of the second inspection results is generated as the final inspection result output of the cost table, thereby completing the inspection of each cost data in the cost table.

2. The construction engineering cost data verification method according to claim 1, characterized in that: The step of performing a value abnormality check on the content cell according to the historical value interval corresponding to the column header to generate a second test result corresponding to the row table data includes: Determine whether there is a target column header having a logical relationship with the column header in the standardized table, and if the target column header exists, search for target data corresponding to the target column header in the row table data; Performing a logic check on the content cell according to the logical relationship and the target data, and determining whether the logic check passes; If the logic check passes, the value of the content cell is checked for abnormality according to the historical value interval, a value check result corresponding to the content cell is generated, and the value check result corresponding to each content cell is formed as the second check result.

3. The construction engineering cost data verification method according to claim 1, characterized in that: The step of performing a null value check and a numerical attribute check on each of the table data in the columns to generate a first check result corresponding to each of the table data in the columns comprises: For each of the table data, determine whether there is a null value column cell in each column cell included in the table data, and generate a null value test result; Determine the column attribute corresponding to the column table data, and judge whether there is a target value that does not match the column attribute in the value of each column cell, and generate an attribute inspection result; The null value check result and the attribute check result are jointly generated as the first check result.

4. The construction engineering cost data verification method according to any one of claim 1, characterized in that: The steps of converting the cost table into a standardized table, splitting the standardized table into columns and rows, and generating a plurality of column table data and a plurality of row table data include: Identify the merged cells in the cost table, and split the merged cells to obtain split cells; Filling the values ​​in the merged cells into each of the split cells to form the cost table into a standardized table; Splitting the standardized table into a row header part and a row content part, and performing row splitting on the row content part to generate a plurality of row table data; The standardized table is split into a column header part and a column content part, and the column content part is split into columns to generate a plurality of column table data.

5. The construction engineering cost data verification method according to claim 4, characterized in that: The step of splitting the standardized table into a row header part and a row content part comprises: For each cell in the standardized table, based on a preset word segmentation model, the text in the cell is segmented to obtain cell keywords; Generate a word meaning weight matrix corresponding to each of the cell keywords, and modify the keyword vector corresponding to each of the cell keywords according to the word meaning weight matrix, perform mean processing on each of the modified keyword vectors, and obtain a text vector corresponding to the cell; Generate the text vectors corresponding to each of the cells into a vector matrix corresponding to the standardized table, and determine the row vector similarity values ​​between adjacent vector rows in the vector matrix; Compare the row vector similarity values ​​to determine the target row vector similarity value with the smallest value among the row vector similarity values; A target row in the standardized table corresponding to the target row vector similarity value is determined, and the standardized table is split into a row header part and a row content part according to the target row.

6. The construction engineering cost data verification method according to claim 5, characterized in that: The step of segmenting the text in the cell based on a preset segmentation model to obtain cell keywords includes: Acquire a corpus data set, and iteratively train the preset word segmentation model based on the corpus data set, and update each model parameter according to the learning rate of each model parameter in the preset word segmentation model during the iterative training process: When the number of iterative training reaches the preset number, the objective function of the preset word segmentation model is calculated to obtain a calculation result, and the calculation formula is: Among them, LM represents the calculation result, Q represents the number of corpora in the corpus dataset, and m p Indicates the number of words that the preset word segmentation model divides the pth corpus in the corpus dataset, m p0 represents the number of reference words corresponding to the p-th corpus, S p S represents the word vector of the p-th corpus divided by the preset word segmentation model, p0 represents the reference word vector corresponding to the p-th corpus, γ represents the restriction parameter, b represents the bias value of the preset word segmentation model, and w represents the weight value of the preset word segmentation model; According to the calculation result, judging whether the preset word segmentation model has converged, and if the preset word segmentation model has converged, completing the iterative training of the preset word segmentation model; If the preset word segmentation model has not converged, a step of iteratively training the preset word segmentation model based on the corpus data set is executed.

7. The construction engineering cost data verification method according to any one of claims 1 to 6, characterized in that: The step of performing a value abnormality check on the content cell according to the historical value interval corresponding to the column header includes: Reading a plurality of historical data corresponding to the column header, and selecting any target historical data from each of the historical data, and calculating distance parameters between the target historical data and each of the historical data; Arrange the distance parameters in ascending order, and generate the difference between any two adjacent distance parameters according to the arrangement order; Determine a clustering radius according to each of the differences, and cluster each of the historical data according to the clustering radius to form a historical data group; Sorting each group of data in the historical data group class in ascending order to generate a historical sequence; Quartile values ​​in the historical sequence are determined, and the historical value interval is determined based on the quartile values.

8. The construction engineering cost data verification method according to claim 7, characterized in that: The step of clustering the historical data according to the cluster radius to form historical data groups comprises: Obtaining a preset clustering density, and clustering the historical data according to the clustering radius and the preset clustering density to form an initial data group; The density index and proximity index of the initial data group class are calculated according to a preset evaluation formula, and the preset evaluation formula is: Among them, ρ represents the density index, C represents the number of initial data group classes, represents the variance of the i-th initial data group, N represents the number of data contained in the historical data, x k represents the kth data in the historical data, μ represents the proximity index, exp represents the natural exponential function, Xci represents the center of the i-th initial data group class, Xcj represents the center of the j-th initial data group class, represents the Gaussian constant. Obtaining density weights and proximity weights corresponding to the density index and proximity index, respectively, and respectively modifying the density index and proximity index according to the density weight and the proximity weight, to obtain an evaluation index corresponding to the initial data group class; Compare the evaluation index with a preset index threshold to determine whether the evaluation index is greater than the preset index threshold, and if so, generate the initial data group class as a historical data group class; If the evaluation index is less than or equal to the preset index threshold, the preset clustering density is updated, and based on the updated preset clustering density, the step of clustering the historical data according to the clustering radius and the preset clustering density is performed until the evaluation index is less than the preset index threshold.

9. A construction engineering cost data verification system based on big data, characterized in that: The rail transit file parsing system includes a storage device, a processor, a communication bus, and a control program stored in the storage device: The communication bus is used to realize the connection and communication between the processor and the storage; The processor is used to execute the control program to implement the steps of the construction project cost data verification method based on big data as described in any one of claims 1-8.

10. A medium, characterized in that The medium is a readable storage medium, on which a control program is stored. When the control program is executed by a processor, the steps of the construction project cost data verification method based on big data as described in any one of claims 1 to 8 are implemented.

Images