An artificial intelligence-based engineering cost data cleaning method

Abstract

The application discloses an engineering cost data cleaning method based on artificial intelligence and relates to the technical field of data processing.The application comprises the following steps: identifying each tab category according to a cost file category, using a decision tree method to train a tab application model through machine learning, identifying a file table header according to a tab type, identifying a hierarchical relationship and a containing relationship for a sub-item related tab, intelligently coding a label for a hierarchical relationship according to a set business regulation code, identifying and nesting a cross-tab content existing in a related relationship, and returning in a structured data mode, identifying an index attribute of content in a tab, and packing a file content identification result into formatted data and returning to an application system.The application adopts artificial intelligence identification technology, identifies and cleans original cost files, greatly improves index system data entry efficiency, and solves the problems of low process efficiency, large errors and high cost in the process of data arrangement and entry by using manpower.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data processing, and particularly relates to an engineering cost data cleaning method based on artificial intelligence. BACKGROUND

[0002] The engineering cost refers to the construction cost of a project in a construction period. The engineering cost management refers to the work of predicting, planning, controlling, accounting, analyzing and evaluating the engineering cost by comprehensively using the knowledge and skills of management, economics and engineering technology and the like. In the field of engineering cost, the data is very large and complex, and not all data in the massive data can be utilized. Therefore, the existing technology proposes an engineering cost data cleaning method to solve the problem.

[0003] As disclosed in CN116303398A, a historical engineering cost data cleaning method, which specifically is to distribute a corresponding number of data cleaning nodes for cleaning processing according to the cleaning coefficient FM; the computing power occupation of each data cleaning node is monitored and analyzed, and the computing power surplus coefficient KY of each data cleaning node is calculated, and the data cleaning nodes in the top MK are selected as target nodes; the cleaning resources are reasonably distributed to maximize resource utilization and effectively improve data processing efficiency; the target node cleans the original engineering cost data to screen out invalid data in the original engineering cost data and obtains target business data; as disclosed in CN115759015A, a cost analysis data processing method and system, which specifically is to import and read the file provided by the user as a template entry file for analysis, match the parsed data to generate an intelligent parsing template, use matching strategies to meet the customer's demand for cost data weight, use the intelligent parsing template directly to parse subsequent files, improve the process efficiency of cost file parsing, and finally the parsed file is visually displayed for the convenience of the customer; as disclosed in CN115587232A, an engineering cost data intelligent management method and system, which specifically is to classify historical engineering data according to the difference value, calculate the first difference influence coefficient of each project cost in each classification, extract a number of influence factors in the first difference influence coefficient effective project and the factor influence coefficient corresponding to the influence factor, and take the factor influence coefficient as the influence coefficient of the subsequent cost process to participate in the engineering cost; the data processing unit takes the engineering cost difference influence coefficient corresponding to the influence factor as the influence coefficient of the subsequent cost process to participate in the engineering cost, which improves the matching degree precision of the estimated engineering cost and the actual engineering cost; as disclosed in CN115630620A, an engineering cost analysis data acquisition and processing method and system, which specifically is to query the total table column title corresponding to the sub-table column title and the standard format corresponding to the total table column title in the preset total table according to the sub-table column title; clear the format of the column data corresponding to the sub-table column title, and reset the column data after clearing the format to obtain standard data according to the standard format; when all the column data corresponding to the sub-table column title in the sub-table are reset to standard data, the sub-table is integrated into the preset total table; when all the sub-tables to be integrated are integrated into the total table, the total table is output.

[0004] Although the engineering cost historical data is formed according to the list standard or quota, there is no systematic unified standard in the data format and data organization structure, even in the subdivided engineering field. Different cost staff and different projects have their own cost data organization methods. Although the same list standard or budget quota is adopted, the names, numbers, characteristic information of the quota items, list items and summary items, and even the units of part of the values in the cost data are difficult to unify, such as the units of human, financial and machine data, and there are many non-uniform situations.

[0005] Under the above circumstances, when collecting and cleaning the engineering cost historical data, various coefficients need to be used as main parameters to determine the limitations of the existence of distorted data. The above CN116303398A and CN115587232A two technical solutions are difficult to truly solve the problem of difficult identification caused by inaccurate data. Incorrect identification of original data makes it difficult to achieve the effect of subsequent processing. Using the data file provided by the user as a template, the user file often only reflects the user's organization method and habit of engineering cost data, and the compatibility is not high and cannot be widely applied. CN115759015A also has differences in the table structure of the original data under the condition of non-standard data, so that the table headers of the total table and the sub-table have limitations in corresponding to the simple key or full name, and it is difficult to correspond to the situation. Therefore, the above-mentioned solutions of the prior art have certain defects. SUMMARY

[0006] The present application provides an engineering cost data cleaning method based on artificial intelligence, which solves the problem that after the development of the cost index system in the past, the huge amount of historical cost data needs to be re-written according to the fixed standard format during the implementation process, and at the same time, the context level relationship in the original EXCEL file needs to be manually added with a digital number, such as using 1.1.2 to represent 1.1 as the superior of the current line, and 1.1.2.1 as the subordinate of the current line. The process not only has a huge workload, but also is prone to errors when manually labeling the level. The present application uses artificial intelligence recognition technology to recognize and clean the original cost file, greatly improves the index system data entry efficiency, and solves the problems in the background technology.

[0007] To solve the above technical problems, the present application is realized by the following technical scheme:

[0008] The engineering cost data cleaning method based on artificial intelligence provided by the present application comprises the following steps:

[0009] S1, according to the cost file category, identify each tab category, take the file tab name as a parameter, through the key variable, use the decision tree method to train the tab application model by machine learning;

[0010] S2, according to the tab type, identify the file table header: specifically, the file project category of different categories is predefined, the header is taken as a parameter, the key variable is set, and the table header recognition application model is trained by using the decision tree method by machine learning;

[0011] S3, the hierarchical and containing relationship of the sub-item related tabs is identified;

[0012] S4, the hierarchical relationship is intelligently coded according to the set business regulation code: specifically, the file hierarchical mechanism that has been identified and the predefined label structure are associated and coded;

[0013] S5, the contents of the cross-tab are identified and nested in the associated relationship, and returned in the form of structured data; specifically, the results of the fourth step are selected according to the tab category, and the tabs that need to be associated with each other are selected; according to the data file content key item, the matching is confirmed, and a Json format nested structured data is formed if the matching is successful;

[0014] S6, the index attribute of the content in the tab is identified, which includes the engineering quantity index and the consumption quantity index, specifically according to the project name as a parameter, setting the key variable, machine learning training of decision tree algorithm, outputting engineering quantity index and consumption quantity index application recognition model;

[0015] S7, the file content recognition result is packaged into formatted data and returned to the application system.

[0016] Further, the cost file category includes estimation file, budget file, budget file, settlement file and final account file.

[0017] Further, the relationship identification includes means of word semantics, writing method, font color, number sequence, number case distinction.

[0018] Further, the data file content key item includes project name, project specification, engineering quantity, engineering amount.

[0019] Further, the engineering cost data cleaning method based on artificial intelligence is realized based on an engineering cost data cleaning system, which includes:

[0020] Data interface module: used for data exchange with the main application system, accepting request parameters and data files;

[0021] File pre-checking module: after receiving the file, the file is pre-checked to check whether the table tabs are complete or not, whether the necessary content in the tabs is included or can be successfully identified, and if the check fails, the corresponding error code is returned;

[0022] Tab and header identification module: specifically, according to the pre-defined file project category of different tab categories, the table header is used as a parameter, key variables are set, and a decision tree method is used to train a table header identification application model through machine learning;

[0023] Hierarchical structure identification module: intelligent coding tags are performed according to the set business regulation code, specifically, the identified file hierarchical structure and the pre-defined tag structure are associated, and a code is assigned;

[0024] Cross-tab association and inclusion relationship identification module: according to the tab category, the tabs that need to be associated with each other are selected, and the data file content key items are matched and confirmed, and a Json format nested structured data is formed if the matching is successful;

[0025] Supply and consumption index identification module: the index attributes of the engineering quantity index and the consumption index contained in the tab are identified, specifically, the project name is used as a parameter, key variables are set, and a decision tree algorithm is used for machine learning training to output an engineering quantity index and consumption index application identification model.

[0026] The present application has the following beneficial effects compared with the prior art:

[0027] The present application uses artificial intelligence recognition technology, can simulate the work process of manual sorting of cost file data, and has the advantages of standardization and good correlation, can greatly improve the work efficiency and work quality of data entry, and solves the problems of low efficiency, large error and high cost in the process of manual data sorting and entry.

[0028] Of course, implementing any product of the present application does not necessarily require achieving all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed for the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0030] Figure 1 A step diagram of a method for cleaning engineering cost data based on artificial intelligence according to the present application;

[0031] Figure 2 The system topology of the engineering cost data cleaning system. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the protection scope of the present application.

[0033] In the past, after the development of the cost index system is completed, in the historical data input implementation process, a large amount of historical cost files need to be re-written according to fixed standard formats, and at the same time, the context level relationship in the original file (ECEL table) needs to be manually added with a digital number (such as 1.1.2) to mark that 1.1 is the superior of the current line, and 1.1.2.1 is the subordinate of the current line. This process not only has a huge workload, but also is prone to errors in manual marking of the level. The present application uses artificial intelligence recognition technology to recognize and clean the original cost file, greatly improving the data input efficiency of the index system.

[0034] Please refer to Figure 1 The engineering cost data cleaning method based on artificial intelligence provided by the present application comprises the following steps:

[0035] S1. According to the cost file category, identify each tab category, take the file tab name as a parameter, use the decision tree method to perform machine learning training to obtain a tab application model; the cost file category includes estimation file, budget estimation file, budget file, settlement file and final accounting file;

[0036] S2. According to the tab type, identify the file table header: specifically, put the different categories of predefined files into the project category, take the table header as a parameter, set the key variable, and use the decision tree method to perform machine learning training to obtain a table header recognition application model;

[0037] S3. Identify the level and containing relationship of the sub-item related tabs; the relationship identification includes means of text semantics, writing method, font color, number sequence, number case distinction;

[0038] S4. Intelligently encode the level relationship according to the set business regulation code: specifically, the identified file level organization and the predefined label structure are correspondingly associated, and a code is assigned;

[0039] S5, the content of the cross-tab is identified to be nested in the associated relationship, and is returned in the form of structured data; specifically, the results generated in the fourth step are selected according to the tab category, and the tabs that need to be associated with each other are selected; the matching is confirmed according to the data file content key item, and a Json format nested structured data is formed if the matching is successful; the data file content key item includes project name, project specification, engineering quantity, and engineering amount;

[0040] S6, the index attribute of the content in the tab is identified, including the engineering quantity index and the consumption quantity index, specifically, the project name is set as a parameter, the key variable is set, the machine learning training of the decision tree algorithm is performed, and the engineering quantity index and the consumption quantity index application identification model are output;

[0041] S7, the file content recognition result is packaged into formatted data and returned to the application system.

[0042] As shown in Figure 2 An engineering cost data cleaning method based on artificial intelligence, characterized in that the engineering cost data cleaning method based on artificial intelligence is realized based on an engineering cost data cleaning system, and the system comprises:

[0043] Data interface module: used for data exchange with the main application system, accepting request parameters and data files;

[0044] File pre-checking module: after receiving the file, the file is pre-checked to check whether the table tabs are complete or not, whether the necessary content in the tabs is included or not, or whether the necessary content in the tabs can be successfully identified, and if the check fails, the corresponding error code is returned;

[0045] Tab and table header identification module: specifically, according to the pre-defined file project category of different tab categories, the table header is set as a parameter, the key variable is set, and the machine learning training of the decision tree is used to train the table header identification application model;

[0046] Hierarchical structure identification module: intelligent coding label is performed according to the set business regulation code, specifically, the file hierarchical structure that has been identified and the pre-defined label structure are correspondingly associated, and the code is assigned;

[0047] Cross-tab associated relationship identification module: according to the tab category, the tabs that need to be associated with each other are selected, the matching is confirmed according to the data file content key item, and a Json format nested structured data is formed if the matching is successful;

[0048] Supply and consumption index identification module: identify the index attribute containing the engineering quantity index and the consumption index in the tab, specifically, set key variables for decision tree algorithm machine learning training according to the project name as a parameter, and output the engineering quantity index and consumption index application identification model.

[0049] The cost file format is various, and there is no fixed and strict unified standard, even if the result files generated in the same stage of the same project, the file format, content and expression method are different from person to person, which leads to that the traditional cost index system taking the cost result file as the data source faces a huge artificial data arrangement and input work, so that the index system cannot be continuously operated in the actual application because of the input work and work quality problems. The application utilizes artificial intelligence identification technology, can simulate the manual arrangement work process of the cost file, identifies, standardizes and associates the non-standard cost file data, greatly improves the data input work efficiency and work quality, and is a powerful tool for sustainable application of the index system.

[0050] The preferred embodiments disclosed above are only used to help explain the application. The preferred embodiments do not describe all the details, and the application is not limited to the specific embodiments described. Obviously, according to the content of the specification, many modifications and changes can be made. The specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the application, so that those skilled in the art can well understand and utilize the application. The application is limited by the claims and the entire scope and equivalents thereof.

Claims

1. An artificial intelligence-based engineering cost data cleaning method, characterized in that, Comprise the following steps: S1, according to the cost file category identification each tab category, file tab name as parameter, through the key variable, using decision tree method machine learning training tab application model; S2, according to the tab type, identify file table header: specifically, the different categories of predefined file project category, table header as parameter, through the setting key variable, using decision tree method machine learning training table header recognition application model; S3, to the part of the item related tab level and containing relationship identification; S4, to the hierarchical relationship according to the set business rules coding intelligent coding label: specifically, the file level mechanism has been identified and the predefined label structure is associated and coded; S5, the content of the cross tab exists in the relationship is identified and nested, and is returned in the form of structured data; Specifically, the results of the fourth step are selected according to the tab category, the tabs that need to be associated with each other are selected; the data file content key items are matched and confirmed, and a Json format nested structured data is formed if the matching is successful; S6, identify the index attribute of the content in the tab, which includes the engineering quantity index and the consumption index, specifically according to the project name as parameter, setting key variable, machine learning training of decision tree algorithm, output engineering quantity index and consumption index application recognition model; S7, the file content recognition result is packaged into formatted data and returned to the application system.

2. The artificial intelligence-based engineering cost data cleaning method according to claim 1, characterized in that, The cost file category includes estimation file, budget file, budget file, settlement file and final accounts file.

3. The artificial intelligence-based engineering cost data cleaning method according to claim 1, wherein, The relationship identification includes means of text semantics, writing method, font color, number sequence, number case distinction.

4. The artificial intelligence-based engineering cost data cleaning method according to claim 1, wherein, The data file content key items include project name, project specification, engineering quantity, engineering amount.

5. The artificial intelligence-based engineering cost data cleaning method according to any one of claims 1-4, characterized in that, The engineering cost data cleaning method based on artificial intelligence is realized based on an engineering cost data cleaning system, which comprises: Data interface module: used for data exchange with the main application system, accepting request parameters and data files; File pre-checking module: after receiving the file, the file is pre-checked to check whether the table tab is complete or not, whether the necessary content in the tab is contained or not, and whether it can be successfully identified, and the error code is returned if the check fails; Tab and table header identification module: specifically, according to the different tab category predefined file project category, table header as parameter, through the setting key variable, using decision tree method machine learning training table header recognition application model; Hierarchical structure identification module: according to the set business rules coding intelligent coding label, specifically, the file level mechanism has been identified and the predefined label structure is associated and coded; Cross tab associated containing relationship identification module: according to the tab category, the tabs that need to be associated with each other are selected, the data file content key items are matched and confirmed, and a Json format nested structured data is formed if the matching is successful; The supply and consumption index identification module identifies the index attributes containing the engineering quantity index and the consumption index in the tab, specifically, sets key variables for decision tree algorithm machine learning training according to the project name as a parameter, and outputs the engineering quantity index and the consumption index application identification model.

Images