Rail transit file parsing method, system and storage medium based on artificial intelligence

Abstract

The present invention discloses a rail transit file parsing method, system and storage medium based on artificial intelligence, the method comprising: obtaining the cost file of rail transit, and parsing the subject sentence of each cost file based on the subject parsing module in the artificial intelligence model, and classifying each cost file according to the parsed subject sentence to generate multiple cost file groups; performing format conversion on each cost file in the cost file group according to the reference format corresponding to the cost file group; dividing the converted cost file into a header part and a content part, and performing header parsing and content parsing respectively based on the header parsing module and the content parsing module in the artificial intelligence model to generate a standardized header and standardized content; merging the standardized header and the standardized content to generate a standardized cost file. The present invention improves the efficiency and accuracy of cost file parsing by setting an artificial intelligence model to group and parse rail transit cost files.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a rail transit file parsing method, system and storage medium based on artificial intelligence. Background Art

[0002] As economic strength continues to grow, the pace of urbanization is also gradually accelerating. As an important means to solve urban traffic problems, rail transit is also increasingly valued. The construction of rail transit usually involves a large number of manpower, materials and engineering items, and the related costs are also varied, such as labor costs for various types of work, usage fees for various construction tools, various steel, cement, concrete and other material costs, as well as safety construction fees, civilized construction fees, night construction additional fees, etc. All costs must be compiled into cost documents by relevant personnel, and finally summarized into a document representing the total cost of rail transit.

[0003] Although cost documents are usually in the form of tables, different personnel have different habits in compiling tables, which leads to that the formats of cost documents vary greatly and there may be errors in the content. For example, for the same material, different personnel use different names in cost documents, or different units of measurement, or there may be typos. As a result, when summarizing the cost documents of rail transit, relevant personnel must first conduct standardized analysis and processing of various cost documents. The efficiency of analysis and processing is low and errors are prone to occur again. Therefore, how to analyze and process the cost documents in rail transit in a unified, efficient and accurate manner is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the present invention is to provide a rail transit file parsing method, system and storage medium based on artificial intelligence, aiming to solve the technical problem of how to uniformly, efficiently and accurately analyze and process cost documents in rail transit.

[0005] To achieve the above object, the present invention provides a rail transit file parsing method based on artificial intelligence, and the rail transit file parsing based on artificial intelligence includes:

[0006] Obtaining cost documents of rail transit, parsing subject sentences of each cost document based on a subject parsing module in an artificial intelligence model, and classifying each cost document according to the parsed subject sentences to generate a plurality of cost document groups;

[0007] According to the reference format corresponding to each of the cost file groups, format conversion is performed on each of the cost file groups;

[0008] Divide each of the cost documents after format conversion into a header part and a content part, and perform header parsing and content parsing on the divided header part and content part respectively based on the header parsing module and the content parsing module in the artificial intelligence model to generate a standardized header and standardized content;

[0009] The standardized header and the standardized content are combined to generate a standardized cost document.

[0010] Preferably, the step of parsing the subject sentences of each of the cost documents based on the subject parsing module in the artificial intelligence model includes:

[0011] Extracting the subject sentences of each of the cost documents according to the subject analysis module, and performing forward word segmentation and reverse word segmentation on each of the subject sentences to obtain forward phrases and reverse phrases;

[0012] The positive information index between two adjacent positive words in the positive phrase is calculated based on the preset information degree formula in the topic analysis module, and the reverse information degree index between two adjacent reverse words in the reverse phrase is calculated, wherein the preset information degree calculation formula is:

[0013]

[0014] Among them, M (i，i+1) represents the information index between the i-th word and the i+1-th word in the phrase, w (i，i+1) represents the weight between the i-th word and the i+1-th word in the phrase, a i Represents the word vector of the i-th word in the phrase, a i+1 represents the word vector of the i+1th word in the phrase, T represents the transposition operation, and |·| represents the modulo operation;

[0015] The forward information indexes are summed to obtain a first operation result, and the reverse information indexes are summed to obtain a second operation result, and the parsing of the subject sentence is completed based on the size relationship between the first operation result and the second operation result.

[0016] Preferably, the header parsing module and the content parsing module in the artificial intelligence model respectively perform header parsing and content parsing on the divided header part and content part to generate a standardized header and standardized content, comprising:

[0017] Extracting text sentences and header keywords in the header part based on the header parsing module, and parsing each of the text sentences according to a preset sentence library to obtain a standard text sentence corresponding to each of the text sentences;

[0018] Based on the header parsing module, a preset rail transit vocabulary is called to parse each of the header keywords, obtain a standard keyword corresponding to each of the header keywords, and generate the standardized header with each of the standard text sentences and each of the standard keywords;

[0019] Based on the content analysis module, each cell in the content part is analyzed, an abnormal cell among the cells is determined, and the abnormal cell is corrected according to the abnormal type of the abnormal cell to obtain standardized content.

[0020] Preferably, the step of parsing each of the text sentences according to a preset sentence library to obtain a standard text sentence corresponding to each of the text sentences comprises:

[0021] For each of the text sentences, obtain word vectors of the characters contained in the text sentence, and remove connecting words in the text sentence according to each of the word vectors to obtain the text sentence to be parsed;

[0022] Based on the similarity calculation formula in the header parsing module, the similarity between the text sentence to be parsed and each reference text sentence in the preset sentence library is calculated to obtain a text similarity value, wherein the similarity calculation formula is:

[0023]

[0024] Among them, P τ represents the text similarity value between the text sentence to be parsed and the τth reference text sentence in the preset sentence library, n represents the number of characters in the text sentence to be parsed, sim(·) represents the function used to calculate the similarity of the text meaning, Z1 k 、Z1 k+1 Respectively represent the word vectors of the kth and k+1th words in the text sentence to be parsed, Z2 τk 、Z2 τ(k+1) denote the word vectors of the kth and k+1th words in the τth reference text sentence, respectively; γ denotes the importance parameter of word order similarity to word semantic similarity; |·| denotes a modular operation;

[0025] The maximum value among the text similarity values ​​is determined, and based on the reference text sentence corresponding to the maximum value in a preset sentence library, the text sentence is converted into a standard text sentence.

[0026] Preferably, the step of parsing each of the header keywords by calling a preset rail transit vocabulary based on the header parsing module to obtain a standard keyword corresponding to each of the header keywords includes:

[0027] For each of the header keywords, the header parsing module compares the first character in the header keyword with the index words in the preset rail transit vocabulary to determine whether there is a target index word corresponding to the first character in each of the index words;

[0028] If the target index word exists, determining the index keyword set corresponding to the target index word in the preset rail transit vocabulary, and the word frequency weight value and adjustment weight value of each index keyword in the index keyword set;

[0029] Based on the vector distance calculation formula in the header parsing module, the vector distance values ​​between the header keywords and each index keyword in the index keyword set are calculated. The vector distance calculation formula is:

[0030]

[0031] Among them, Q μ represents the vector distance value between the header keyword and the μth index keyword in the index keyword set, m represents the vector dimension, C1 represents the word vector of the header keyword, C2 μ Represents the word vector of the μth index keyword, V1 μ Indicates the frequency weight value of the μth index keyword, V2 μ represents the adjusted weight value of the μth index keyword;

[0032] The minimum value among the vector distance values ​​and the target index keyword corresponding to the minimum value in the index keyword set are determined, and a reference keyword corresponding to the target index keyword is searched as the standard keyword.

[0033] Preferably, the step of parsing each cell in the content part based on the content parsing module and determining an abnormal cell in each cell comprises:

[0034] Parsing each cell of the content part based on the content parsing module, determining a first type of cell lacking a value and a second type of cell having an invalid value in each of the cells;

[0035] For the other cells in the cells except the first type of cells and the second type of cells, based on the row restrictions and column restrictions corresponding to the other cells, determine the third type of cells in the other cells that have abnormal values;

[0036] The first type of cells, the second type of cells and the third type of cells are collectively determined as the abnormal cells.

[0037] Preferably, before the step of obtaining the cost document of rail transit, the step further includes:

[0038] Collecting the specification document of the rail transit, obtaining corpus samples based on the specification document, and dividing the corpus samples into training corpus samples and test corpus samples;

[0039] Based on the training corpus sample, forward training and reverse training are performed on the topic analysis module in the artificial intelligence model, and after the training reaches a preset number of times, a word segmentation test is performed on the topic analysis module based on the test corpus sample to obtain a word segmentation test result;

[0040] The performance parameters of the topic analysis module are calculated according to the word segmentation test results, and the calculation formula is:

[0041]

[0042] Among them, F represents the performance parameter, β represents the number of test corpus samples, and Y 1α represents the αth test result generated by the αth test corpus sample, Y 2α represents the αth reference result corresponding to the αth test corpus sample, X1 and X2 represent the accuracy weight and ambiguity weight respectively, θ(·) represents a statistical function, which is used to count the number of ambiguities in the test result based on the ambiguous reference results in the reference result, and Y1 represents the test result;

[0043] Comparing the performance parameter with a preset performance threshold to determine whether the performance parameter is less than the preset performance threshold, and if so, completing the training of the topic analysis module;

[0044] If the performance parameter is greater than or equal to a preset performance threshold, the topic analysis module is iteratively trained based on the training corpus sample until the performance parameter is less than the preset performance threshold.

[0045] Preferably, the step of merging the standardized header and the standardized content to generate a standardized cost document comprises:

[0046] Determining whether a first arrangement order of each standardized content in the standardized header is the same as a second arrangement order of each header content in the header part;

[0047] If the first arrangement order is the same as the second arrangement order, merging the standardized header and the standardized content to generate the standardized cost file;

[0048] If the first arrangement order is different from the second arrangement order, determining the content to be adjusted in the standardized content and the adjustment order of the content to be adjusted according to the difference between the first arrangement order and the second arrangement order;

[0049] According to the adjustment sequence, the arrangement sequence of the contents to be adjusted in the standardized contents is adjusted, and the adjusted standardized contents are merged with the standardized header to generate the standardized cost file.

[0050] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a rail transit file parsing system based on artificial intelligence, the rail transit file parsing system based on artificial intelligence includes a storage, a processor, a communication bus and a control program stored on the storage:

[0051] The communication bus is used to realize the connection and communication between the processor and the storage;

[0052] The processor is used to execute the control program to implement the steps of the rail transit file parsing method based on artificial intelligence as described above.

[0053] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a storage medium, on which a control program is stored, and when the control program is executed by a processor, the steps of the rail transit file parsing method based on artificial intelligence as described above are implemented.

[0054] The present invention is based on artificial intelligence rail transit file parsing method, system and storage medium, which are pre-set with artificial intelligence model. After obtaining the cost file of rail transit, the subject sentence of each cost file is first parsed by the subject parsing module in the artificial intelligence model, and then the cost file is divided into multiple cost file groups according to the parsed subject sentence; and for each cost file group, a respective reference format is pre-set, and the format conversion is performed on each cost file in the cost file group according to the reference format, and the cost file after format conversion is divided into a header part and a content part, and then according to the header parsing module and the content parsing module in the artificial intelligence model, the divided header part and the content part are respectively subjected to header parsing and content parsing, and a standardized header and standardized content are generated; and according to the corresponding relationship between the divided header part and the content part, the standardized header and the standardized content are merged to generate a parsed standardized cost file. In this way, by setting the artificial intelligence model, the same type of rail transit cost files are divided into the same cost file group for unified format conversion, header parsing and content parsing, thereby improving the efficiency and accuracy of rail transit cost file parsing. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1This is a flow chart of a first embodiment of a rail transit file parsing method based on artificial intelligence according to the present invention;

[0056] Figure 2 This is a flow chart of a second embodiment of a rail transit file parsing method based on artificial intelligence according to the present invention;

[0057] Figure 3 This is a flow chart of a third embodiment of a rail transit file parsing method based on artificial intelligence according to the present invention;

[0058] Figure 4 It is a structural schematic diagram of the hardware operating environment involved in an embodiment of the rail transit file parsing system based on artificial intelligence of the present invention.

[0059] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0061] The present invention provides a rail transit file parsing method based on artificial intelligence, please refer to Figure 1 , Figure 1 It is a flowchart of the first embodiment of the rail transit file parsing method based on artificial intelligence of the present invention.

[0062] The embodiment of the present invention provides an embodiment of a rail transit file parsing method based on artificial intelligence. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here. Specifically, the rail transit file parsing method based on artificial intelligence in this embodiment includes:

[0063] Step S10, obtaining the cost documents of rail transit, and parsing the subject sentences of each cost document based on the subject parsing module in the artificial intelligence model, and classifying each cost document according to the parsed subject sentences to generate multiple cost document groups.

[0064] The rail transit file parsing method based on artificial intelligence in this embodiment is applied to the backend server, and various cost files of rail transit are parsed by the backend server, and various cost files of rail transit are converted into cost files with standardized formats, headers and data contents. Among them, the cost file of rail transit can be a word file or an EXCEL file. This embodiment is preferably illustrated with an EXCEL file, which includes a file subject, a file header and specific content. The backend server is pre-set with an artificial intelligence model that can parse the EXCEL file, and the artificial intelligence model includes a theme parsing module that can parse the file subject. After obtaining the cost file of rail transit, the theme sentence of each cost file is identified by the theme parsing module, and the theme sentence is parsed. By parsing the theme keywords representing the specific content contained in the cost file from the theme sentence, each cost file is classified, and the cost files with uniform content are divided into a group to form multiple cost file groups. For example, for the cost document of "No. 3 Air Outlet of XX City Rail Transit Line 2 Project", the keyword "air outlet" can be parsed out, and then it can be classified into the same cost document group with other cost documents of air outlets. Specifically, the step of parsing the subject sentences of each cost document based on the subject parsing module in the artificial intelligence model includes:

[0065] Step S11, extracting the subject sentences of each of the cost documents according to the subject analysis module, and performing forward word segmentation and reverse word segmentation on each of the subject sentences to obtain forward phrases and reverse phrases;

[0066] Step S12, calculating the positive information index between two adjacent positive words in the positive phrase based on the preset information formula in the topic analysis module, and calculating the reverse information index between two adjacent reverse words in the reverse phrase;

[0067] Step S13, summing up the positive information indexes to obtain a first calculation result, and summing up the reverse information indexes to obtain a second calculation result, and completing the parsing of the subject sentence according to the size relationship between the first calculation result and the second calculation result.

[0068] Furthermore, the name of the cost document can be identified through the subject parsing module to extract the subject sentence of the cost document, and then the extracted subject sentence is processed by forward segmentation and reverse segmentation to obtain positive phrases and reverse phrases. Among them, the forward segmentation is to segment the subject sentence in the order from left to right, such as the above-mentioned "No. 3 Air Outlet of XX City Rail Transit Line 2 Project", and the forward segmentation from left to right is to segment the subject sentence in the order from "XX City" to "Air Outlet". The reverse segmentation is to segment the subject sentence in the order from right to left, such as the above-mentioned "No. 3 Air Outlet of XX City Rail Transit Line 2 Project", and the reverse segmentation from right to left is to segment the subject sentence in the order from "Air Outlet" to "XX City". And both forward and reverse word segmentation are set with a maximum word length, for example, set to 5. For forward word segmentation, the first 5 words are extracted from left to right, and it is determined whether the extracted 5 words exist in the preset forward dictionary. If they exist in the preset forward dictionary, the first 5 words are separated from the sentence as a word segmentation. If they do not exist in the preset forward dictionary, the last word extracted, that is, the first 4 words are retained to determine whether they exist in the preset forward dictionary. This cycle is repeated until the word composed of the extracted words is in the preset forward dictionary. Reverse word segmentation can also be performed in this way after setting the maximum word length, which will not be repeated here.

[0069] Furthermore, in order to ensure the accuracy of the forward word segmentation and the reverse word segmentation, a preset information degree formula is pre-set in the topic analysis module, and the preset information degree formula is used to calculate the positive information degree index between two adjacent positive words in the forward phrase, and the reverse information degree index between two adjacent reverse words in the reverse phrase is calculated. The positive information degree index between two adjacent positive words represents the word meaning information correlation between the two adjacent positive words, and the information correlation reflects that the two positive words are ambiguous words. For the positive words with accurate division, the weaker the word meaning correlation between the two words, the less ambiguous the two words are. On the contrary, if the word meaning correlation between the two words is stronger, the two words are ambiguous words, and the word segmentation may be inaccurate. For example, the above-mentioned "air outlet" may be divided into "out", "air outlet", or it may be divided into "outlet", "mouth", and of course it may be accurately divided into "air outlet". Therefore, in order to reflect the accuracy, the positive information degree index between the two adjacent positive words needs to be calculated. The reverse information index between two adjacent reverse words also reflects the relevance of word meaning information. It is necessary to calculate the reverse information index between two adjacent reverse words in the reverse phrase. The preset information calculation formula can be specifically referred to in the following formula (1).

[0070]

[0071] Among them, M (i，i+1) represents the information index between the i-th word and the i+1-th word in the phrase, w (i，i+1)represents the weight between the i-th word and the i+1-th word in the phrase, a i Represents the word vector of the i-th word in the phrase, a i+1 represents the word vector of the i+1th word in the phrase, T represents the transposition operation, and |·| represents the modulo operation. For the positive phrase, M (i，i+1) represents the positive information index calculated based on the word vectors and weight values ​​of the i-th positive word and the i+1-th positive word in the positive phrase. For the reverse phrase, M (i，i+1) It represents the reverse information index calculated based on the word vectors and weight values ​​of the i-th reverse word and the i+1-th reverse word in the reverse phrase. In addition, the weight value is generated according to the probability of incorrect classification of ambiguous words during the word segmentation training of the topic analysis module. The greater the probability of error, the greater the weight value, and the greater the value of the information index, indicating that the two adjacent words are ambiguous words and may be inaccurately classified.

[0072] Further, after each adjacent positive word in the forward phrase is calculated to obtain its own positive information index, each positive information index is summed to obtain a first operation result corresponding to the forward phrase. At the same time, after each adjacent reverse word in the reverse phrase is calculated to obtain its own reverse information index, each reverse information index is summed to obtain a second operation result corresponding to the reverse phrase. Then, a comparison is made between the first operation result and the second operation result to determine the size relationship between the two. In the case where both the forward participle and the reverse participle are accurate, the first operation result and the second operation result should be the same, that is, equal in size. At this time, any one of the forward phrase and the reverse phrase is selected as the target phrase, and the subject keyword is identified from the target phrase to complete the parsing of the subject sentence. If the size relationship between the first operation result and the second operation result is determined by comparison to be that the first operation result is greater than the second operation result, it means that the accuracy of the forward participle is lower than that of the reverse participle. At this time, the reverse phrase is used as the target phrase, and the subject keyword is identified from it to complete the parsing of the subject sentence. On the contrary, if the comparison determines that the size relationship between the first operation result and the second operation result is that the first operation result is smaller than the second operation result, it means that the accuracy of forward word segmentation is higher than that of reverse word segmentation. At this time, the forward phrase is used as the target phrase, and the topic keywords are identified from it to complete the analysis of the topic sentence.

[0073] Step S20: converting the format of each cost file in the cost file group according to the reference format corresponding to each cost file group.

[0074] Furthermore, for each type of cost document group, a reference format for standardizing its format is pre-set, such as font, line spacing, line height, etc. Therefore, according to the reference format corresponding to each type of cost document group, the format of each cost document contained in this type of cost document group is converted. Specifically, for each cost document, it is identified whether its font is consistent with the font in the reference format; if it is consistent, there is no need to convert the font of the cost document; otherwise, if it is inconsistent, the font of the cost document needs to be converted to the font in the reference format. Then, it is determined whether the line spacing and line height of the cells in the cost document are consistent with the line spacing and line height in the reference format; if it is consistent, there is no need to adjust the line spacing and line height of the cells in the cost document; otherwise, if it is inconsistent, the line spacing and line height in the cost document need to be adjusted to be consistent with the line spacing and line height in the reference format. By converting the format of each cost document in the cost document group, each cost document has the same standardized format.

[0075] Step S30, divide each of the cost documents after format conversion into a header part and a content part, and based on the header parsing module and content parsing module in the artificial intelligence model, perform header parsing and content parsing on the divided header part and content part respectively to generate a standardized header and standardized content.

[0076] Furthermore, each cost file after format conversion is split into a header part and a content part. The header part indicates the meaning of each data in the cost file, such as unit price, total price, etc., while the content part indicates the specific values ​​corresponding to each meaning. Moreover, the header part can be divided into only column headers and both column headers and row headers according to the differences in cost files. Only containing column headers means that the data of each cell in the cost file only has the meaning of the column header. Containing both column headers and row headers means that the data of each cell in the cost file includes both the meaning of the column header and the meaning of the row header. For example, if the column header is the unit price and the row header is the material model, the data of the cell represents the unit price of a certain model of material.

[0077] Furthermore, the artificial intelligence model also includes a header parsing module for parsing the header part, and a content parsing module for parsing the content part. After the cost document is divided into a header part and a content part, the header parsing module can be used to parse the divided header part to generate a standardized header, and the content parsing module can be used to parse the divided content part to generate standardized content. Among them, because the header part mainly contains various meaning concepts in rail transit, the parsing of the header part is to analyze whether the various meaning concepts belong to standardized concept names, and to correct non-standardized concepts; at the same time, the meanings represented by each cell in the header are arranged in a standardized order, including arranging the first cell in the same row as the material model, the second cell as the material quantity, and the third cell as the material unit price. In this way, a standardized header that meets the specifications in terms of concept name and arrangement order is obtained. For the parsing of the content part, it is mainly to analyze whether there are non-standard values ​​in the content part, and to correct the non-standard values ​​to obtain a standardized header.

[0078] Step S40, merging the standardized header and the standardized content to generate a standardized cost document.

[0079] Furthermore, after parsing the divided header part and content part to obtain the standardized header and standardized content, the meaning of each cell in the standardized header may change relative to the original header. Therefore, in order to keep the correspondence between the standardized header and the standardized content unchanged, it is necessary to merge the standardized header and the standardized content according to the correspondence between the header part and the content part during the division, and form the parsed standardized cost file by merging. Specifically, the step of merging the standardized header and the standardized content to generate the standardized cost file includes:

[0080] Step S41, determining whether the first arrangement order of each standardized content in the standardized header is the same as the second arrangement order of each header content in the header part;

[0081] Step S42, if the first arrangement order is the same as the second arrangement order, merging the standardized header and the standardized content to generate the standardized cost file;

[0082] Step S43, if the first arrangement order is different from the second arrangement order, determining the content to be adjusted in the standardized content and the adjustment order of the content to be adjusted according to the difference between the first arrangement order and the second arrangement order;

[0083] Step S44, adjusting the arrangement order of the contents to be adjusted in the standardized contents according to the adjustment order, and merging the adjusted standardized contents with the standardized header to generate the standardized cost file.

[0084] Furthermore, the meaning of each cell in the standardized header is used as the standardized content, and it is determined whether the first arrangement order of each standardized content is the same as the second arrangement order of each original header content of the header part. If they are the same, it means that the parsing of the header part has not adjusted the meaning of each cell, and the corresponding relationship between the standardized header and the standardized content has not changed compared to before the parsing, so the standardized header and the standardized content can be directly merged to generate a standardized cost file.

[0085] Further, if it is determined that the first arrangement order of each standardized content in the standardized header is different from the second arrangement order of each header content in the header part, it means that when parsing the header part, the header contents of the header part are adjusted. At this time, it is necessary to determine the content to be adjusted in the standardized content and the adjustment order of each content to be adjusted based on the difference between the second arrangement order of each header content before the adjustment and the first arrangement order of each standardized content after the adjustment. For example, the first arrangement order is that the first cell in the same row means the material model, the second cell means the material quantity, and the third cell means the material unit price; the second arrangement order is that the first cell in the same row means the material model, the second cell means the material unit price, and the third cell means the material quantity. Therefore, according to the difference between the first arrangement order and the second arrangement order, it is determined that the content to be adjusted in the standardized content is the second column data and the third column data, and the relevant adjustment order is the swapping of the second column data and the third column data. Then, according to the adjustment order of the content to be adjusted in the standardized content, the arrangement order of each content to be adjusted is adjusted, and after the adjustment, the standardized content and the standardized header are merged to obtain a standardized cost document.

[0086] The rail transit file parsing method based on artificial intelligence of this implementation is pre-set with an artificial intelligence model. After obtaining the cost file of rail transit, the subject sentence of each cost file is first parsed by the subject parsing module in the artificial intelligence model, and then the cost files are divided into multiple cost file groups according to the parsed subject sentence; and for each cost file group, a respective reference format is pre-set, and the format conversion is performed on each cost file in the cost file group according to the reference format, and the cost file after format conversion is divided into a header part and a content part, and then the header parsing module and the content parsing module in the artificial intelligence model are used to perform header parsing and content parsing on the divided header part and content part respectively, and a standardized header and standardized content are generated; and according to the corresponding relationship between the divided header part and the content part, the standardized header and the standardized content are merged to generate a parsed standardized cost file. In this way, by setting the artificial intelligence model, the same type of rail transit cost files are divided into the same cost file group for unified format conversion, header parsing and content parsing, thereby improving the efficiency and accuracy of rail transit cost file parsing.

[0087] For further information, please refer to Figure 2 Based on the first embodiment of the rail transit file parsing method based on artificial intelligence of the present invention, a second embodiment of the rail transit file parsing method based on artificial intelligence of the present invention is proposed.

[0088] The difference between the second embodiment of the rail transit file parsing method based on artificial intelligence and the first embodiment of the rail transit file parsing method based on artificial intelligence is that the header parsing module and the content parsing module in the artificial intelligence model respectively perform header parsing and content parsing on the divided header part and content part, and the steps of generating a standardized header and standardized content include:

[0089] Step S31, extracting text sentences and header keywords in the header part based on the header parsing module, and parsing each of the text sentences according to a preset sentence library to obtain a standard text sentence corresponding to each of the text sentences;

[0090] Further, the header part of the cost document usually contains keywords with clear meanings, such as "cement mortar", "reinforced concrete", etc., and relatively short sentences, such as "Cement mortar plastering for Air Duct No. 8". The keywords and short sentences in the header document are respectively used as header keywords and text sentences, and are extracted through the header parsing module. At the same time, a preset sentence library is pre-set, and the preset sentence library contains a large number of reference text sentences with standardized expressions related to the cost of rail transit. The header parsing module parses each of the extracted text sentences according to the preset sentence library to obtain a standard paragraph text sentence corresponding to each text sentence, and the standard text sentence is a text sentence that expresses the same meaning as the text sentence and conforms to the unified specification. Specifically, the step of parsing each of the text sentences according to the preset sentence library to obtain a standard text sentence corresponding to each of the text sentences includes:

[0091] Step S311, for each of the text sentences, obtain the word vectors of the words included in the text sentence, and according to each of the word vectors,剔除 the connecting words in the text sentence to obtain the text sentence to be parsed;

[0092] Step S312, based on the similarity calculation formula in the header parsing module, calculate the similarity between the text sentence to be parsed and each reference text sentence in the preset sentence library to obtain a text similarity value;

[0093] Step S313, determine the maximum value among each of the text similarity values, and based on the reference text sentence corresponding to the maximum value in the preset sentence library, convert the text sentence into a standard text sentence.

[0094] Furthermore, perform character segmentation on each text sentence to obtain each word included in the text sentence, and then obtain the word vectors of each word. According to each word vector, identify the connecting words in the text sentence and剔除 the identified connecting words. The remaining words after剔除 form the text sentence to be parsed in the original order. Among them, the connecting words are words that have nothing to do with the cost of rail transit, such as "的", "地", "和", "与", etc. The word vectors of such connecting words are pre-set as a word vector library. After obtaining the word vectors of each word in the text sentence, compare each of the obtained word vectors with the preset word vector library, screen out the target word vectors existing in the word vector library among each of the word vectors, and剔除 the word from which the target word vector is derived from the text sentence. The text sentence after剔除 forms the text sentence to be parsed.

[0095] It should be noted that the Chinese character "剔除" in the original text is directly translated as "剔除" in the English text as there is no exact equivalent word in English. If a more appropriate English expression is required, it can be adjusted according to the specific context. Also, the tags ,

[0091] , etc. are preserved as they are as per the requirements.Furthermore, a similarity calculation formula is pre-set in the header parsing module, and the similarity between the text sentence to be parsed and each reference text sentence included in the preset sentence library is calculated by the similarity calculation formula to obtain the text similarity value between the text sentence to be parsed and each reference text sentence.

[0096] The similarity calculation formula can be specifically referred to as the following formula (2).

[0097]

[0098] Among them, P τ represents the text similarity value between the text sentence to be parsed and the τth reference text sentence in the preset sentence library, n represents the number of characters in the text sentence to be parsed, sim(·) represents the function used to calculate the similarity of the text meaning, Z1 k 、Z1 k+1 Respectively represent the word vectors of the kth and k+1th words in the text sentence to be parsed, Z2 τk 、Z2 τ(k+1) They represent the word vectors of the kth and k+1th words in the τth reference text sentence respectively, γ represents the importance parameter of word order similarity to word semantic similarity, and |·| represents the modulo operation.

[0099] Considering that the text sentences to be parsed that express the same meaning may not only differ in the specific words used, but also in the number of words and the order of words. For example, for the reference text sentence "air duct reinforced concrete", the text sentence to be parsed generated by a certain cost document is "air shaft reinforced concrete", while the text sentence to be parsed generated by another cost document is "air duct air shaft reinforced concrete". Therefore, in order to avoid the inaccuracy caused by the difference in the number of words and the order of words in the similarity calculation, a parameter representing the importance of sequence similarity to semantic similarity is set in the similarity calculation formula. The importance parameter can be determined in advance through experiments on the importance of word order and word meaning in the text sentence. The difference between the moduli of two adjacent word vectors is corrected by the importance parameter to reduce the influence of the similarity of the number of words and the order of words in the sentence, so that the calculation of the similarity value is more accurate.

[0100] Furthermore, after calculating and obtaining the text similarity values ​​between the text sentence to be parsed and each reference text sentence in the preset sentence library, a comparison is performed between each text similarity value to determine the maximum value. The reference text sentence that generates the maximum value is the standardized sentence corresponding to the text sentence to be parsed, so as to find the reference text sentence corresponding to the maximum value in the preset sentence library, and convert the text sentence into a standard text sentence based on the reference text sentence found. Among them, the conversion method can be to directly replace the text sentence with the reference text sentence found, or to first determine the difference words between the text sentence to be parsed and the reference text sentence found, and replace the difference words in the text sentence to be parsed with the difference words in the reference text sentence, and then use the replaced text sentence to be parsed as the standard text sentence.

[0101] Step S32, based on the header parsing module, calling the preset rail transit vocabulary to parse each of the header keywords, obtaining the standard keywords corresponding to each of the header keywords, and generating the standardized header with each of the standard text sentences and each of the standard keywords;

[0102] Furthermore, for the header keywords in the header part, a preset rail transit vocabulary is pre-set, and the preset rail transit vocabulary contains keywords with expression specifications related to rail transit cost. The header parsing module calls the preset rail transit vocabulary to parse each header keyword, and obtains a standard keyword corresponding to each header keyword. The standard keyword is a keyword that expresses the same meaning as the header keyword and conforms to a unified specification. Specifically, the step of parsing each header keyword based on the header parsing module calling the preset rail transit vocabulary to obtain a standard keyword corresponding to each header keyword includes:

[0103] Step S321, for each of the header keywords, based on the header parsing module, compare the first character in the header keyword with the index words in the preset rail transit vocabulary, and determine whether there is a target index word corresponding to the first character in each of the index words;

[0104] Step S322: if the target index word exists, determining the index keyword set corresponding to the target index word in the preset rail transit vocabulary, and the word frequency weight value and adjustment weight value of each index keyword in the index keyword set;

[0105] Step S323, calculating the vector distance values ​​between the header keyword and each index keyword in the index keyword set based on the vector distance calculation formula in the header parsing module;

[0106] Step S324, determining the minimum value among the vector distance values ​​and the target index keyword corresponding to the minimum value in the index keyword set, and searching for a reference keyword corresponding to the target index keyword as the standard keyword.

[0107] Furthermore, in order to facilitate the rapid search for each keyword in the preset rail transit vocabulary, an index word is set for each keyword. Preferably, the first word is set as the index word, all keywords with the same first word in each keyword are searched, and the first word is set as the index word for all the keywords found, and all the keywords found form an index keyword set under the index word. For each header keyword, the first word is extracted through the header loading and unloading module, and the extracted first word is compared with each index word in the preset rail transit vocabulary to determine whether there is a target index word matching the first word in each index word. Among them, the comparison matching can be achieved by word vector comparison, that is, the word vector of the first word is compared with the word vector of each index word, and it is determined whether there is a word vector in the word vector of each index word that is the same as the word vector of the first word. If there is an identical word vector, it is determined that there is a matching target index word. Otherwise, there is no matching target index word.

[0108] Furthermore, if it is determined through comparison that there is a target index word that matches the first character, then the keyword set corresponding to the target index word is searched as the index keyword set corresponding to the target index word in the preset rail transit vocabulary, and each keyword in the keyword set constitutes the index keyword of the index keyword set accordingly. At the same time, the word frequency weight and adjustment weight of each keyword in the keyword set corresponding to the target index word are searched as the word frequency weight value and adjustment weight value of each index keyword in the index keyword set. Among them, the word frequency weight of a certain keyword is determined by counting the frequency of the keyword appearing in various normative documents of rail transit, reflecting the importance of the standardization of the keyword to various documents of rail transit. The higher the frequency of the keyword, the greater the word frequency weight generated, and the higher the importance of the standardized use of the keyword to the rail transit document. The adjustment weight of a certain keyword is determined by the number of adjustments of the keyword in the previous rail transit cost documents, reflecting the probability of the non-standard use of the keyword. The more times the keyword is adjusted, the greater the adjustment weight generated, indicating that the probability of the non-standard use of the keyword is higher.

[0109] Furthermore, a vector distance calculation formula is pre-set in the header parsing module to calculate the distance between the word vectors of keywords. The closer the distance, the higher the degree of proximity of the keywords. Conversely, the lower the degree of proximity. The vector distance calculation formula is used to calculate the vector distance values ​​between the header keyword and each index keyword in the index keyword set. The vector distance calculation formula can be specifically referred to in the following formula (3).

[0110]

[0111] Among them, Q μ represents the vector distance value between the header keyword and the μth index keyword in the index keyword set, m represents the vector dimension, C1 represents the word vector of the header keyword, C2 μ Represents the word vector of the μth index keyword, V1 μ Indicates the frequency weight value of the μth index keyword, V2 μ Represents the adjusted weight value of the μth index keyword. The word vector of the index keyword is modified by the word frequency weight value and the adjusted weight value corresponding to the index keyword, so that the vector distance value calculated between the header keyword and the index keyword better reflects the importance and possibility of the header keyword according to its adjustment.

[0112] Furthermore, after calculating the vector distance values ​​between the header keyword and each index keyword in the index keyword set, a comparison is performed between each vector distance value to determine the minimum value. The index keyword that generates the minimum value is the normalized keyword corresponding to the header keyword, so as to find the index keyword corresponding to the minimum value, and use the found index keyword as the target index keyword. Moreover, in addition to setting the index word and the index keyword set corresponding to the index word in the preset rail transit vocabulary, the index keyword set is also set to include multiple index keyword groups, and the same index keyword group corresponds to the same meaning. That is, various types of synonyms and the standardized standard words corresponding to such synonyms are collectively formed into an index keyword group. After determining the target index keyword, the standard word in the index keyword group where the target index keyword is located can be found. The standard word is the normalized reference keyword corresponding to the target index keyword, and it is used as the standard keyword.

[0113] It should be noted that if it is determined through comparison that the target index word corresponding to the first character in the header keyword does not exist in the index words of the preset rail transit vocabulary, then the header keyword can be marked so that relevant personnel can manually process the header keyword after checking the mark. After all the header keywords in the header part are processed and converted into standard keywords, and all the text sentences in the header part are converted into standard text sentences, each standard text sentence and each standard keyword can be formed into a standardized header.

[0114] Step S33, analyzing each cell in the content part based on the content analysis module, determining abnormal cells among the cells, and correcting the abnormal cells according to the abnormal types of the abnormal cells to obtain standardized content.

[0115] Furthermore, for the content part divided by the cost document, each cell therein is parsed according to the content parsing module, and it is determined whether there are abnormal cells therein. Abnormal cells include but are not limited to cells with missing values, cells with invalid values, cells with incorrect values, etc. If it is determined that there are abnormal cells therein, corrections are made through the content parsing module to obtain standardized content. Specifically, the steps of parsing each cell in the content part based on the content parsing module and determining abnormal cells in each of the cells include:

[0116] Step S331, analyzing each cell of the content part based on the content analysis module, determining a first type of cell lacking a value and a second type of cell having an invalid value in each of the cells;

[0117] Step S332, performing numerical analysis on the other cells in the cells except the first type of cells and the second type of cells based on the row restrictions and column restrictions corresponding to the other cells, and determining that there are third type of cells with abnormal values ​​in the other cells;

[0118] Step S333: collectively determine the first-category cells, the second-category cells, and the third-category cells as the abnormal cells.

[0119] Furthermore, cells lacking numerical values ​​are regarded as first-class cells, and cells with invalid numerical values ​​are regarded as second-class cells. The content parsing module is used to parse each cell of the content part, and analyze whether there is a numerical value in each cell, and whether the numerical value is a valid numerical value. If the content parsing module parses and determines that a certain cell lacks a numerical value, the cell is identified as a first-class cell. Then, other cells with numerical values ​​are identified to determine whether there is an invalid numerical value in a non-numerical form in the numerical value of each cell. If there is an invalid numerical value in a non-numerical form, the cell where the invalid numerical value is located is determined as a second-class cell. Then, for other cells except the first-class cells and the second-class cells, numerical values ​​are parsed based on the row restrictions of the respective rows and the column restrictions of the respective columns, and it is determined whether there is a third-class cell with abnormal numerical values ​​in each other cell. Among them, the column restrictions and row restrictions of other cells are different according to the different meaning concepts of the rows and columns of other cells. For example, the meaning of the row of a certain cell is unit price, and the meaning of the column is concrete, then the corresponding row restriction and column restriction are the amount of money in a certain range of concrete. By analyzing whether the value of the cell is within the fee range, it is determined whether the cell is a third-category cell with an abnormal value.

[0120] Furthermore, after analyzing all other cells and determining the third type of cells that exist therein, the first type of cells, the second type of cells, and the third type of cells are collectively determined as abnormal cells. If it is determined that the third type of cells do not exist in other cells, the first type of cells and the second type of cells are collectively determined as abnormal cells. Similarly, if the second type of cells do not exist, the first type of cells are determined as abnormal cells, and if the first type of cells do not exist, it is determined that there are no abnormal cells in each cell.

[0121] Furthermore, for the existing abnormal cells, corrections can be made according to the abnormal types of the abnormal cells. Corresponding to the first, second, and third types of cells, the abnormal types of abnormal cells include missing value types, invalid value types, and abnormal value types. The same correction method can be set for each abnormal type, such as adding the same or different abnormal identifiers; different correction methods can also be set, such as setting a default value for the missing type, adding a representation for the invalid value type, taking the maximum value in the numerical limit for the abnormal value type, etc. By correcting the abnormal cells, standardized content is obtained. Among them, the content with the added identifier can be processed manually later, that is, the relevant personnel correct the data in the identified cell after checking the identifier.

[0122] This embodiment sets a header parsing module and a content parsing module in the artificial intelligence model, and performs differentiated parsing based on the differences in the information in the header part and the content part, so that the parsing of the header part and the content part is more targeted and the parsing accuracy is higher. At the same time, the header part is further divided into text sentences and header keywords for parsing. The text sentences are parsed by considering the number of words and the order of words, and the header keywords are parsed by combining the word frequency weight and the adjustment weight, which further improves the accuracy of the header parsing. Therefore, accurate parsing of the header part and the content part is achieved, making the generated standardized header and standardized content more accurate.

[0123] For further information, please refer to Figure 3 Based on the first and second embodiments of the rail transit file parsing method based on artificial intelligence of the present invention, a third embodiment of the rail transit file parsing method based on artificial intelligence of the present invention is proposed.

[0124] The third embodiment of the method for parsing rail transit documents based on artificial intelligence is different from the first and second embodiments of the method for parsing rail transit documents based on artificial intelligence in that, before the step of obtaining the cost document of rail transit, the method further includes:

[0125] Step S50, collecting the standard document of the rail transit, obtaining a corpus sample based on the standard document, and dividing the corpus sample into a training corpus sample and a test corpus sample;

[0126] Step S60, performing forward training and reverse training on the topic analysis module in the artificial intelligence model based on the training corpus sample, and after the training reaches a preset number of times, performing a word segmentation test on the topic analysis module based on the test corpus sample to obtain a word segmentation test result;

[0127] Step S70, calculating the performance parameters of the topic analysis module according to the word segmentation test result;

[0128] Step S80, comparing the performance parameter with a preset performance threshold to determine whether the performance parameter is less than the preset performance threshold. If the performance parameter is less than the preset performance threshold, the training of the topic analysis module is completed;

[0129] Step S90: If the performance parameter is greater than or equal to a preset performance threshold, the topic analysis module is iteratively trained based on the training corpus sample until the performance parameter is less than the preset performance threshold.

[0130] Furthermore, in order for the topic parsing module in the artificial intelligence model to parse the topic sentences in the cost document more accurately, the topic parsing module needs to be iteratively trained in advance. Specifically, a large number of normative documents on rail transit are collected, and short text sentences are obtained from each normative document as corpus samples. The corpus sample can be obtained by manual annotation or by neural network model recognition. The obtained corpus samples are then divided into training corpus samples and test corpus samples according to a preset ratio. The preset ratio can be 8 to 2, 7 to 3, or 9 to 1. There is no restriction on this, but the proportion of training corpus samples must be higher than that of test corpus samples to ensure the accuracy of iterative training.

[0131] Furthermore, the topic parsing module in the artificial intelligence model is forward trained and reversely trained through the training corpus samples, that is, the topic parsing model performs forward word segmentation and reverse word segmentation on the training corpus samples. In addition, a preset number of times is also pre-set. When the training number of forward training and the training number of reverse training reach the preset number of times, the topic parsing module is tested for word segmentation through the test corpus samples. The topic parsing model performs forward word segmentation and reverse word segmentation on the test corpus samples, and obtains the forward word segmentation test result and the reverse word segmentation test result, which are used together as the word segmentation test result.

[0132] Furthermore, in order to determine the accuracy of the word segmentation of the topic analysis module, a calculation formula for calculating the performance parameters of the topic analysis module is pre-set, and the word segmentation test results are calculated by the calculation formula to obtain the performance parameters representing the performance of the topic analysis module after training. The calculation formula can be specifically referred to as the following formula (4).

[0133]

[0134] Among them, F represents the performance parameter, β represents the number of test corpus samples, and Y 1α represents the αth test result generated by the αth test corpus sample, Y 2α represents the αth reference result corresponding to the αth test corpus sample, X1 and X2 represent the accuracy weight and the ambiguity weight respectively, θ(·) represents a statistical function, which is used to count the number of ambiguities in the test result based on the ambiguous reference results in the reference result, and Y1 represents the test result.

[0135] Usually, there are words in a text sentence that can be divided in different ways. The middle words in this type of words can be divided into one word together with the words before it, or can be divided into one word together with the words after it. For example, for word ABC, it can be divided into words AB and word C, or it can be divided into words A and word BC, such as "air outlet". The words in the test corpus sample that can be divided in different ways are used as ambiguous test samples. Each corpus sample in the test corpus sample is set with its own corresponding word segmentation result, which is the reference result corresponding to the test corpus sample. The word segmentation result set for the ambiguous test sample is used as the ambiguous reference result in the reference result, and the topic analysis module performs word segmentation on the ambiguous test sample, and the generated word segmentation result is used as the ambiguous test result in the test result.

[0136] Furthermore, in order to reflect the accuracy of the division of such words during the training process of the topic resolution module, an accuracy weight representing the overall word segmentation accuracy and an ambiguity weight representing the accuracy of the ambiguous word segmentation are set in formula (4). The difference between the test result generated by the test corpus sample test and the reference result actually possessed by the test corpus sample is corrected by the accuracy weight, which reflects the accuracy performance of the topic resolution module in the word segmentation training of the test corpus sample. At the same time, by setting a statistical function in formula (4), the statistical function counts the number of ambiguities in the test result based on the ambiguous reference result in the reference result, that is, each ambiguous test result generated by the ambiguous test sample test is compared with the corresponding ambiguous reference result, and the ambiguous test result in the ambiguous test result that does not match its ambiguous reference result is determined, and the number of ambiguous test results that do not match is counted to obtain the number of ambiguities in the test result. Then, the ambiguity weight is used to weight the statistical number of ambiguities, which reflects the accuracy of the topic resolution module in parsing the ambiguous test sample in the test corpus sample.

[0137] Furthermore, in order to reflect the performance of the topic analysis module after training, a preset performance threshold is set in advance, and the performance parameter calculated by formula (4) is compared with the preset performance threshold to determine whether the performance parameter is less than the preset performance threshold. If it is less than the preset performance threshold, it means that the difference between the test result obtained by the topic analysis module for analyzing the test corpus sample and the reference result is small, the analysis performance of the topic analysis module is good, and it can be used for the analysis of actual topic sentences, so the training of the topic analysis module is completed. On the contrary, if it is determined by comparison that the performance parameter is greater than or equal to the preset performance threshold, it means that the difference between the test result obtained by the topic analysis module for analyzing the test corpus sample and the reference result is large, the analysis performance of the topic analysis module is poor, and it is necessary to iteratively train the topic analysis module based on the training corpus sample, and re-test the test corpus sample to generate performance parameters and compare them with the preset performance threshold until the generated performance parameter is less than the preset performance threshold, and then the training of the topic analysis module is completed.

[0138] This embodiment iteratively trains the topic analysis module by setting accuracy weights and ambiguity weights, so that the topic analysis module can not only accurately segment sentences that do not contain ambiguous words, but also accurately segment sentences that contain ambiguous words, thereby improving the accuracy of the topic analysis module in parsing topic sentences.

[0139] In addition, the embodiment of the present invention also provides a rail transit file parsing system based on artificial intelligence. Figure 4 , Figure 4 It is a structural schematic diagram of the equipment hardware operating environment involved in the implementation scheme of the rail transit file parsing system based on artificial intelligence of the present invention.

[0140] like Figure 4 As shown, the rail transit file parsing system based on artificial intelligence may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a storage 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The storage 1005 may be a high-speed RAM storage, or it may be a stable storage (non-volatile memory), such as a disk storage. The storage 1005 may also be a storage device independent of the aforementioned processor 1001.

[0141] Those skilled in the art will understand that Figure 4The hardware structure of the artificial intelligence-based rail transit file parsing system shown in the figure does not constitute a limitation of the artificial intelligence-based rail transit file parsing system, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0142] like Figure 4 As shown, the storage 1005 as a storage medium may include an operating system, a network communication module, a user interface module and a control program. The operating system is a program for managing and controlling the rail transit file parsing system and software resources based on artificial intelligence, and supports the operation of the network communication module, the user interface module, the control program and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.

[0143] exist Figure 4 In the hardware structure of the rail transit file parsing system based on artificial intelligence shown in the figure, the network interface 1004 is mainly used to connect to the system server and communicate data with the system server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; the processor 1001 can call the control program stored in the storage 1005 and perform the following operations:

[0144] Obtaining cost documents of rail transit, parsing subject sentences of each cost document based on a subject parsing module in an artificial intelligence model, and classifying each cost document according to the parsed subject sentences to generate a plurality of cost document groups;

[0145] According to the reference format corresponding to each of the cost file groups, format conversion is performed on each of the cost file groups;

[0146] Divide each of the cost documents after format conversion into a header part and a content part, and perform header parsing and content parsing on the divided header part and content part respectively based on the header parsing module and the content parsing module in the artificial intelligence model to generate a standardized header and standardized content;

[0147] The standardized header and the standardized content are combined to generate a standardized cost document.

[0148] Furthermore, the step of parsing the subject sentences of each of the cost documents based on the subject parsing module in the artificial intelligence model includes:

[0149] Extracting the subject sentences of each of the cost documents according to the subject analysis module, and performing forward word segmentation and reverse word segmentation on each of the subject sentences to obtain forward phrases and reverse phrases;

[0150] The positive information index between two adjacent positive words in the positive phrase is calculated based on the preset information degree formula in the topic analysis module, and the reverse information degree index between two adjacent reverse words in the reverse phrase is calculated, wherein the preset information degree calculation formula is:

[0151]

[0152] Among them, M (i，i+1) represents the information index between the i-th word and the i+1-th word in the phrase, w (i，i+1) represents the weight between the i-th word and the i+1-th word in the phrase, a i Represents the word vector of the i-th word in the phrase, a i+1 represents the word vector of the i+1th word in the phrase, T represents the transposition operation, and |·| represents the modulo operation;

[0153] The forward information indexes are summed to obtain a first operation result, and the reverse information indexes are summed to obtain a second operation result, and the parsing of the subject sentence is completed based on the size relationship between the first operation result and the second operation result.

[0154] Furthermore, the header parsing module and the content parsing module in the artificial intelligence model respectively perform header parsing and content parsing on the divided header part and content part to generate a standardized header and standardized content, including:

[0155] Extracting text sentences and header keywords in the header part based on the header parsing module, and parsing each of the text sentences according to a preset sentence library to obtain a standard text sentence corresponding to each of the text sentences;

[0156] Based on the header parsing module, a preset rail transit vocabulary is called to parse each of the header keywords, obtain a standard keyword corresponding to each of the header keywords, and generate the standardized header with each of the standard text sentences and each of the standard keywords;

[0157] Based on the content analysis module, each cell in the content part is analyzed, an abnormal cell among the cells is determined, and the abnormal cell is corrected according to the abnormal type of the abnormal cell to obtain standardized content.

[0158] Furthermore, the step of parsing each of the text sentences according to the preset sentence library to obtain a standard text sentence corresponding to each of the text sentences includes:

[0159] For each of the text sentences, obtain word vectors of the characters contained in the text sentence, and remove the connecting words in the text sentence according to each of the word vectors to obtain the text sentence to be parsed;

[0160] Based on the similarity calculation formula in the header parsing module, the similarity between the text sentence to be parsed and each reference text sentence in the preset sentence library is calculated to obtain a text similarity value, wherein the similarity calculation formula is:

[0161]

[0162] Among them, P τ represents the text similarity value between the text sentence to be parsed and the τth reference text sentence in the preset sentence library, n represents the number of characters in the text sentence to be parsed, sim(·) represents the function used to calculate the similarity of the text meaning, Z1 k 、Z1 k+1 Respectively represent the word vectors of the kth and k+1th words in the text sentence to be parsed, Z2 τk 、Z2 τ(k+1) denote the word vectors of the kth and k+1th words in the τth reference text sentence, respectively; γ denotes the importance parameter of word order similarity to word semantic similarity; and |·| denotes a modular operation;

[0163] The maximum value among the text similarity values ​​is determined, and based on the reference text sentence corresponding to the maximum value in a preset sentence library, the text sentence is converted into a standard text sentence.

[0164] Furthermore, the step of parsing each of the header keywords by calling a preset rail transit vocabulary based on the header parsing module to obtain a standard keyword corresponding to each of the header keywords includes:

[0165] For each of the header keywords, the header parsing module compares the first character in the header keyword with the index words in the preset rail transit vocabulary to determine whether there is a target index word corresponding to the first character in each of the index words;

[0166] If the target index word exists, determining the index keyword set corresponding to the target index word in the preset rail transit vocabulary, and the word frequency weight value and adjustment weight value of each index keyword in the index keyword set;

[0167] Based on the vector distance calculation formula in the header parsing module, the vector distance values ​​between the header keywords and each index keyword in the index keyword set are calculated. The vector distance calculation formula is:

[0168]

[0169] Among them, Q μ represents the vector distance value between the header keyword and the μth index keyword in the index keyword set, m represents the vector dimension, C1 represents the word vector of the header keyword, C2 μ Represents the word vector of the μth index keyword, V1 μ Indicates the frequency weight value of the μth index keyword, V2 μ represents the adjusted weight value of the μth index keyword;

[0170] The minimum value among the vector distance values ​​and the target index keyword corresponding to the minimum value in the index keyword set are determined, and a reference keyword corresponding to the target index keyword is searched as the standard keyword.

[0171] Furthermore, the step of parsing each cell in the content part based on the content parsing module to determine an abnormal cell in each cell includes:

[0172] Parsing each cell of the content part based on the content parsing module, determining a first type of cell lacking a value and a second type of cell having an invalid value in each of the cells;

[0173] For the other cells in the cells except the first type of cells and the second type of cells, based on the row restrictions and column restrictions corresponding to the other cells, determine the third type of cells in the other cells that have abnormal values;

[0174] The first type of cells, the second type of cells and the third type of cells are collectively determined as the abnormal cells.

[0175] Furthermore, before the step of obtaining the cost file of rail transit, the processor 1001 may call the control program stored in the storage 1005 and perform the following operations:

[0176] Collecting the specification document of the rail transit, obtaining corpus samples based on the specification document, and dividing the corpus samples into training corpus samples and test corpus samples;

[0177] Based on the training corpus sample, forward training and reverse training are performed on the topic analysis module in the artificial intelligence model, and after the training reaches a preset number of times, a word segmentation test is performed on the topic analysis module based on the test corpus sample to obtain a word segmentation test result;

[0178] The performance parameters of the topic analysis module are calculated according to the word segmentation test results, and the calculation formula is:

[0179]

[0180] Among them, F represents the performance parameter, β represents the number of test corpus samples, and Y 1α represents the αth test result generated by the αth test corpus sample, Y 2α represents the αth reference result corresponding to the αth test corpus sample, X1 and X2 represent the accuracy weight and ambiguity weight respectively, θ(·) represents a statistical function, which is used to count the number of ambiguities in the test result based on the ambiguous reference results in the reference result, and Y1 represents the test result;

[0181] Comparing the performance parameter with a preset performance threshold to determine whether the performance parameter is less than the preset performance threshold, and if so, completing the training of the topic analysis module;

[0182] If the performance parameter is greater than or equal to a preset performance threshold, the topic analysis module is iteratively trained based on the training corpus sample until the performance parameter is less than the preset performance threshold.

[0183] Furthermore, the step of merging the standardized header and the standardized content to generate a standardized cost document includes:

[0184] Determining whether a first arrangement order of each standardized content in the standardized header is the same as a second arrangement order of each header content in the header part;

[0185] If the first arrangement order is the same as the second arrangement order, merging the standardized header and the standardized content to generate the standardized cost file;

[0186] If the first arrangement order is different from the second arrangement order, determining the content to be adjusted in the standardized content and the adjustment order of the content to be adjusted according to the difference between the first arrangement order and the second arrangement order;

[0187] According to the adjustment sequence, the arrangement sequence of the contents to be adjusted in the standardized contents is adjusted, and the adjusted standardized contents are merged with the standardized header to generate the standardized cost file.

[0188] The specific implementation of the rail transit file parsing system based on artificial intelligence of the present invention is basically the same as the embodiments of the rail transit file parsing method based on artificial intelligence described above, and will not be repeated here.

[0189] The embodiment of the present invention further provides a storage medium having a control program stored thereon, and when the control program is executed by a processor, the steps of the rail transit file parsing method based on artificial intelligence are implemented.

[0190] The storage medium of the present invention may be a computer-readable storage medium, and its implementation method is substantially the same as the above-mentioned embodiments of the rail transit file parsing method based on artificial intelligence, and will not be described in detail here.

[0191] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.

Claims

1. A rail transit file parsing method based on artificial intelligence, characterized in that: The rail transit file parsing method comprises: Obtaining cost documents of rail transit, parsing subject sentences of each cost document based on a subject parsing module in an artificial intelligence model, and classifying each cost document according to the parsed subject sentences to generate a plurality of cost document groups; According to the reference format corresponding to each of the cost file groups, format conversion is performed on each of the cost file groups; Divide each of the cost documents after format conversion into a header part and a content part, and perform header parsing and content parsing on the divided header part and content part respectively based on the header parsing module and the content parsing module in the artificial intelligence model to generate a standardized header and standardized content; The standardized header and the standardized content are combined to generate a standardized cost document; The step of parsing the subject sentences of each cost document based on the subject parsing module in the artificial intelligence model includes: Extracting the subject sentences of each of the cost documents according to the subject analysis module, and performing forward word segmentation and reverse word segmentation on each of the subject sentences to obtain forward phrases and reverse phrases; The positive information index between two adjacent positive words in the positive phrase is calculated based on the preset information degree formula in the topic analysis module, and the reverse information degree index between two adjacent reverse words in the reverse phrase is calculated, wherein the preset information degree calculation formula is: Among them, M (i，i+1) represents the information index between the i-th word and the i+1-th word in the phrase, w (i，i+1) represents the weight between the i-th word and the i+1-th word in the phrase, a i Represents the word vector of the i-th word in the phrase, a i+1 represents the word vector of the i+1th word in the phrase, T represents the transposition operation, and |·| represents the modulo operation; The forward information indexes are summed to obtain a first operation result, and the reverse information indexes are summed to obtain a second operation result, and the parsing of the subject sentence is completed based on the size relationship between the first operation result and the second operation result.

2. The rail transit file parsing method according to claim 1, characterized in that: The step of performing header parsing and content parsing on the divided header part and content part respectively based on the header parsing module and the content parsing module in the artificial intelligence model to generate a standardized header and standardized content includes: Extracting text sentences and header keywords in the header part based on the header parsing module, and parsing each of the text sentences according to a preset sentence library to obtain a standard text sentence corresponding to each of the text sentences; Based on the header parsing module, a preset rail transit vocabulary is called to parse each of the header keywords, obtain a standard keyword corresponding to each of the header keywords, and generate the standardized header with each of the standard text sentences and each of the standard keywords; Based on the content analysis module, each cell in the content part is analyzed, an abnormal cell among the cells is determined, and the abnormal cell is corrected according to the abnormal type of the abnormal cell to obtain standardized content.

3. The rail transit file parsing method according to claim 2, characterized in that: The step of parsing each of the text sentences according to the preset sentence library to obtain a standard text sentence corresponding to each of the text sentences comprises: For each of the text sentences, obtain word vectors of the characters contained in the text sentence, and remove the connecting words in the text sentence according to each of the word vectors to obtain the text sentence to be parsed; Based on the similarity calculation formula in the header parsing module, the similarity between the text sentence to be parsed and each reference text sentence in the preset sentence library is calculated to obtain a text similarity value, wherein the similarity calculation formula is: Among them, P τ represents the text similarity value between the text sentence to be parsed and the τth reference text sentence in the preset sentence library, n represents the number of characters in the text sentence to be parsed, sim(·) represents the function used to calculate the similarity of the text meaning, Z1 k 、Z1 k+1 Respectively represent the word vectors of the kth and k+1th words in the text sentence to be parsed, Z2 τk 、Z2 τ(k+1) They represent the word vectors of the kth and k+1th words in the τth reference text sentence, respectively. γ · represents the importance of the similarity of the order of words to the similarity of the semantics of the words; · represents the modulo operation; The maximum value among the text similarity values ​​is determined, and based on the reference text sentence corresponding to the maximum value in a preset sentence library, the text sentence is converted into a standard text sentence.

4. The rail transit file parsing method according to claim 2, characterized in that: The step of parsing each of the header keywords by calling a preset rail transit vocabulary based on the header parsing module to obtain a standard keyword corresponding to each of the header keywords comprises: For each of the header keywords, the header parsing module compares the first character in the header keyword with the index words in the preset rail transit vocabulary to determine whether there is a target index word corresponding to the first character in each of the index words; If the target index word exists, determining the index keyword set corresponding to the target index word in the preset rail transit vocabulary, and the word frequency weight value and adjustment weight value of each index keyword in the index keyword set; Based on the vector distance calculation formula in the header parsing module, the vector distance values ​​between the header keywords and each index keyword in the index keyword set are calculated. The vector distance calculation formula is: Among them, Q μ represents the vector distance value between the header keyword and the μth index keyword in the index keyword set, m represents the vector dimension, C1s represents the value of the word vector of the header keyword in the sth dimension, C2 μs V1 represents the value of the word vector of the μth index keyword in the sth dimension, μ Indicates the frequency weight value of the μth index keyword, V2 μ represents the adjusted weight value of the μth index keyword; The minimum value among the vector distance values ​​and the target index keyword corresponding to the minimum value in the index keyword set are determined, and a reference keyword corresponding to the target index keyword is searched as the standard keyword.

5. The rail transit file parsing method according to claim 2, characterized in that: The step of parsing each cell in the content part based on the content parsing module and determining an abnormal cell in each cell comprises: Parsing each cell of the content part based on the content parsing module, determining a first type of cell lacking a value and a second type of cell having an invalid value in each of the cells; For the other cells in the cells except the first type of cells and the second type of cells, based on the row restrictions and column restrictions corresponding to the other cells, determine the third type of cells in the other cells that have abnormal values; The first type of cells, the second type of cells and the third type of cells are collectively determined as the abnormal cells.

6. The rail transit file parsing method according to any one of claims 1 to 5, characterized in that: The step of obtaining the cost document of rail transit also includes: Collecting the specification document of the rail transit, obtaining corpus samples based on the specification document, and dividing the corpus samples into training corpus samples and test corpus samples; Based on the training corpus sample, forward training and reverse training are performed on the topic analysis module in the artificial intelligence model, and after the training reaches a preset number of times, a word segmentation test is performed on the topic analysis module based on the test corpus sample to obtain a word segmentation test result; The performance parameters of the topic analysis module are calculated according to the word segmentation test results, and the calculation formula is: Among them, F represents the performance parameter, β represents the number of test corpus samples, and Y 1α represents the αth test result generated by the αth test corpus sample, Y 2α represents the αth reference result corresponding to the αth test corpus sample, X1 and X2 represent the accuracy weight and ambiguity weight respectively, θ(·) represents a statistical function, which is used to count the number of ambiguities in the test result based on the ambiguous reference results in the reference result, and Y1 represents the test result; Comparing the performance parameter with a preset performance threshold to determine whether the performance parameter is less than the preset performance threshold, and if so, completing the training of the topic analysis module; If the performance parameter is greater than or equal to a preset performance threshold, the topic analysis module is iteratively trained based on the training corpus sample until the performance parameter is less than the preset performance threshold.

7. The rail transit file parsing method according to any one of claims 1 to 5, characterized in that: The step of merging the standardized header and the standardized content to generate a standardized cost document comprises: Determining whether a first arrangement order of each standardized content in the standardized header is the same as a second arrangement order of each header content in the header part; If the first arrangement order is the same as the second arrangement order, merging the standardized header and the standardized content to generate the standardized cost file; If the first arrangement order is different from the second arrangement order, determining the content to be adjusted in the standardized content and the adjustment order of the content to be adjusted according to the difference between the first arrangement order and the second arrangement order; According to the adjustment sequence, the arrangement sequence of the contents to be adjusted in the standardized contents is adjusted, and the adjusted standardized contents are merged with the standardized header to generate the standardized cost file.

8. A rail transit file parsing system based on artificial intelligence, characterized in that: The rail transit file parsing system includes a storage device, a processor, a communication bus, and a control program stored in the storage device: The communication bus is used to realize the connection and communication between the processor and the storage; The processor is used to execute the control program to implement the steps of the rail transit file parsing method based on artificial intelligence as described in any one of claims 1-7.

9. A storage medium, characterized in that: The storage medium stores a control program, and when the control program is executed by the processor, the steps of the rail transit file parsing method based on artificial intelligence as described in any one of claims 1 to 7 are implemented.

Images