Dynamic updating and maintaining method and system for standard file library of construction project detection mechanism, computer equipment and storage medium

Abstract

The invention discloses a method and a system for dynamically updating and maintaining a standard file library of a construction project detection mechanism, computer equipment and a storage medium. The method comprises the following steps of: generating a list of to-be-searched new standard files from a standard file library of a construction project detection mechanism, and setting a standard number of each standard file in the list as a character string in a URL (Uniform Resource Locator) form; respectively adding the website character string of the retrieval page of the standard query website and the URL form character string of each standard number in the list to obtain a retrieval word list; crawling a corresponding standard query website by using the search word list to obtain an analytic tree of a search result page; traversing the analytic tree to obtain standard file information of a retrieval result different from the standard file information in the list, and updating the novelty search log; traversing all the standard query websites, obtaining standard files needing to be updated into a standard file library, and performing dynamic classification maintenance on the standard files. According to the method, the standard file can be accurately and efficiently searched and maintained.

Description

Technical Field

[0001] The present invention belongs to the technical field of standard file data processing, and specifically relates to a method, system, computer equipment and storage medium for dynamically updating and maintaining a standard file library of a construction engineering inspection organization. Background Art

[0002] In order to ensure the authenticity, reliability and standardization of construction project inspection work, the inspection agency should strictly implement the requirements, parameters and steps of the inspection standard documents when engaging in inspection activities, and form a controlled standard document library management system. Construction project inspection standard documents are the basis for a type of inspection agency to issue engineering quality inspection data or inspection conclusions to the society. Only by implementing the current engineering inspection standards can the accuracy, effectiveness and fairness of the inspection and inspection work be guaranteed, and the risks in the inspection and inspection process be reduced. Timely understanding of the upcoming inspection standards can also enable the inspection agencies and inspection personnel to adjust the inspection plans and inspection methods and replace the inspection equipment in a timely manner. Therefore, the inspection agency needs to regularly check the new construction project inspection standard documents to replace the abolished standards and prompt the standards to be implemented. However, there are usually thousands of standards involved in engineering inspection, involving the fields of building materials inspection, steel structure inspection, foundation inspection, highway inspection, bridge inspection, etc. If the traditional manual standard search method is used, the search time is long, the workload is large, and a large amount of labor costs for repeated mechanical work are generated.

[0003] In addition to ensuring that the reference standards are valid, those engaged in construction engineering inspection activities should also be familiar with all the inspection basis standards and all the corresponding inspection parameters involved in the inspected samples. At present, the controlled management of construction engineering inspection standards by inspection agencies only stays at the file archiving stage. The types of construction engineering inspection standard documents involve inspection method standards, product standards, acceptance standards, instrument standards, etc. Even for inspectors with rich technical experience, facing a wide variety of inspection standards, under the current management level, inspectors still need to manually retrieve and manually check the new information from the website that publishes the standard documents, which takes a long time. Under the existing inspection standard system, there is a strong correlation between the types of inspected samples, technical parameters, inspection methods, and equipment and instruments. The information contained in the massive standard content has not been mined and utilized, and a standard system with intuitive content, clear structure, scientific classification, and convenient archiving has not been formed. The existing standard document library management lacks the function of automatically and accurately updating standard documents. The efficiency of standard document management is low, which is not convenient for a clearer and deeper grasp of the existing service system of the inspection agency, and it is also not convenient to discover the gaps in its own business and its correlation with existing qualifications, and to develop new inspection business. Summary of the invention

[0004] In order to overcome one or more defects and shortcomings of the prior art, the first purpose of the present invention is to provide a method for dynamically updating and maintaining a standard file library of a construction engineering inspection agency, the second purpose is to provide a system for dynamically updating and maintaining a standard file library of a construction engineering inspection agency, the third purpose is to provide a computer device, and the fourth purpose is to provide a storage medium that can provide real-time, accurate, and batch standard file updates, with high update accuracy and maintenance management efficiency.

[0005] The first object of the present invention adopts the following technical scheme: A method for dynamically updating and maintaining a standard document library of a construction engineering inspection organization comprises the following steps: Generate a list of new standard documents to be checked from the standard document library, and set the standard number of each standard document in the list as a string in the form of a URL; The URL string of the search page of the standard query website is added to the URL format string of each standard number in the list to obtain a search word string, and the search word list of the corresponding standard query website is composed of all the search word strings; Use the search term list to crawl the corresponding standard query website to obtain the parse tree of the search result page corresponding to all the search term strings; Traverse the parsing tree, obtain standard file information of the search results that are different from the standard file information in the list, and update the new search log; Traverse all standard query websites, merge the new query logs to update the standard file information, obtain the standard files that need to be updated into the standard file library, and dynamically classify and maintain the standard files.

[0006] The first preferred technical solution is to traverse the parsing tree, obtain standard file information of the search results that are different from the standard file information in the list, and update the new search log, the steps comprising: Traverse the tags in the parsed tree and find the tags of the elements where the standard file information is located; Among all the elements with labels in the parse tree, obtain the elements whose label content is one of current, abolished, and invalidated, find the parent element of the corresponding element at the previous level, and obtain the parent element with the same standard number as the standard file; Compare the standard file information in the list with the parent element with the same standard number, and store the inconsistent standard file information in the difference list variable; Replace the original information of the corresponding standard file according to the difference list variables and update the new search log.

[0007] A second preferred technical solution is to traverse the parsing tree, obtain standard file information of the search results that are different from the standard file information in the list, and update the new search log, the steps comprising: Traverse the tags in the parsed tree and find the tags of the elements where the standard file information is located; Among all the elements with tags in the parse tree, get the element whose tag content is to be implemented, and get the corresponding standard number string from its corresponding parent element; Subtract the year information of the standard number in the parent element from the standard number string to obtain a corresponding string, and when the corresponding string is the same as the search term string in the search term list, obtain the standard file information of the parent element corresponding to the corresponding string; Compare the standard file information in the list with the standard file information obtained from the corresponding parent element, and store the inconsistent standard file information in the difference list variable; Replace the original information of the corresponding standard file according to the difference list variables and update the new search log.

[0008] A more preferred technical solution of the first or second technical solution is to replace the original information of the corresponding standard file according to the difference list variable, and the steps include: If the format of the standard name and standard number of the standard document information in the list is written incorrectly, the original standard document information in the standard document library shall be replaced with the new standard document information; If the standard year of the standard document information in the list is different, the original standard document information in the standard document library will be replaced with the updated standard document information; If there is no current standard file information in the difference list variable, the original standard file information and standard files in the standard file library will be deleted.

[0009] The third preferred technical solution is that when using the search term list to crawl the corresponding standard query website, if the standard query website cannot be directly crawled because it obtains data by interacting with the server and renders a dynamic page, then the crawler simulates the behavior of the browser, executes JavaScript code and parses JavaScript to dynamically load data to obtain the parse tree of the search result page.

[0010] The fourth preferred technical solution is to dynamically classify and maintain the standard files, the steps comprising: Convert all standard files in the standard file library into plain text files in batches through image recognition; Create a stop word list to enter strings of meaningless words; Segment the text content in all plain text files and mark the parts of speech to form a list format matrix; Perform data cleaning on the list format matrix, exclude meaningless word strings through the stop word list, and only extract words with noun part of speech and more than one character. Repeat the data cleaning for at least three times to form a noun set. Traverse all words in the noun set, count the word frequency and inverse document frequency of the corresponding nouns, and then calculate the frequency enhancement index of the corresponding noun based on the word frequency and inverse document frequency; Sort the values ​​of the frequency enhancement index of the corresponding nouns from large to small, take no less than the first ten nouns as the keyword set, and use all the keywords and the frequency enhancement indexes corresponding to the keywords to form a keyword matrix; Establish a graph relationship network model, in which the number of nodes is consistent with the number of standard files in the current standard file library, and there are relationship weights between each node; Traverse all nodes in the graph relationship network model, and determine whether there is a keyword in the standard file corresponding to each node, which is the same as the keyword in the standard file corresponding to another node. When the same keyword exists, multiply the frequency enhancement index of the two standard files on the keyword as the increment of the relationship weight, and then use the relationship weight after adding the increment to form a new relationship weight matrix of the graph relationship network model; According to the relationship weight matrix, all standard files in the current standard file library are classified by modularity, and each classification is aggregated into a new node in the graph relationship network model to generate a relationship network of the new graph relationship network model until the relationship network no longer changes, thereby completing the classification of standard files.

[0011] A fifth preferred technical solution is that after dynamically classifying and maintaining the standard files, there are further steps including: Send maintenance completion message to new search users and provide the most suitable standard search query interface; The optimal standard search includes: searching from the maintained standard document library according to the search term set input by the new search user, and outputting the standard documents in descending order of relevance between the standard documents and each search term, wherein the relevance is calculated based on the frequency enhancement index and the search term.

[0012] The second object of the present invention adopts the following technical solution: A construction engineering inspection agency standard file library dynamic update and maintenance system using any of the aforementioned construction engineering inspection agency standard file library dynamic update and maintenance methods, comprising a novelty search preparation module, a standard file library, a crawler module, and a maintenance module; The novelty search preparation module is connected to the standard file library and the crawler module respectively, and the crawler module is connected to the standard file library and the maintenance module respectively; The novelty search preparation module is used to generate a list of standard documents to be searched from the standard document library, and set the standard number of each standard document in the list as a string in the form of a URL; The novelty search preparation module is also used to add the URL string of the search page of the standard search website to the URL format string of each standard number in the list to obtain a search word string, and the search word list of the corresponding standard search website is composed of all the search word strings; The crawler module is used to crawl the corresponding standard query website using the search word list to obtain the parse tree of the search result page corresponding to all the search word strings; The crawler module is also used to traverse the parsing tree, obtain standard file information of the search results that are different from the standard file information in the list, and update the new search log; The maintenance module is used to merge the new query logs to update the standard file information after the crawler module traverses all standard query websites, obtain the standard files that need to be updated into the standard file library, and maintain the files and information.

[0013] The third object of the present invention adopts the following technical solution: A computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor; when the computer program is executed by the processor, any of the aforementioned methods for dynamically updating and maintaining a standard file library of a construction engineering inspection agency is implemented.

[0014] The fourth object of the present invention adopts the following technical solution: A storage medium is a computer-readable storage medium storing a computer program. When the computer program is executed by a computer, any of the above-mentioned methods for dynamically updating and maintaining a standard file library of a construction engineering inspection agency is implemented.

[0015] Compared with the prior art, the present invention has the following advantages and beneficial effects: Compared with the current manual novelty search method, the present invention can provide real-time and accurate standard document novelty search, and can also search standard documents in large quantities. It has the functions of regular automatic novelty search, fuzzy query, and similarity query, which improves the accuracy of novelty search and avoids the time-consuming and difficult to grasp accuracy problems of manual query and update. It will improve the management efficiency of construction project inspection standard documents, enable inspection agencies to have a clearer and in-depth grasp of the existing service system, and at the same time facilitate inspection agencies to discover gaps in their own business and their correlation with existing qualifications, and develop new inspection business. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a process for dynamically updating and maintaining a standard file library of a construction engineering inspection organization according to the present invention; Figure 2It is a flow chart of another method for dynamically updating and maintaining a standard file library of a construction engineering inspection organization according to the present invention; Figure 3 It is a schematic diagram of the framework structure of the dynamic updating and maintenance system of the standard file library of the construction engineering inspection organization of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0018] It should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the disclosure of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the disclosure of the present invention.

[0019] In addition, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. Similarly, words such as "one", "an", or "the" do not indicate a quantitative limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" and the like mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "electrically connected" or "connected" and the like are not limited to physical or mechanical electrical connections, but may include electrical connections, whether direct or indirect.

[0020] Example 1 like Figure 1 This embodiment provides a method for dynamically updating and maintaining a standard file library of a construction engineering inspection organization, which is run in a local computer system and includes the following steps: S1. According to a new search instruction issued by a user or a new search instruction automatically issued at a set period, a list of standard files to be searched is generated from a standard file library, and the standard number of each standard file in the list is set as a string in the form of a URL; In this embodiment, the novelty search instruction may be a current novelty search or an advance novelty search, and the standard files are stored in a local computer in the form of a database to form a standard file library; In this embodiment, the method of setting the string in URL format is as follows: the slash " / " in the standard number is changed to the corresponding URL code "%2F", a plus sign "+" is inserted between the type and the code of the standard number, and the space is cancelled. For example, the standard number "GB / T 24717-2009" is changed to a string in URL format as "GB%2FT+24717-2009", where "GB / T" is the code of the standard issuing organization, "24717" is the code, and "2009" is the year; S2. Set the standard query website W i The URL string of the search page (i=1,2,…,g, g is the total number of standard query websites) is added to the URL format string of each standard number in the list to obtain the search word string Str j (j=1,2,…,n, n is the total number of standard documents in the standard document library before the new search), all the search terms string Str j A list of search terms that compose the corresponding standard query website; This embodiment is preferred, the standard query website W i Including but not limited to the National Standard Document Information Public Service Platform, Industrial Standards Network, Wanfang, China Standards Service Network and other websites with standard search functions; S3. Use the search term list to query the website W for the corresponding standard i Crawling, getting the jth search word string Str j The corresponding parse tree T of the search result page j (i=1,2,…,g, j=1,2,…,n, g is the total number of standard query websites, n is the total number of standard documents in the standard document library before the new search), that is, different search word strings Str j On the same standard query website W i The corresponding parse tree T can be obtained in j ; In this embodiment, preferably, a Python crawler program preset in a local computer is used to query the corresponding standard website W i Perform crawling; In this embodiment, preferably, in step S3, the search word list is used to query the website W according to the corresponding standard. i When crawling, if the standard query website W i In the case where data is obtained by interacting with the server and rendering to form a dynamic page, it is impossible to crawl directly. In this case, the crawler simulates the behavior of the browser, executes JavaScript code, and parses JavaScript to dynamically load data to obtain the parse tree T of the search result page. j ; S4, traverse all parse trees T j, obtain the standard file information of the search results that are different from the standard file information in the list, and update the new search log Log i (i=1,2,…,g, g is the total number of standard query websites); the steps include: S41. Traverse the parse tree T j Find the tag of the element where the standard file information is located <k>; S42, in the parse tree T j Iterate through all the tags <k>Get the element a whose tag content is one of "current", "revoked", and "invalidated" j,1 , a j,2 ,…,a j,q (j=1,2,…,g, g is the search string Str j , q is the total number of elements with the same label that can be found), from its corresponding parent element A j,1 , A j,2 ,…,A j,q Get the corresponding standard number string N j,1 , N j,2 ,…,N j,q , from which to obtain j The same string N j and the standard file information of its corresponding parent element; S43, the search word string Str j Corresponding to the standard file information in the standard file library, and N j Compare the standard file information with its corresponding parent element, and store the inconsistent standard file information into the difference list variable G i (i=1,2,…,g, g is the total number of standard query websites); In this embodiment, the compared standard file information includes but is not limited to standard status, standard number, standard name, standard release date, and standard implementation date; S44, after traversing all the parse trees T j Then, according to the difference list variable G i Replace the original information of the corresponding standard file and update the new log i ; Among them, step S44 is based on the difference list variable G i Replace the original information of the corresponding standard file. The specific steps are as follows: S441. If the format of the standard name and standard number of the standard file information in the list is incorrect, replace the original standard file information in the standard file library with the new standard file information, and then log it in the new log. i Added logging of modified standard file information; S442. If the standard year of the standard file information in the list is different, the original standard file information in the standard file library is replaced with the new standard file information, and then the new standard file information is recorded in the new log. i Added log records of abolishing old standard documents and implementing new standard documents; S443, if the difference list variable G i If there is no current standard file information, delete the original standard file information and standard files in the standard file library, and then add a log record in the novelty search log Log i indicating that the old standard has been repealed; S5. Traverse all standard query websites W i and merge all novelty search logs Log i After updating the standard file information, purchase and download the electronic version through online automatic payment, and / or purchase the paper version offline and scan and enter the standard files B j (j = 1, 2, …, f, where f is the total number of standard files to be updated) into the standard file library for dynamic classification and maintenance of the standard files; Preferably in this embodiment, the dynamic classification and maintenance of the standard files in step S5 specifically includes the following steps: S51. Batch convert all standard files {B1, B2, …, B m} in the standard file library into plain text format files {P1, P2, …, P m} through image recognition; S52. Establish a stop word list for entering strings of meaningless words; Preferably in this embodiment, the stop word list is used by being preset and entered. Common meaningless strings include nouns without special meanings, common high-frequency words in standards, auxiliary words, punctuation marks, prepositions, pronouns, adverbs, numerals, quantifiers, single-digit symbols, and English words corresponding to the above words; More preferably in this embodiment, nouns without special meanings include "problem", "date", "method", "requirement", "rule", etc.; common high-frequency words in standards include "standard", "report", "building", "material", "research", "clause", "classification", "term", "sample", "personnel", etc.; auxiliary words include "de", "di", "le", "ke", "ying", etc.; punctuation marks include "《", "-", ",", "(", "、", etc.; prepositions include "zai", "yu", "zi", "cong", "ba", etc.; pronouns include "ta", "na", "zhe", "ni", "wo", etc.; adverbs include "kuai", "da", "gao", "zhong", "qiang", etc.; numerals include "yi", "liang", "er", "shi", "bai", etc.; quantifiers include "ge", "miao", "ren", "tai", "mi", etc.; Arabic numerals include "0", "1", "2", "3", "4", etc.; S53. Perform word segmentation on the text content in all plain text files {P1, P2, …, P m} and mark the part of speech to form a list format matrix; Preferably in this embodiment, call the pseg function of the python jieba library to segment the plain text files {P1, P2, …, P m } is processed by word segmentation, part of speech marking, and list format matrix is ​​formed. For example, the text content "This standard applies to general silicate cement." is converted into a list matrix [['this', 'standard', 'applicable', 'to', 'general', 'silicate', 'cement', '. '] ['r', 'n', 'v', 'p', 'v', 'n', 'n', 'wp']], where r represents a pronoun, n is a noun, v is a verb, p is a preposition, and wp is a punctuation mark; S54, perform data cleaning on the list format matrix, exclude meaningless word strings through the stop word list, and only extract words with the part of speech being noun "n" and the number of characters being greater than one. All standard files {B1, B2, ..., B m } form a noun set {H1,H2,…,H m }, where H i For {h i,1 ,h i ,2,…,h i,d }(d is standard file B i The total number of nouns after data cleaning); S55, traverse the noun set {H1, H2, ..., H m }, and count the corresponding nouns h i,j The term frequency TF and inverse document frequency IDF of (j=1,2,…,d), and then the frequency enhancement index TF-IDF of the corresponding noun is calculated based on the term frequency TF and inverse document frequency IDF i,j , the specific calculation is shown as follows: TF =n i,j / , IDF =log(m / (1+m i,j )) , TF-IDF i,j = TF·IDF·I, Among them, n i,j h i,j In H i The number of occurrences in (i=1,2,…,m), H i All h i,j The sum of the respective times, m is the total number of standard files, m i,j To include h i,j The total number of standard files, I is the set enhancement coefficient (i.e., weight value, which is a constant) used to i,j Perform word enhancement and text position enhancement, log() is a logarithmic function with a base of 10; Word enhancement is used to highlight features when a word contains strings of some standard key information, such as standard type words starting with strings like "GB", "CECS", etc., words representing detection devices containing "meter", "instrument", "device", etc., and words representing detection parameters containing "degree", "value", "coefficient", "quantity", "property", etc. Text position enhancement is used to strengthen the position feature when a word appears in specific positions of a standard document, such as the cover page, preface page, general principles page, terms and definitions page, and basic regulations page; The term frequency TF represents the frequency of a certain word in a certain text content, and the inverse document frequency IDF represents the distribution of a certain word in a certain text content. Both the term frequency TF and the inverse document frequency IDF are common word feature statistical indicators in the field of document information technology; S56. Sort the values of the frequency enhancement index TF-IDF of the corresponding nouns in descending order, and select no less than the top ten nouns as the keyword set K i,j and form a keyword matrix M with all the keywords and the corresponding frequency enhancement index TF-IDF i , i,j M i ={[K i,1 , K i,2 ,…, K i,c [ TF-IDF i,1 , TF-IDF i,2 ,…, TF-IDF i,c} (c represents the total number of keywords taken out, K i =[K i,1 , K i,2 ,…, K i,c ); In this embodiment, preferably, the total number of keywords K i taken out is optimally 20, and the keyword set K i = [K i,1 , K i,2 ,…,K i,20 ; In this embodiment, it is further preferred to select standard documents and their corresponding K i according to the ratio of 5%-10% of the standard document library, and add nouns without special meaning or common high-frequency words in the standard to the stop word list; In this embodiment, it is further preferred that if there are nouns with significant features in the selected standard text but not falling into K i , then reset the enhancement coefficient I of this noun so that it meets the condition of entering the keyword set K i ; ​​S57, establish a graph relationship network model, the number of nodes in the graph relationship network model and the number of standard files B in the current standard file library i The number of nodes remains the same, and there is a relationship weight e between each node; In this embodiment, preferably, when the graph relationship network model is just constructed, the relationship weight e between each node is initialized to 0; S58, traverse all nodes in the graph relationship network model, determine whether there is a keyword in the standard file corresponding to each node, which is the same as the keyword in the standard file corresponding to another node, and when the same keyword exists, the frequency enhancement index TF-IDF of the two standard files on the keyword is increased. i,j Multiply them together as the increment of the relationship weight e of the two standard files, and then use the incremental relationship weight to form a new relationship weight matrix e[m,m] of the graph relationship network model; S59, classify all standard files in the current standard file library according to the relationship weight matrix e[m,m], aggregate each classification into a new node in the graph relationship network model to generate a relationship network of the new graph relationship network model, until the relationship network no longer changes, thereby completing the classification of the standard files; In this embodiment, preferably, the steps of modularity classification are: S591, generate m categories {F1, F2, ..., F m }, each category contains only one standard file; S592, calculating the modularity Q between each standard file and other standard files whose relationship weight is greater than that of the standard file, as shown in the following formula: , Among them, e i,j is the value of the relationship weight between two standard files, e it Standard file B i With a relationship weight greater than B i The sum of the relationship weights of other standard documents, e jt Standard file B j With a relationship weight greater than B j The sum of the relationship weights of other standard files, 2E is the sum of all relationship weights, Indicates B i and B j If they are of the same category, the value is 1, otherwise it is 0; S593, for only standard file B i Classification C i , judge when adding to the category C with the largest modularity j When Is it positive? If the modularity gain If positive, the standard file B i Put it into C j The calculation of is as follows: , in, For B i With C j The sum of the relationship weights of other standard files in C j The sum of the relationship weights of all standard documents in the original, For C j The sum of the relationship weights of other categories that establish relationships with the node; S594, aggregating each classification into a node to generate a new relationship network; S595, repeating steps S591 to S594 to classify the modularity of different standard files until the relationship network no longer changes; In this embodiment, preferably, step S5 implements the maintenance process through a preset Python maintenance program.

[0021] Compared with the prior art, this embodiment has the following beneficial effects: It can provide real-time and accurate standard document novelty search, as well as batch novelty search of standard documents. It has the functions of regular automatic novelty search, fuzzy query and similarity query, which improves the accuracy of novelty search and avoids the time-consuming and difficult to grasp accuracy problems of manual query and update. It improves the efficiency of construction engineering testing standard document management, enables testing institutions to have a clearer and in-depth grasp of the existing service system, and is also convenient for testing institutions to discover gaps in their own business and their relevance to existing qualifications, and to develop new testing business.

[0022] Example 2 like Figure 2 This embodiment provides a method for dynamically updating and maintaining a standard file library of a construction engineering inspection organization, which is run in a server system and includes the following steps: S1. According to a new search instruction issued by a user or a new search instruction automatically issued at a set period, a list of standard files to be searched is generated from a standard file library, and the standard number of each standard file in the list is set as a string in the form of a URL; In this embodiment, the novelty search instruction may be a current novelty search or an advance novelty search, and the standard files are stored in the server in the form of a database to form a standard file library; S2. Set the standard query website W i The URL string of the search page (i=1,2,…,g, g is the total number of standard query websites) is added to the URL format string of each standard number in the list to obtain the search word string Str j (j=1,2,…,n, n is the total number of standard documents in the standard document library before the new search), all the search terms string Str j A list of search terms that compose the corresponding standard query website; This embodiment is preferred, the standard query website W i Including but not limited to the National Standard Document Information Public Service Platform, Gongbiao.com, Wanfang, China Standards Service Network and other websites with standard search functions; S3. Use the search term list to query the website W for the corresponding standard i Crawling, getting the jth search word string Str j The corresponding parse tree T of the search result page j (i=1,2,…,g, j=1,2,…,n, g is the total number of standard query websites, n is the total number of standard documents in the standard document library before the new search), that is, different search word strings Str j On the same standard query website W i The corresponding parse tree T can be obtained in j ;; In this embodiment, preferably, a Python crawler program preset in the server is used to query the corresponding standard website W i Perform crawling; In this embodiment, preferably, in step S3, the search word list is used to query the website W according to the corresponding standard. i When crawling, if the standard query website W i In the case where data is obtained by interacting with the server and rendering to form a dynamic page, it is impossible to crawl directly. In this case, the crawler simulates the behavior of the browser, executes JavaScript code, and parses JavaScript to dynamically load data to obtain the parse tree T of the search result page. j ; S4, traverse all parse trees T j , obtain the standard file information of the search results that are different from the standard file information in the list, and update the new search log Log i (i=1,2,…,g, g is the total number of standard query websites); the steps include: S41. Traverse the parse tree T j Find the tag of the element where the standard file information is located <k>; S42, in the parse tree T j Iterate through all tags <k>Get the element a with the tag content "to be implemented" i,1 , a i,2 ,…,a i,q (i=1,2,…,g, g is the total number of standard query websites, q is the total number of corresponding elements), get the corresponding standard number string N from its corresponding parent element j,1 , N j,2 ,…,N j,q ; S43, the standard number string N j,1 , N j,2 ,…,N j,q Subtract the year information of the standard number in the parent element to get the corresponding string N j + , in the corresponding string N j + The search term string Str in the search term list j If they are the same, get the string N from j + and the standard file information of its corresponding parent element; S44, the search word string Str j Corresponding to the standard file information in the standard file library, and the string N j + Compare the standard file information with its corresponding parent element, and store the inconsistent standard file information into the difference list variable G i (i=1,2,…,g); In this embodiment, the compared standard file information includes but is not limited to standard status, standard number, standard name, standard release date, and standard implementation date; S45, after traversing all the parse trees T j Then, according to the difference list variable G i Replace the original information of the corresponding standard file and update the new log i ; Among them, step S45 is based on the difference list variable G i Replace the original information of the corresponding standard file. The specific steps are as follows: S451. If the format of the standard name and standard number of the standard file information in the list is incorrect, replace the original standard file information in the standard file library with the new standard file information, and then log it in the new log. i Added logging of modified standard file information; S452. If the standard year of the standard file information in the list is different, the original standard file information in the standard file library is replaced with the standard file information after the new search, and then the new search log Log i Added log records of abolishing old standard documents and implementing new standard documents; S453, if the difference list variable G i If there is no current standard file information in the standard file library, delete the original standard file information and standard files in the standard file library, and then search for new logs in the Log i Added log records of old standards that have been abolished; S5. Traverse all standard query websites W i , merge all new search logs Log i After updating the standard document information, purchase it online through automatic payment and download the electronic version, and / or purchase the paper version offline and scan and enter the standard document B that needs to be updated j (j=1,2,…,f, f is the total number of standard files that need to be updated) to the standard file library, and dynamically classify and maintain the standard files; In this embodiment, the dynamic classification maintenance of standard files in step S5 is preferably performed in the following specific steps: S51, all standard files {B1, B2, ..., B m }Batch conversion to plain text format files {P1, P2, ..., P m }; S52, establishing a stop word list for entering character strings of meaningless words; S53, all plain text files {P1, P2, ..., P m } is segmented and marked with parts of speech to form a list format matrix; In this embodiment, preferably, the pseg function of the python jieba library is called to convert the plain text file {P1, P2, ..., P m } to perform word segmentation, mark the part of speech, and form a list format matrix; S54, perform data cleaning on the list format matrix, exclude meaningless word strings through the stop word list, and only extract words with the part of speech being noun "n" and the number of characters being greater than one. All standard files {B1, B2, ..., B m } form a noun set {H1,H2,…,H m }, where H i For {h i,1 ,h i ,2,…,h i,d }(d is standard file B i The total number of nouns after data cleaning); S55, traverse the noun set {H1, H2, ..., H m }, and count the corresponding nouns h i,j The term frequency TF and inverse document frequency IDF of (j=1,2,…,d), and then the frequency enhancement index TF-IDF of the corresponding noun is calculated based on the term frequency TF and inverse document frequency IDF i,j , the specific calculation is shown as follows: TF =n i,j / , IDF =log(m / (1+m i,j )) , TF-IDF i,j = TF·IDF·I, Among them, n i,j h i,j In H i The number of occurrences in (i=1,2,…,m), H i All h i,j The sum of the respective times, m is the total number of standard files, m i,j To include h i,j The total number of standard files, I is the set enhancement coefficient (i.e., weight value, which is a constant) used to i,j Perform word enhancement and text position enhancement, log() is a logarithmic function with a base of 10; S56, the frequency enhancement index TF-IDF of the corresponding noun i,j The values ​​of are sorted from large to small, and no less than the first ten nouns are selected as the keyword set K. i , using all keywords and the frequency enhancement index TF-IDF corresponding to the keywords i,j Composition of keyword matrix M i ={[K i,1 , K i,2 ,…, K i,c ][TF-IDF i,1 , TF-IDF i,2 ,…, TF-IDF i,c ]} (c represents the total number of keywords extracted, K i =[K i,1 , K i,2 ,…, K i,c ]); In this embodiment, the keyword K i The optimal total number is 20, and the keyword set K i = [K i,1 , K i,2 ,…,K i,20 ]; This embodiment further preferably selects standard files and their corresponding K according to a ratio of 5%-10% of the standard file library. i , K i Nouns without special meaning or common high-frequency words in the standard are added to the stop word list; In this embodiment, if there are nouns with significant features in the selected standard text but they do not fall into the K i Then reset the enhancement coefficient I of the noun so that it meets the requirements of entering the keyword set K i conditions; S57, establish a graph relationship network model, the number of nodes in the graph relationship network model and the number of standard files B in the current standard file library i The number of nodes remains the same, and there is a relationship weight e between each node; In this embodiment, preferably, when the graph relationship network model is just constructed, the relationship weight e between each node is initialized to 0; S58, traverse all nodes in the graph relationship network model, determine whether there is a keyword in the standard file corresponding to each node, which is the same as the keyword in the standard file corresponding to another node, and when the same keyword exists, the frequency enhancement index TF-IDF of the two standard files on the keyword is increased. i,j Multiply them together as the increment of the relationship weight e of the two standard files, and then use the incremental relationship weight to form a new relationship weight matrix e[m,m] of the graph relationship network model; S59, classify all standard files in the current standard file library according to the relationship weight matrix e[m,m], aggregate each classification into a new node in the graph relationship network model to generate a relationship network of the new graph relationship network model, until the relationship network no longer changes, thereby completing the classification of the standard files; In this embodiment, preferably, the steps of modularity classification are: S591, generate m categories {F1, F2, ..., F m }, each category contains only one standard file; S592, calculating the modularity Q between each standard file and other standard files whose relationship weight is greater than that of the standard file, as shown in the following formula: , Among them, e i,j is the value of the relationship weight between two standard files, e it Standard file B i With a relationship weight greater than B i The sum of the relationship weights of other standard documents, e jt Standard file B j With a relationship weight greater than B j The sum of the relationship weights of other standard files, 2E is the sum of all relationship weights, Indicates B i and B j If they are of the same category, the value is 1, otherwise it is 0; S593, for only standard file B i Classification C i , judge when adding to the category C with the largest modularity j When Is it positive? If the modularity gain If positive, the standard file B i Put it into C j The calculation of is as follows: , in, For B i With C j The sum of the relationship weights of other standard files in , C j The sum of the relationship weights of all standard documents in the original, For C j The sum of the relationship weights of other categories that establish relationships with the node; S594, aggregating each classification into a node to generate a new relationship network; S595, repeating steps S591 to S594 to classify the modularity of different standard files until the relationship network no longer changes; In this embodiment, preferably, step S5 implements the maintenance process through a preset Python maintenance program; In this embodiment, after the dynamic classification maintenance of the standard files is completed in step S5, there is a step S6 of message notification and providing a query interface, and the step S6 of message notification and providing a query interface includes: S61, sending a maintenance completion message to the new search user and providing the most suitable standard search query interface; The optimal standard search includes: according to the search term set {k1, k2, ..., k n }(k j is the search term, j = 1, 2, ..., n), and then search from the maintained standard document library according to the standard document B i (i=1,2,…,m, m is the total number of standard documents) and each search term k j The correlation degree L i (i=1,2,…,n) Output in order from high to low, with a correlation degree of L i According to the frequency enhancement index TF-IDF i,j 、Search term k j Calculate the correlation L i The specific calculation is shown in the following formula: L i = k j TF-IDF i,j ; S62, sending a maintenance completion message to the new search user, and providing a standard library-map relationship network model display query interface; The standard library graph relationship network model display is used to display the graph relationship network model in the form of a knowledge graph. This embodiment is further preferred and uses the neo4j graph database for display. The graph(), node() and relationship() functions of the py2neo library of python are used to construct the graph relationship network model. At the same time, a retrieval function is provided to display the position of the retrieved standard file or search term in the knowledge graph, its relationship with adjacent standards, and its relationship with classification.

[0023] Compared with the prior art, this embodiment has the following beneficial effects: It can provide real-time and accurate standard document novelty search, as well as batch novelty search of standard documents. It has the functions of regular automatic novelty search, fuzzy query and similarity query, which improves the accuracy of novelty search and avoids the time-consuming and difficult to grasp accuracy problems of manual query and update. It improves the efficiency of construction engineering testing standard document management, enables testing institutions to have a clearer and in-depth grasp of the existing service system, and is also convenient for testing institutions to discover gaps in their own business and their relevance to existing qualifications, and to develop new testing business.

[0024] Example 3 like Figure 3 As shown. This embodiment provides a construction engineering inspection agency standard file library dynamic update maintenance system, which is used to execute the construction engineering inspection agency standard file library dynamic update maintenance method in embodiment 1 or embodiment 2. On the one hand, this system connects one or more users with new search needs through a human-computer interaction interface, and on the other hand, connects to one or more standard query websites that store standard file machine standard file information through the Internet. Standard files are stored in a local computer and / or server in the form of a database to form a standard file library, which contains texts of various formats of standard files and data of standard file information.

[0025] The construction engineering inspection agency standard document library dynamic update and maintenance system of this embodiment includes a novelty search preparation module, a standard document library, a crawler module, a maintenance module, an instruction generation module, and a message and query interface module.

[0026] The instruction generation module is connected to the novelty search preparation module; the instruction generation module is used to generate a novelty search instruction for a standard file and send it to the novelty search preparation module.

[0027] The instruction generation module is provided with an automatic novelty search mode for initiating novelty search instructions for standard files on a set date and / or period.

[0028] In this embodiment, the novelty search instruction can be a current novelty search or an advance novelty search. In this embodiment, the instruction generation module is further preferably provided with a manual novelty search mode, which interfaces with the user through a human-computer interaction interface, and the user manually initiates a novelty search instruction for the standard file.

[0029] The novelty search preparation module is connected to the standard document library and the crawler module respectively; the novelty search preparation module is used to generate a list of standard documents to be searched from the standard document library, and then set the standard number of each standard document in the list as a string in the form of a URL; the novelty search preparation module is also used to send the standard query website W i The URL string of the search page (i=1,2,…,g, g is the total number of standard query websites) is added to the URL format string of each standard number in the list to obtain the search term string, and the search term list of the corresponding standard query website is composed of all the search term strings.

[0030] The search preparation module is also used to send the search word list to the crawler module. i Including but not limited to the national standard document information public service platform, Gongbiao.com, Wanfang, China Standard Service Network and other websites with standard search functions, the required standard query website W i The URL string of the search page is pre-stored in the novelty search preparation module.

[0031] The crawler module is connected to the standard file library and the maintenance module respectively; the crawler module is used to query the website W for the corresponding standard using the search word list i Run a crawler to obtain the parse tree T of the search result page corresponding to the search term string j (i=1,2,…,g, j=1,2,…,n, g is the total number of standard query websites, n is the total number of standard documents in the standard document library before the new search); the crawler module is also used to traverse the parsing tree T j , obtain the standard file information of the search results that are different from the standard file information in the list, and update the new search log. The crawler module includes a crawling module, a parsing module, and a simulation module.

[0032] The crawling module is respectively connected to the novelty preparation module and the standard query website W i Connection; The crawling module is used to query the website W from the corresponding standard according to the search word list i Climb up and get the jth search word string Str j The corresponding parse tree T of the search result page j .

[0033] The simulation module is connected to the crawling module; the simulation module is used to simulate the behavior of the browser, execute JavaScript code and parse JavaScript to dynamically load data; In this embodiment, the crawling module is preferably a Python crawler function program preset in a local computer and / or server. In this embodiment, the crawler function uses a search word list to query the corresponding standard website W i When crawling, if the standard query website W i In the case where data is obtained by interacting with the server and rendered to form a dynamic page, it is impossible to crawl directly. In this case, the simulation module is used when crawling to simulate the behavior of the browser, execute JavaScript code and parse JavaScript to dynamically load data to obtain the parse tree T of the search result page. j .

[0034] The parsing module is connected to the crawling module; the parsing module is used to traverse the parse tree T j , obtain the standard file information of the search results that are different from the standard file information in the list, and update the new search log Log i (i=1,2,…,g, g is the total number of standard query websites). The parsing module includes the difference list variable G i , check new logs, and sequentially connect the traversal search module, comparison module, replacement module, and difference list variable G i Connected with the comparison module and replacement module respectively.

[0035] The traversal search module is connected to the crawling module; the traversal search module is used to traverse the parsing tree T j Find the tag of the element where the standard file information is located <k>; The traversal search module is also used to parse the tree T j Iterate through all the tags <k>Get the element a with the tag content "to be implemented" i,1 , a i,2 ,…,a i,q (i=1,2,…,g, g is the total number of standard query websites, q is the total number of corresponding elements), get the corresponding standard number string N from its corresponding parent element j,1 , N j,2 ,…,N j,q ; Set the standard number string N j,1 , N j,2 ,…,N j,q Subtract the year information of the standard number in the parent element to get the corresponding string. j + The search term string Str in the search term list j If the same, get string N from it j + and the standard file information of its corresponding parent element; The traversal search module is also used to parse the tree T i Find the tag of the element where the standard file information is located <k>, traverse T j All with the tag <k>Get the element a whose tag content is one of "current", "revoked", and "invalidated" j,1 , a j,2 ,…,a j,q (j=1,2,…,g, g is the search string Str j , q is the total number of elements with the same label that can be found), from its corresponding parent element A j,1 , A j,2 ,…,A j,q Get the corresponding standard number string N j,1 , N j,2 ,…,N j,q , from which to obtain j The same string N j and its corresponding parent element's standard file information.

[0036] The comparison module is connected to the traversal search module and the standard file library respectively; the comparison module is used to convert the search word string Str j Corresponding to the standard file information in the standard file library, and N j or N j + Compare the standard file information of the corresponding parent elements, and store the inconsistent standard file information in the difference list variable G i (i=1, 2, ..., g). In this embodiment, preferably, the standard file information compared in the comparison module includes but is not limited to the standard status, standard number, standard name, standard release date, and standard implementation date.

[0037] The replacement module is connected to the standard file library and the new search log respectively; the replacement module is used to calculate the difference list variable G i Replace the original information of the corresponding standard file and update the new log i ; Replace the module according to the difference list variable G i Replace the original information of the corresponding standard file. The specific steps are as follows: If the format of the standard name and standard number of the standard file information in the list is written incorrectly, the original standard file information in the standard file library will be replaced with the new standard file information, and then the new log will be recorded. i Added logging of modified standard file information; If the standard year of the standard file information in the list is different, the original standard file information in the standard file library will be replaced with the new standard file information, and then the new log will be recorded. i Added log records of abolishing old standard documents and implementing new standard documents; If the difference list variable G i If there is no current standard file information in the standard file library, delete the original standard file information and standard files in the standard file library, and then search for new logs in the Log i Added log records for old standards that have been deprecated.

[0038] The maintenance module is used to merge the new search logs to update the standard file information after the crawler module traverses all standard query websites, and scan the standard files that need to be updated into the standard file library through online download and / or offline purchase to maintain the files and information.

[0039] The maintenance module includes an acquisition module, a format processing module, a stop word list, a text processing module, a data cleaning module, a statistical analysis module, a graph relationship network model, and a classification module.

[0040] The acquisition module is connected to the novelty search log, standard query website, and standard file library respectively; the acquisition module is used to purchase the electronic version through online automatic payment according to the log records of the novelty search log, and / or purchase the paper version offline and scan and enter the standard file that needs to be updated into the standard file library after the crawler module traverses all standard query websites and merges the novelty search log to update the standard file information.

[0041] The format processing module is connected to the standard file library; the format processing module is used to convert all standard files {B1, B2, ..., B m }Batch conversion to plain text format files {P1, P2, ..., P m }.

[0042] The stop word list is used to enter strings of meaningless words.

[0043] The text processing module is connected to the format processing module; the text processing module converts all plain text files {P1, P2, ..., P m } is segmented and marked with parts of speech to form a list format matrix; in this embodiment, the pseg function of the python jieba library is called to convert the plain text file {P1, P2, ..., P m } is segmented, marked with parts of speech, and formed into a list format matrix.

[0044] The data cleaning module is connected to the stop word list and the text processing module respectively; the data cleaning module is used to clean the list format matrix, and after eliminating the string of meaningless words through the stop word list, only the words with the part of speech of noun "n" and the number of words greater than one are extracted. All standard files {B1, B2, ..., B m } form a noun set {H1,H2,…,H m }, where H i For {h i,1 ,h i ,2,…,h i,d }(d is standard file B i The total number of nouns after data cleaning); The statistical analysis module is connected to the data cleaning module; the statistical analysis module is used to perform the following statistical analysis processing steps: (1) Traverse the noun set {H1,H2,…,H m }, and count the corresponding nouns h i,j The term frequency TF and inverse document frequency IDF of (j=1,2,…,d), and then the frequency enhancement index TF-IDF of the corresponding noun is calculated based on the term frequency TF and inverse document frequency IDF i,j , the specific calculation is shown as follows: TF =n i,j / , IDF =log(m / (1+m i,j )) , TF-IDF i,j = TF·IDF·I, Among them, n i,j h i,j In H i The number of occurrences in (i=1,2,…,m), H i All h i,j The sum of the respective times, m is the total number of standard files, m i,j To include h i,j The total number of standard files, I is the set enhancement coefficient (i.e., weight value, which is a constant) used to i,j Perform word enhancement and text position enhancement, log() is a logarithmic function with a base of 10; (2) The frequency enhancement index of the corresponding noun is TF-IDF i,j The values ​​of are sorted from large to small, and no less than the first ten nouns are selected as the keyword set K. i , using all keywords and the frequency enhancement index TF-IDF corresponding to the keywords i,j Composition of keyword matrix M i ={[K i,1 , K i,2 ,…, K i,c ][TF-IDF i,1 , TF-IDF i,2 ,…, TF-IDF i,c ]} (c represents the total number of keywords extracted, K i =[K i,1 , K i,2 ,…, K i,c ]); In this embodiment, the keyword K i The optimal total number is 20, and the keyword set K i = [K i,1 , K i,2 ,…,K i,20 ]; This embodiment further preferably selects standard files and their corresponding K according to a ratio of 5%-10% of the standard file library. i , K i Nouns without special meaning or common high-frequency words in the standard are added to the stop word list; In this embodiment, if there are nouns with significant features in the selected standard text but they do not fall into the K i Then reset the enhancement coefficient I of the noun so that it meets the requirements of entering the keyword set K i conditions.

[0045] The graph relationship network model is connected to the standard file library and the statistical analysis module respectively. The number of nodes in the graph relationship network model is related to the number of standard files in the current standard file library. i The number of nodes is kept consistent, and there is a relationship weight e between each node; preferably, in this embodiment, when the graph relationship network model is just constructed, the relationship weight e between each node is initialized to 0; The classification module is connected to the standard file library, the statistical analysis module, and the graph relationship network model respectively; the classification module performs the following steps to classify the standard files: (1) Traverse all nodes in the graph relationship network model and determine whether there is a keyword in the standard file corresponding to each node that is the same as the keyword in the standard file corresponding to another node. If the same keyword exists, the frequency enhancement index TF-IDF of the two standard files on the keyword is increased. i,j Multiply them together as the increment of the relationship weight e of the two standard files, and then use the incremental relationship weight to form a new relationship weight matrix e[m,m] of the graph relationship network model; (2) Classify the modularity of all standard files in the current standard file library according to the relationship weight matrix e[m,m], aggregate each classification into a new node in the graph relationship network model to generate a relationship network of the new graph relationship network model until the relationship network no longer changes, thereby completing the classification of standard files; In this embodiment, preferably, the modularity classification performs the following steps: (2.1) Generate m categories {F1, F2, …, F m }, each category contains only one standard file; (2.2) Calculate the modularity Q between each standard file and other standard files whose relationship weight is greater than that of the standard file. The calculation is as follows: , Among them, e i,j is the value of the relationship weight between two standard files, e it Standard file B i With a relationship weight greater than B i The sum of the relationship weights of other standard documents, e jt Standard file B j With a relationship weight greater than B j The sum of the relationship weights of other standard files, 2E is the sum of all relationship weights, Indicates B i and B j If they are of the same category, the value is 1, otherwise it is 0; (2.3) For a file containing only standard files B i Classification C i , judge when adding to the category C with the largest modularity j When Is it positive? If the modularity gain If positive, the standard file B i Put it into C j The calculation of is as follows: , in, For B i With C j The sum of the relationship weights of other standard files in , C j The sum of the relationship weights of all standard documents in the original, For C j The sum of the relationship weights of other categories that establish relationships with the node; (2.4) Aggregate each classification into a node to generate a new relationship network; (2.5) Repeat steps S591 to S594 to classify the modularity of different standard files until the relationship network no longer changes; In this embodiment, preferably, step S5 implements the maintenance process through a preset Python maintenance program.

[0046] The message and query interface module is connected to the standard file library; the message and query interface module is also connected to users with new search needs through the human-computer interaction interface; the message and query interface module is used to send a message notification of the completion of the new search to the user and provide a query interface.

[0047] The message and query interface module includes a message module, an optimal standard retrieval query module, and a standard library-image relationship network model display query module. The message module, the optimal standard retrieval query module, and the standard library-image relationship network model display query module are respectively connected to the standard file library and connected to users with new search needs through a human-computer interaction interface.

[0048] The message module is connected with the classification module and the standard file library respectively; the message module is used to confirm that the standard file classification and storage in the standard file library are completed after the classification module completes the module classification, and then send a message to the user that the dynamic update and maintenance of the standard file library is completed.

[0049] The optimal standard search query module is connected to the standard document library to provide the optimal standard search query interface. The optimal standard search includes: according to the search term set {k1, k2, ..., k n }(k j is the search term, j = 1, 2, ..., n), and then search from the maintained standard document library according to the standard document B i (i=1,2,…,m, m is the total number of standard documents) and each search term k j The correlation degree L i (i=1,2,…,n) Output in order from high to low, with a correlation degree of L i According to the frequency enhancement index TF-IDF i,j 、Search term k j Calculate the correlation L i The specific calculation is shown in the following formula: L i = k j TF-IDF i,j ; The standard library graph relationship network model display query module is connected to the standard file library to provide a standard library graph relationship network model display query interface. The standard library graph relationship network model display is used to display the graph relationship network model in the form of a knowledge graph. This embodiment is further preferred to use the neo4j graph database display, and use the graph(), node() and relationship() functions of the py2neo library of python to construct the graph relationship network model. At the same time, a search function is provided to display the position of the retrieved standard file or search term in the knowledge graph, the relationship with the adjacent standards, and the relationship with the classification.

[0050] Compared with the prior art, this embodiment has the following beneficial effects: It can provide real-time and accurate standard document novelty search, as well as batch novelty search of standard documents. It has the functions of regular automatic novelty search, fuzzy query and similarity query, which improves the accuracy of novelty search and avoids the time-consuming and difficult to grasp accuracy problems of manual query and update. It improves the efficiency of construction engineering testing standard document management, enables testing institutions to have a clearer and in-depth grasp of the existing service system, and is also convenient for testing institutions to discover gaps in their own business and their relevance to existing qualifications, and to develop new testing business.

[0051] Example 4 A computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor. When the computer program is executed by the processor, the method for dynamically updating and maintaining a standard file library of a construction engineering inspection organization of Embodiment 1 and / or Embodiment 2 is implemented.

[0052] Example 5 A storage medium is a computer-readable storage medium storing a computer program; when the computer program is executed by a computer, the method for dynamically updating and maintaining a standard file library of a construction engineering inspection organization of Example 1 and / or Example 2 is implemented.

[0053] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.< / k> < / k> < / k> < / k> < / k> < / k> < / k> < / k>

Claims

1. A method for dynamically updating and maintaining a standard document library of a construction engineering inspection organization, characterized in that: The steps include: Generate a list of new standard documents to be checked from the standard document library, and set the standard number of each standard document in the list as a string in the form of a URL; The URL string of the search page of the standard query website is added to the URL format string of each standard number in the list to obtain a search word string, and the search word list of the corresponding standard query website is composed of all the search word strings; Use the search term list to crawl the corresponding standard query website to obtain the parse tree of the search result page corresponding to all the search term strings; Traverse the parsing tree, obtain standard file information of the search results that are different from the standard file information in the list, and update the new search log; Traverse all standard query websites, merge the new query logs to update the standard file information, obtain the standard files that need to be updated into the standard file library, and dynamically classify and maintain the standard files.

2. According to claim 1, the construction engineering inspection agency standard file library dynamic update and maintenance method is characterized in that: The steps of traversing the parsing tree, obtaining standard file information of the search results different from the standard file information in the list, and updating the new search log include: Traverse the tags in the parsed tree and find the tags of the elements where the standard file information is located; Among all the elements with labels in the parse tree, obtain the elements whose label content is one of current, abolished, and invalidated, find the parent element of the corresponding element at the previous level, and obtain the parent element with the same standard number as the standard file; Compare the standard file information in the list with the parent element with the same standard number, and store the inconsistent standard file information in the difference list variable; Replace the original information of the corresponding standard file according to the difference list variables and update the new search log.

3. According to claim 1, the construction engineering inspection agency standard file library dynamic update and maintenance method is characterized in that: The steps of traversing the parsing tree, obtaining standard file information of the search results different from the standard file information in the list, and updating the new search log include: Traverse the tags in the parsed tree and find the tags of the elements where the standard file information is located; Among all the elements with tags in the parse tree, get the element whose tag content is to be implemented, and get the corresponding standard number string from its corresponding parent element; Subtract the year information of the standard number in the parent element from the standard number string to obtain a corresponding string, and when the corresponding string is the same as the search term string in the search term list, obtain the standard file information of the parent element corresponding to the corresponding string; Compare the standard file information in the list with the standard file information obtained from the corresponding parent element, and store the inconsistent standard file information in the difference list variable; Replace the original information of the corresponding standard file according to the difference list variables and update the new search log.

4. The method for dynamically updating and maintaining the standard file library of a construction engineering inspection organization according to claim 2 or 3 is characterized in that: The step of replacing the original information of the corresponding standard file according to the difference list variable includes: If the format of the standard name and standard number of the standard document information in the list is written incorrectly, the original standard document information in the standard document library shall be replaced with the new standard document information; If the standard year of the standard document information in the list is different, the original standard document information in the standard document library will be replaced with the updated standard document information; If there is no current standard file information in the difference list variable, the original standard file information and standard files in the standard file library will be deleted.

5. According to claim 1, the construction engineering inspection agency standard file library dynamic update and maintenance method is characterized in that: When the corresponding standard query website is crawled using the search term list, if the standard query website cannot be directly crawled because it obtains data by interacting with the server and renders a dynamic page, the crawler simulates the behavior of the browser, executes JavaScript code and parses JavaScript to dynamically load data to obtain the parse tree of the search result page.

6. The method for dynamically updating and maintaining the standard file library of a construction engineering inspection organization according to claim 1 is characterized in that: The steps of dynamically classifying and maintaining the standard files include: Convert all standard files in the standard file library into plain text files in batches through image recognition; Create a stop word list to enter strings of meaningless words; Segment the text content in all plain text files and mark the parts of speech to form a list format matrix; Perform data cleaning on the list format matrix, exclude meaningless word strings through the stop word list, and only extract words with noun part of speech and more than one character. Repeat the data cleaning for at least three times to form a noun set. Traverse all words in the noun set, count the word frequency and inverse document frequency of the corresponding nouns, and then calculate the frequency enhancement index of the corresponding noun based on the word frequency and inverse document frequency; Sort the values ​​of the frequency enhancement index of the corresponding nouns from large to small, take no less than the first ten nouns as the keyword set, and use all the keywords and the frequency enhancement indexes corresponding to the keywords to form a keyword matrix; Establish a graph relationship network model, in which the number of nodes is consistent with the number of standard files in the current standard file library, and there are relationship weights between each node; Traverse all nodes in the graph relationship network model, and determine whether there is a keyword in the standard file corresponding to each node, which is the same as the keyword in the standard file corresponding to another node. When the same keyword exists, multiply the frequency enhancement index of the two standard files on the keyword as the increment of the relationship weight, and then use the relationship weight after adding the increment to form a new relationship weight matrix of the graph relationship network model; According to the relationship weight matrix, all standard files in the current standard file library are classified by modularity, and each classification is aggregated into a new node in the graph relationship network model to generate a relationship network of the new graph relationship network model until the relationship network no longer changes, thereby completing the classification of standard files.

7. The method for dynamically updating and maintaining the standard document library of a construction engineering inspection organization according to claim 1 is characterized in that: After the standard files are dynamically classified and maintained, the following steps are also included: Send maintenance completion message to new search users and provide the most suitable standard search query interface; The optimal standard search includes: searching from the maintained standard document library according to the search term set input by the new search user, and outputting the standard documents in descending order of relevance between the standard documents and each search term, wherein the relevance is calculated based on the frequency enhancement index and the search term.

8. A construction engineering inspection agency standard document library dynamic update and maintenance system according to any one of claims 1 to 7, characterized in that: Includes novelty search preparation module, standard file library, crawler module, and maintenance module; The novelty search preparation module is connected to the standard file library and the crawler module respectively, and the crawler module is connected to the standard file library and the maintenance module respectively; The novelty search preparation module is used to generate a list of standard documents to be searched from the standard document library, and set the standard number of each standard document in the list as a string in the form of a URL; The novelty search preparation module is also used to add the URL string of the search page of the standard search website to the URL format string of each standard number in the list to obtain a search word string, and the search word list of the corresponding standard search website is composed of all the search word strings; The crawler module is used to crawl the corresponding standard query website using the search word list to obtain the parse tree of the search result page corresponding to all the search word strings; The crawler module is also used to traverse the parsing tree, obtain standard file information of the search results that are different from the standard file information in the list, and update the new search log; The maintenance module is used to merge the new query logs to update the standard file information after the crawler module traverses all standard query websites, obtain the standard files that need to be updated into the standard file library, and maintain the files and information.

9. A computer device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and executable by the processor; when the computer program is executed by the processor, it implements the dynamic update and maintenance method of the standard file library of the construction engineering inspection agency described in any one of claims 1-7.

10. A storage medium, characterized in that: It is a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the method for dynamically updating and maintaining the standard file library of a construction engineering inspection agency as described in any one of claims 1 to 7.

Images