Production construction project water and soil conservation measure recommendation method based on natural language processing and Apriori algorithm

Abstract

A production and construction project water and soil conservation measure recommendation method based on natural language processing and an Apriori algorithm comprises the following steps: collecting replied production and construction project water and soil conservation scheme report data, and performing text preprocessing to obtain a report text; generating a first embedding vector of each report text; generating a second embedded vector for the production construction project which needs to recommend the water and soil conservation measures at present; indexing the first embedded vectors of all the report texts by using a vectorization retrieval tool, and finding a plurality of report texts most similar to the current production and construction project by matching the first embedded vectors with the second embedded vectors; and screening out the water and soil conservation measure which is most matched with the natural geographical condition of the current region from the similar report texts based on an association rule between the water and soil conservation measure and the natural geographical condition. According to the method, the most matched water and soil conservation measures can be efficiently and accurately recommended for production and construction projects in different areas, and the scientificity and operability of water and soil conservation work are improved.

Description

Technical Field

[0001] The present invention relates to the fields of computer science and soil and water conservation technology, and in particular to a method for recommending soil and water conservation measures for production and construction projects by using an SBERT model, a Faiss (FastAI Similarity Search and Retrieval System) vectorization retrieval tool, and an Apriori algorithm. Background Art

[0002] Soil and water conservation refers to a series of measures to protect agricultural production and the ecological environment by rationally utilizing soil and water resources while protecting land resources and preventing soil erosion. With the continuous intensification of global climate change and human activities, the problem of soil and water loss has become more serious, causing serious impacts on agricultural production, the ecological environment and human life. Therefore, the study and implementation of effective soil and water conservation measures are of great significance to promoting sustainable development.

[0003] Traditional soil and water conservation measures mainly rely on experience and qualitative analysis. These methods can solve the problem of soil erosion to a certain extent, but due to the lack of systematicity and scientificity, they are often difficult to cope with the complex and changing natural environment. In addition, with the advancement of technology and the accumulation of data, how to use big data and intelligent algorithms to make scientific decisions on soil and water conservation measures has become a hot topic and difficulty in current research. Summary of the invention

[0004] The purpose of the present invention is to provide a soil and water conservation measures recommendation method based on the natural language processing model SBERT, the Faiss vectorized retrieval tool and the association rule mining algorithm Apriori, which can recommend the most suitable soil and water conservation measures for different regions through efficient similarity search and association rule mining, thereby overcoming the limitations of traditional methods.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0006] A method for recommending soil and water conservation measures for production and construction projects based on natural language processing and Apriori algorithm includes the following steps:

[0007] Collect the soil and water conservation plan report materials of the production and construction projects that have been approved by the water administration department, and pre-process the soil and water conservation plan report materials to obtain the report text;

[0008] Generate a first embedding vector of each report text using a natural language processing model;

[0009] For the production and construction projects that currently need to recommend soil and water conservation measures, a natural language processing model is used to generate a second embedding vector describing the project type and the natural geographical conditions of the area where the project is located, wherein the natural geographical conditions include topography, climate, soil, and vegetation;

[0010] Use a vectorized search tool to index the first embedding vectors of all report texts, and by matching them with the second embedding vectors, find several report texts that are most similar to the project types and descriptions of the natural geographical conditions of the production and construction projects that currently need to recommend soil and water conservation measures;

[0011] Apriori algorithm is used to mine the association rules between soil and water conservation measures and natural geographical conditions from the most similar report texts;

[0012] Based on the mined association rules, the soil and water conservation measures that best match the natural geographical conditions of the current region are screened out from similar report texts.

[0013] Furthermore, according to the Standard for Soil and Water Loss Prevention and Control in Production and Construction Projects (GB / T50434-2018), the effects of the selected soil and water conservation measures on soil and water loss prevention and control are evaluated by calculating soil and water loss prevention and control indicators; the soil and water loss prevention and control indicators include soil and water loss control degree, soil loss control ratio, slag protection rate, topsoil protection rate, forest and grass vegetation recovery rate and forest and grass coverage rate.

[0014] Furthermore, the natural language processing model is an SBERT model, which is trained for sentence-level tasks to generate word embeddings representing contextual information of sentences or words, and the sentence-level tasks include semantic similarity calculations.

[0015] Furthermore, the vectorized retrieval tool uses Faiss to perform similarity search indexing.

[0016] Furthermore, the text preprocessing of the soil and water conservation program report data includes: word segmentation, stop word removal and stem extraction, wherein word segmentation is to decompose the text into word sequences, stop word removal is to delete meaningless common words in the text, and stem extraction is to restore the words to stem form. The obtained report text is S i ={w1,w2,...,w n}, where wi is the i-th word in the report text, and the preprocessed report text S i Stored in a relational database.

[0017] Furthermore, the step of using a natural language processing model to generate a first embedding vector for each report text includes:

[0018] Step 21: Word embedding layer: Convert the sequence of input text after word segmentation into a format accepted by the machine learning model, and map each word to a fixed-length embedding vector e i :

[0019] e i =SBERT(w i )

[0020] Step 22: Position encoding: Transform the position encoding vector p i Add to each embedding vector e i On the top, we get the word vector x i , to include information about the word's position in the sentence:

[0021] x i =e i +p i

[0022] Step 23: Generate the first embedding vector μ i :Add the word vector sequence {x1,x2,...,x n} is input into the Transformer encoder, which consists of multiple self-attention layers and feedforward neural network layers. The word vector sequence is output as the first embedding vector μ after the Transformer encoder. i :

[0023] μ i = Transformer(x1,x2,...,x n ).

[0024] Furthermore, the vectorized search tool is used to index the first embedding vectors of all report texts, and by matching with the second embedding vectors, several report texts that are most similar to the project type and the description text of the natural geographical conditions of the region of the production and construction project that currently needs to recommend soil and water conservation measures are found, including:

[0025] Step 41: Select the index type: Select memory map index as the similarity search index type. The memory map index speeds up the search by partitioning, and each partition has a representative vector.

[0026] Step 42: Similarity search, by calculating the second embedding vector u2 and the centroid μ of the first embedding vector i The cosine similarity between them is used to perform a similarity search to find the most similar vector to the second embedding vector, thereby finding several reports that are most similar to the current region description.

[0027] Furthermore, the cosine similarity is the vector u2 and μ iThe cosine value of the angle θ between them, cos(θ), is calculated as follows:

[0028]

[0029] Where cos(θ) represents the vector u2 and μ i The cosine value of the angle θ between them. The closer the cosine value is to 1, the more similar the two vectors are; the closer the cosine value is to 0, the less similar the two vectors are.

[0030] Furthermore, the Apriori algorithm is used to mine association rules between soil and water conservation measures and natural geographical conditions from the most similar report texts, including:

[0031] Step 51: Generate frequent itemsets

[0032] (1) Starting from a single item, generate a candidate item set C1 of size 1;

[0033] (2) Scan the database, calculate the support of each candidate item set, and find the frequent item set F1 whose support is greater than or equal to the minimum support threshold;

[0034] (3) Generate candidate item sets through frequent item sets: Using frequent item sets F k Generate a candidate item set C of size k+1 k+1 ;

[0035] (4) Scan the database and calculate each candidate item set C k+1 , find the frequent itemsets F whose support is greater than or equal to the minimum support threshold k+1 ;

[0036] (5) Repeat steps (3) and (4) until no new frequent itemsets can be generated.

[0037] Step 52: Generate association rules

[0038] (2) For each frequent item set F k , generate all possible association rules X→Y, where and Y = F k \X;

[0039] (2) For each association rule, calculate the support and confidence:

[0040] (3) For each frequent item set, generate all possible association rules X→Y and calculate their support and confidence. Record the support and confidence of each rule, set the minimum support and confidence thresholds, and only retain the rules whose support and confidence both meet the minimum support and minimum confidence thresholds to screen out strong association rules.

[0041] Furthermore, step 52 specifically includes:

[0042] Step 521: Calculate support

[0043] Definition: Support refers to the frequency with which an item set appears in all transactions;

[0044] formula:

[0045]

[0046] Step 522: Calculate confidence

[0047] Confidence refers to the conditional probability that Y is also included when X is included in the transaction:

[0048]

[0049] The present invention uses association rules and big data mining methods to accurately recommend soil and water conservation measures based on the characteristics of specific regions, effectively reducing the subjectivity of soil and water conservation analysis and improving the matching degree of soil and water conservation measures. This method can efficiently and accurately recommend the most suitable soil and water conservation measures for production and construction projects in different regions, thereby significantly enhancing the scientificity and operability of soil and water conservation work. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention is a flowchart of a method for recommending soil and water conservation measures for production and construction projects based on natural language processing and Apriori algorithm in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] With the development of big data technology and artificial intelligence, natural language processing technology has shown great potential in text analysis and information extraction. It can extract valuable information from large amounts of text data and automatically analyze text content, thereby providing a basis for decision-making. SBERT (Sentence-BERT) is a natural language processing model for large-scale text analysis. It encodes sentences to generate vector representations of sentences. It has the advantages of sentence-level semantic representation, context awareness, robustness, transferable learning, and wide application. It provides an effective tool and technology for processing natural language processing tasks. SBERT performs well in semantic similarity calculation and is suitable for similarity analysis of large-scale text reports such as soil and water conservation reports.

[0053] Similarity search is an important technology for information retrieval, which aims to quickly find several items similar to the query data from massive data. Traditional similarity search methods mainly rely on keyword matching in texts, which is difficult to accurately reflect the semantic information of texts. With the development of embedding vector technology, text embedding vectors generated by deep learning models can better represent the semantic features of texts, thereby improving the accuracy of similarity search.

[0054] Faiss vectorized retrieval tool is a similarity search tool based on embedded vector technology, which supports indexing and searching of large-scale embedded vectors. Faiss vectorized retrieval tool can be used to build an efficient similarity search index, realize rapid retrieval of a large number of soil and water conservation reports, and find several reports that are most similar to the current regional description, providing a basis for subsequent association rule mining and measure recommendation.

[0055] Apriori algorithm is a classic association rule mining algorithm, which is widely used in data mining, machine learning and other fields. The algorithm mines frequently occurring item sets and association rules by analyzing the association relationship between data item sets. In the field of soil and water conservation, the Apriori algorithm can be used to mine the association rules between soil and water conservation measures and natural geographical conditions (topography, climate, soil, vegetation) from a large number of soil and water conservation reports, providing algorithm support for recommending appropriate measures for specific areas.

[0056] Figure 1 The flowchart of a method for recommending soil and water conservation measures for production and construction projects based on natural language processing and Apriori algorithm according to an embodiment of the present invention is shown. The method comprises the following steps:

[0057] Step 1: Collect a large amount of soil and water conservation plan report materials of production and construction projects that have been approved by the water administration department, and pre-process the soil and water conservation plan report materials to obtain the report text S i ={w1,w2,...,w n}, where wi is the i-th word in the report text.

[0058] The text preprocessing in step 1 includes: word segmentation (breaking the text into sequences of words), stop word removal (removing common meaningless words in the text), and stemming (reducing words to their stem form).

[0059] The preprocessed report text S i Stored in a relational database.

[0060] Step 2: Use the natural language processing model SBERT to generate each report text S i The first embedding vector of . Step 2 specifically includes:

[0061] Step 21: Word embedding layer: Convert the sequence of input text after word segmentation into a format acceptable to the machine learning model, that is, each word is mapped to a fixed-length embedding vector e i :

[0062] e i =SBERT(w i )

[0063] Step 22: Position encoding: Transform the position encoding vector p i Add to each embedding vector e i On the top, we get the word vector x i , to include information about the word's position in the sentence:

[0064] x i =e i +p i

[0065] Step 23: Generate the first embedding vector μ i :Add the word vector sequence {x1,x2,...,x n} is input into the Transformer encoder, which consists of multiple self-attention layers and feedforward neural network layers. The word vector sequence is output as the first embedding vector μ after the Transformer encoder. i :

[0066] μ i = Transformer(x1,x2,...,x n )

[0067] Step 3: For production and construction projects that currently need to recommend soil and water conservation measures, according to the method in step 2, use the natural language processing model SBERT to generate a second embedding vector u2 that describes the project type and the natural geographical conditions of the area where the project is located. The natural geographical conditions include terrain, climate, soil, and vegetation.

[0068] Step 4: Perform similarity search based on the second embedding vector generated in step 3 and the first embedding vector generated in step 2: Use a vectorized retrieval tool to index the first embedding vectors of all report texts, and by matching them with the second embedding vectors, find several report texts that are most similar to the project types and descriptions of the natural geographical conditions of the production and construction projects that currently need to recommend soil and water conservation measures.

[0069] Step 4 specifically includes:

[0070] Step 41: Select the index type. Faiss vectorized retrieval tool provides multiple index types, each of which has different performance and applicable scenarios. Common index types include: Flat Index (FlatIndex), which is suitable for small-scale data sets and directly performs brute force search. Its advantage is high precision because it performs accurate Euclidean distance calculations. Its disadvantage is that as the size of the data set grows, the search speed slows down and the memory consumption is large; Inverted File Index (IVF Index), which is suitable for medium to large-scale data sets and accelerates the search through partitions (i.e. clusters). Each partition has a representative vector (centroid). Its advantage is that it significantly improves the search speed and reduces memory usage while maintaining high precision. The disadvantage is that it takes time to perform clustering training when building the index; HNSW Index (HierarchicalNavigableSmallWorld), which is suitable for large-scale data sets and provides approximate nearest neighbor search based on graph structure. Its advantages are very fast search speed, moderate memory consumption, and high precision. The disadvantage is that it takes more time and memory to build the index.

[0071] According to the data set size and performance requirements, select a suitable index type. In the embodiment of the present invention, the memory mapping index (IVF Index) is selected as the similarity search index type. The IVF index accelerates the search by partitioning (i.e., clustering), and each partition has a representative vector (centroid).

[0072] Step 42: Similarity search, by calculating the second embedding vector u2 and the centroid μ of the first embedding vector i The cosine similarity between them is used to perform a similarity search to find the most similar vector to the second embedding vector, thereby finding several reports that are most similar to the current region description.

[0073] In the embodiment of the present invention, the cosine similarity is the vector u2 and μ i The cosine value of the angle θ between them, cos(θ), is calculated as follows:

[0074]

[0075] Where cos(θ) represents the vector u2 and μ i The cosine value of the angle θ between them. The closer the cosine value is to 1, the more similar the two vectors are; the closer the cosine value is to 0, the less similar the two vectors are.

[0076] Step 5: Use the Apriori algorithm to mine association rules between the soil and water conservation measures and natural geographical conditions (topography, climate, soil, and vegetation) in the report text extracted in step 4.

[0077] Step 5 specifically includes:

[0078] Step 51: Generate frequent item sets: The core idea of ​​the Apriori algorithm is to generate frequent item sets by gradually increasing the size of the item sets. The specific steps include:

[0079] (1) Starting from a single item, generate a candidate item set C1 of size 1.

[0080] (2) Scan the database, calculate the support of each candidate item set, and find the frequent item set F1 whose support is greater than or equal to the minimum support threshold.

[0081] (3) Generate candidate item sets through frequent item sets: Using frequent item sets F k Generate a candidate item set C of size k+1 k+1 . This step is called the "connect" operation.

[0082] (4) Scan the database and calculate each candidate item set C k+1 , find the frequent itemsets F whose support is greater than or equal to the minimum support threshold k+1 .

[0083] (5) Repeat steps (3) and (4) until no new frequent itemsets can be generated.

[0084] Step 52: Generate association rules

[0085] After finding all frequent itemsets, we can then generate association rules from each frequent itemset.

[0086] (1) For each frequent item set F k , we need to generate all possible association rules X→Y (where and Y = F k \X).

[0087] (2) For each association rule, calculate its support and confidence.

[0088] (3) For each frequent item set, all possible association rules X→Y are generated, and their support and confidence are calculated. The support and confidence of each rule are recorded, and usually a minimum support and confidence threshold is set, and only rules whose support and confidence both meet the minimum support and minimum confidence threshold are retained to screen out strong association rules. Step 52 specifically includes:

[0089] Step 521: Calculate support (Support):

[0090] Definition: Support refers to the frequency with which an item set appears in all transactions.

[0091] formula:

[0092]

[0093] Step 522: Calculate confidence: Confidence refers to the conditional probability that Y is also included when X is included in the transaction.

[0094]

[0095] Step 6: According to step 5, the association rules with the largest support and confidence are screened out, and the most suitable soil and water conservation measures for the project are recommended from similar report texts for the areas and specific types of production and construction projects that currently need to be recommended.

[0096] Step 7: Evaluate the effectiveness of the soil and water conservation measures selected in step 6 in preventing and controlling soil erosion.

[0097] Step 7 specifically includes:

[0098] Step 71: Calculate indicators. According to the Standard for Soil and Water Loss Prevention and Control in Production and Construction Projects (GB / T50434-2018), calculate the six major indicators of soil and water loss control degree, soil loss control ratio, slag protection rate, topsoil protection rate, forest and grass vegetation recovery rate, and forest and grass coverage rate.

[0099] Step 72: Based on the calculated indicators, evaluate the effectiveness of the recommended soil and water conservation measures on preventing and controlling soil erosion, which will be used by the project construction unit and the design unit to decide whether the recommended soil and water conservation measures can be included in the final design plan.

[0100] The present invention proposes an efficient and accurate method for recommending soil and water conservation measures by combining the SBERT model, the Faiss vectorization retrieval tool and the Apriori algorithm. The method recommends scientific and reasonable soil and water conservation measures for different regions through the steps of data collection and preprocessing, embedding vector generation, regional feature input, similarity search, association rule mining and measure recommendation. The method can improve the accuracy and efficiency of soil and water conservation measure recommendations and has broad application prospects and practical value.

[0101] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for recommending soil and water conservation measures for production and construction projects based on natural language processing and Apriori algorithm, characterized in that: The following steps are involved: Collect the soil and water conservation plan report materials of the production and construction projects that have been approved by the water administration department, and pre-process the soil and water conservation plan report materials to obtain the report text; Generate a first embedding vector for each report text using a natural language processing model; For the production and construction projects that currently need to recommend soil and water conservation measures, a natural language processing model is used to generate a second embedding vector describing the project type and the natural geographical conditions of the area where the project is located, wherein the natural geographical conditions include topography, climate, soil, and vegetation; Use a vectorized search tool to index the first embedding vectors of all report texts, and by matching them with the second embedding vectors, find several report texts that are most similar to the project types and descriptions of the natural geographical conditions of the production and construction projects that currently need to recommend soil and water conservation measures; Apriori algorithm is used to mine the association rules between soil and water conservation measures and natural geographical conditions from the most similar report texts; Based on the mined association rules, the soil and water conservation measures that best match the natural geographical conditions of the current region are screened out from similar report texts.

2. The method according to claim 1, characterized in that: Also includes: According to the Standard for Soil and Water Loss Prevention and Control in Production and Construction Projects (GB / T 50434-2018), the effects of the selected soil and water conservation measures on soil and water loss prevention and control are evaluated by calculating soil and water loss prevention and control indicators; the soil and water loss prevention and control indicators include soil and water loss control degree, soil loss control ratio, slag protection rate, topsoil protection rate, forest and grass vegetation recovery rate and forest and grass coverage rate.

3. The method according to claim 1, characterized in that The natural language processing model is an SBERT model, which is trained for sentence-level tasks to generate word embeddings representing contextual information of sentences or words. The sentence-level tasks include semantic similarity calculation.

4. The method according to claim 1, characterized in that The vectorized retrieval tool uses Faiss to perform similarity search indexing.

5. The method according to claim 4, characterized in that The text preprocessing of the soil and water conservation program report data includes: word segmentation, stop word removal and stem extraction, wherein word segmentation is to decompose the text into word sequences, stop word removal is to delete common meaningless words in the text, and stem extraction is to restore the words to stem form. The obtained report text is S i ={w1,w2,...,w n }, where w i is the i-th word in the report text, and the preprocessed report text S i Stored in a relational database.

6. The method according to claim 5, characterized in that The step of using a natural language processing model to generate a first embedding vector for each report text includes: Step 21: Word embedding layer: Convert the sequence of input text after word segmentation into a format accepted by the machine learning model, and map each word to a fixed-length embedding vector e i : e i SBERT(w i ) Step 22: Position encoding: Transform the position encoding vector p i Add to each embedding vector e i On the top, we get the word vector x i , to include information about the word's position in the sentence: x i =and i +p i Step 23: Generate the first embedding vector μ i :Add the word vector sequence {x1,x2,...,x n } is input into the Transformer encoder, which consists of multiple self-attention layers and feedforward neural network layers. The word vector sequence is output as the first embedding vector μ after the Transformer encoder. i : μ i =Transform(x1,x2,...,x n ) 7. The method according to claim 6, characterized in that The vectorized search tool is used to index the first embedded vectors of all report texts, and by matching with the second embedded vectors, a number of report texts that are most similar to the project type and the description text of the natural geographical conditions of the region of the production and construction project that currently needs to recommend soil and water conservation measures are found, including: Step 41: Select the index type: Select memory map index as the similarity search index type. The memory map index speeds up the search by partitioning, and each partition has a representative vector. Step 42: Similarity search, by calculating the second embedding vector u2 and the centroid μ of the first embedding vector i The cosine similarity between them is used to perform a similarity search to find the most similar vector to the second embedding vector, thereby finding several reports that are most similar to the current region description.

8. The method according to claim 7, characterized in that The cosine similarity is the vector u2 and μ i The cosine value of the angle θ between them, cos(θ), is calculated as follows: Where cos(θ) represents the vector u2 and μ i The cosine value of the angle θ between them. The closer the cosine value is to 1, the more similar the two vectors are; the closer the cosine value is to 0, the less similar the two vectors are.

9. The method according to claim 1, characterized in that: The first embedding vector generation formula is as follows: The Apriori algorithm is used to mine association rules between soil and water conservation measures and natural geographical conditions from the most similar report texts, including: Step 51: Generate frequent itemsets (1) Starting from a single item, generate a candidate item set C1 of size 1; (2) Scan the database, calculate the support of each candidate item set, and find the frequent item set F1 whose support is greater than or equal to the minimum support threshold; (3) Generate candidate item sets through frequent item sets: Using frequent item sets F k Generate a candidate item set C of size k+1 k+1 ; (4) Scan the database and calculate each candidate item set C k+1 , find the frequent itemsets F whose support is greater than or equal to the minimum support threshold k+1 ; (5) Repeat steps (3) and (4) until no new frequent itemsets can be generated. Step 52: Generate association rules (1) For each frequent item set F k , generate all possible association rules X→Y, where and Y = F k \X; (2) For each association rule, calculate the support and confidence: (3) For each frequent item set, generate all possible association rules X→Y and calculate their support and confidence. Record the support and confidence of each rule, set the minimum support and confidence thresholds, and only retain the rules whose support and confidence both meet the minimum support and minimum confidence thresholds to screen out strong association rules.

10. The method according to claim 9, characterized in that Step 52 specifically includes: Step 521: Calculate support Definition: Support refers to the frequency with which an item set appears in all transactions; formula: Step 522: Calculate confidence Confidence refers to the conditional probability that Y is also included when X is included in the transaction:

Images