A semantic understanding method and system based on agricultural field text
By building semantic understanding methods and systems in the agricultural field, the problems of supply and demand information matching, intelligent question-and-answer and literature retrieval are solved, real-time updates and accuracy of data are achieved, and production efficiency is improved.
Patent Information
- Application Number
- CN202111203860.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-15
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-10-15
AI Technical Summary
The matching of supply and demand information, intelligent Q&A and literature search results in the agricultural field are inaccurate, and the data is slow to be updated, resulting in the data not being updated for many years.
Provide a semantic understanding method and system based on text in the agricultural field. By obtaining text data, it performs word segmentation, part-of-speech annotation, and entity processing, and constructs basic knowledge graphs and semantic knowledge graphs, combining word segmentation annotation model, entity recognition model and semantic recognition model. Achieve information matching, intelligent question-and-answer and literature retrieval, and solve problems through iterative updates and real-time knowledge graph updates.
It improves the accuracy and real-time nature of data in the agricultural field, improves the efficiency of supply and demand information matching, intelligent question-and-answer and literature retrieval, reduces human resource consumption, and realizes iterative optimization of data.
Smart Images

Figure CN113869066B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing, and more specifically, to a semantic understanding method and system based on agricultural field text. Background Art
[0002] With the rapid development of natural language processing technology in various fields, there is an urgent need to build natural language processing technology in the agricultural field.
[0003] The results of existing technologies regarding supply and demand information matching, intelligent question and answer, and literature retrieval in the agricultural field are inaccurate, and the updating of data in the agricultural field is also extremely slow, resulting in the current data on the agricultural field being still in the state of many years ago.
[0004] In view of the characteristics of the agricultural field, such as a wide variety of knowledge, strong heterogeneous correlation, and relatively fixed language sentence patterns, the present invention provides a semantic understanding method and system for agricultural vocabulary based on the natural language characteristics and underlying applications in the agricultural field. Summary of the invention
[0005] The purpose of the present invention is to provide a semantic understanding method and system based on agricultural text to solve the problem of inaccurate results of supply and demand information matching, intelligent question answering, and literature retrieval in the agricultural field.
[0006] The above technical objectives of the present invention are achieved through the following technical solutions:
[0007] In a first aspect, the present invention provides a semantic understanding method based on agricultural field text, the method comprising the following steps:
[0008] Step 1: Obtain text data in the agricultural field;
[0009] Step 2: segment and tag the text data, and perform entity processing on the text data after segmentation and tagging according to the context information of the text data;
[0010] Step 3: construct a basic knowledge graph of homologous text data and a semantic knowledge graph of heterologous text data;
[0011] Step 4: Combining the method for processing text data in step 2 to form a word segmentation and annotation model, an entity recognition model, and a semantic recognition model;
[0012] Step five: iteratively update the text data and update the knowledge graph in real time.
[0013] The present invention performs word segmentation on text data in the agricultural field and marks the parts of speech of vocabulary, and then constructs a basic knowledge graph of homologous text data and a semantic knowledge graph of heterologous text data from the text data after word segmentation and part of speech marking. The present invention proposes a combination of basic natural language processing methods to output in the form of composite models such as word segmentation and marking models, entity recognition models, and semantic recognition models to achieve information matching, intelligent question answering, and literature retrieval for data in the agricultural field. By iteratively updating the data and updating the knowledge graph in real time, the real-time update of the text data and the supplementation or error correction of the knowledge graph are completed.
[0014] Furthermore, in step 2, natural language processing is performed on the text data; wherein natural language processing includes word segmentation, keyword extraction, topic extraction, summary, event extraction and semantic extraction;
[0015] Perform entity processing on text data after natural language processing; entity processing includes part-of-speech tagging, entity recognition, semantic disambiguation, role labeling, coreference resolution and entity alignment.
[0016] Furthermore, in the step three, the text data is entity aligned, and the aligned text data is imported into the basic graph to generate a basic knowledge graph; wherein the attributes of multiple entities in the text data are compared for similarity, and a similarity threshold is set, and when the similarity reaches a certain similarity threshold, the text data is considered to belong to the same entity;
[0017] All atoms in the basic knowledge graph are quantified, and the quantified data are processed using the natural language translation algorithm, path learning algorithm, type constraint algorithm and heterogeneous information algorithm to generate a semantic knowledge graph.
[0018] Furthermore, in the step 4, all parts of speech in the text data are marked, including word segmentation, part of speech tagging and role tagging of the text data;
[0019] Identify all entities in text data, including word segmentation, keyword extraction, entity recognition, and coreference resolution;
[0020] Determine the semantics expressed by text data, including summarizing, event extraction, semantic extraction, topic extraction, keyword extraction, and semantic disambiguation of text data.
[0021] Furthermore, in step 5, the errors existing in the existing data are modified, and the behavior data of the modification action is recorded, and the behavior data is processed and converted into trainable data to achieve iterative training of the text data;
[0022] Realize downstream tasks such as supply and demand information matching, intelligent question answering, and patent literature retrieval in the agricultural field;
[0023] Test and annotate the generated text data;
[0024] Add, modify and delete the knowledge graph to complete the real-time update of the knowledge graph.
[0025] In a second aspect, the present invention further provides a semantic understanding system based on agricultural field text, which is used to implement the semantic understanding method described in the first aspect. The system includes a data processing layer and a data application layer. The data processing layer includes a data acquisition unit, a data processing unit and a graph construction unit. The data application layer includes a data combination unit and a data application unit. Its functions are specifically implemented as follows:
[0026] A data acquisition unit, used for acquiring text data in the agricultural field;
[0027] A data processing unit, used to perform word segmentation and part-of-speech tagging on text data, and perform entity processing on the text data after word segmentation and part-of-speech tagging according to context information of the text data;
[0028] A graph construction unit, which is used to construct a basic knowledge graph for homologous text data and a semantic knowledge graph for heterologous text data;
[0029] A data combination unit, used for combining the method of processing text data by the data processing unit to form a word segmentation and annotation model, an entity recognition model and a semantic recognition model;
[0030] The data application unit is used to iteratively update text data and update the knowledge graph in real time.
[0031] Furthermore, the data processing unit includes a first processing unit and a second processing unit, which are specifically implemented as follows:
[0032] The first processing unit is used to perform natural language processing on the text data; wherein the natural language processing includes word segmentation, keyword extraction, topic extraction, summary, event extraction and semantic extraction;
[0033] The second processing unit is used to perform entity processing on the text data after natural language processing; wherein the entity processing includes part-of-speech tagging, entity recognition, semantic disambiguation, role labeling, reference resolution and entity alignment.
[0034] Furthermore, the graph construction unit includes a basic knowledge graph construction unit and a semantic knowledge graph construction unit, which are specifically implemented as follows:
[0035] The basic knowledge graph construction unit is used to perform entity alignment on the text data, import the aligned text data into the basic graph, and generate a basic knowledge graph; wherein the attributes of multiple entities in the text data are compared for similarity, a similarity threshold is set, and when the similarity reaches a certain similarity threshold, the text data is considered to belong to the same entity;
[0036] The semantic knowledge graph construction unit is used to quantify all atoms in the basic knowledge graph, and process the quantified data using the translation algorithm, path learning algorithm, type constraint algorithm and heterogeneous information algorithm in natural language to generate a semantic knowledge graph.
[0037] Furthermore, the word segmentation and tagging model is used to tag all parts of speech in the text data, including word segmentation, part of speech tagging and role tagging of the text data;
[0038] The entity recognition model is used to identify all entities in text data, including word segmentation, keyword extraction, entity recognition and coreference resolution of text data;
[0039] The semantic recognition model is used to determine the semantics expressed by text data, including summarizing, event extraction, semantic extraction, topic extraction, keyword extraction and semantic disambiguation of text data.
[0040] Furthermore, the data combination unit also forms a click model, and the data application unit includes an AI interface, a basic function interface, a graph query interface, and an error correction and reasoning interface;
[0041] The error correction and reasoning interface is used to modify errors in existing data, record the behavior data of the modification action, process the behavior data and convert it into data that can be trained by the click model, and realize iterative training of the click model;
[0042] The AI interface includes word segmentation and annotation models, entity recognition models, and semantic recognition models, which are used to achieve downstream tasks such as supply and demand information matching, intelligent question answering, and patent document retrieval in the agricultural field;
[0043] Basic functional interface, used to test and annotate text data generated by the data processing unit;
[0044] The graph query interface is used to add, modify, and delete the knowledge graph in the graph construction unit to complete the real-time update of the knowledge graph.
[0045] Compared with the prior art, the present invention has the following beneficial effects:
[0046] The present invention is used to serve specific downstream tasks such as supply and demand information matching, intelligent question and answer, and patent document retrieval in agriculture. All algorithms for processing data in the agricultural field are opened in the form of interfaces through basic functional interfaces. The advantage of opening them as functions is that data testing and new data annotation can be processed using these basic functional interfaces first, and then the data correctness can be checked manually, thereby improving production efficiency and reducing the consumption of human resources. The knowledge graph can be added, modified, deleted, and other operations can be performed through the graph query interface. The open interface form is convenient to operate and can be modified in real time. The error correction and reasoning interface is a click model presentation method. In the form of an interface, not only can errors in existing data be modified, but the behavioral data of the modification action can also be recorded at the same time. After data processing, it is converted into data that can be trained by the model, which can realize iterative optimization of data. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:
[0048] Figure 1 A flowchart of a semantic understanding method provided by an embodiment of the present invention;
[0049] Figure 2 An architectural diagram of a semantic understanding system provided by an embodiment of the present invention;
[0050] Figure 3 A flowchart of obtaining text data provided by an embodiment of the present invention;
[0051] Figure 4 A flow chart of constructing a word segmentation and annotation model provided in one embodiment of the present invention;
[0052] Figure 5 A flow chart of constructing an entity recognition model provided by an embodiment of the present invention;
[0053] Figure 6 A flowchart of constructing a semantic recognition model provided by an embodiment of the present invention;
[0054] Figure 7 A flowchart of constructing a basic knowledge graph provided by an embodiment of the present invention; DETAILED DESCRIPTION
[0055] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.
[0056] It should be noted that when a component is referred to as being "fixed to" or "disposed on" another component, it can be directly on the other component or indirectly on the other component. When a component is referred to as being "connected to" another component, it can be directly or indirectly connected to the other component.
[0057] It should be understood that the orientation or position relationship indicated by terms such as "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0058] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0059] Example
[0060] like Figure 1 As shown, this embodiment provides a semantic understanding method based on agricultural field text, and the method includes the following steps:
[0061] Step 1: Obtain text data in the agricultural field;
[0062] Step 2: segment and tag the text data, and perform entity processing on the text data after segmentation and tagging according to the context information of the text data;
[0063] Step 3: construct a basic knowledge graph of homologous text data and a semantic knowledge graph of heterologous text data;
[0064] Step 4: Combining the method for processing text data in step 2 to form a word segmentation and annotation model, an entity recognition model, and a semantic recognition model;
[0065] Step five: iteratively update the text data and update the knowledge graph in real time.
[0066] Specifically, Figure 3As shown, in step 1, the data types of text data can be divided into text data, behavior data, encyclopedia data, and core word data. Text data mainly comes from open forum comment information, news information, supply and demand order information and other data on the Internet. This type of data can be directly obtained through data crawling tools, and then the obtained data set is cleaned and sorted. Behavioral data comes from the operator behavior log data generated by the click model in the core engine layer; encyclopedia data can use official open source data such as Baidu Encyclopedia and Wikipedia. The advantage of this type of data is that the content is detailed and the accuracy is high; core word data refers to proper nouns in the agricultural field or words with special meanings in the field. This type of data is provided by agricultural experts. Because agricultural experts have been deeply engaged in this field for many years, the core words keep pace with the times and are more suitable for keyword retrieval in practical applications.
[0067] In step 2, the processing of text data can be divided into two categories: NLP processing and NLP entity processing. Word segmentation and part-of-speech tagging belong to NLP processing, and entity processing belongs to NLP entity processing. Among them, natural language processing is referred to as NLP processing, and NLP processing includes word segmentation, keyword extraction, topic extraction, summary, event extraction, and semantic extraction; NLP entity processing includes part-of-speech tagging, entity recognition, semantic disambiguation, role labeling, reference resolution, and entity alignment.
[0068] In step three, agriculture can be broadly divided into five categories: planting, fishery, animal husbandry, forestry, and sideline industries. Since there are connections between categories, the agricultural industry is rich in knowledge and the relationships are complex. It needs to use knowledge graphs to assist semantic understanding and reasoning. The basic knowledge graph is mainly constructed by agricultural experts to ensure the accuracy and authority of the graph. One of the major characteristics of the agricultural field is that there is a certain relationship between animals and plants. The knowledge graph of animals and plants belongs to heterogeneous knowledge graphs in the field of natural language. The basic knowledge graph cannot satisfy the reasoning of heterogeneous knowledge graphs. The heterogeneous knowledge graph needs to be translated in natural language, path learning, type constraints, and heterogeneous information algorithm processing to discover and infer possible hidden relationships and combine heterogeneous information to form a semantic knowledge graph.
[0069] In step 4, the text data processed in step 2 is output through three models to complete information matching, intelligent question answering and literature retrieval.
[0070] In step five, as agriculture keeps pace with the times, data needs to be updated and corrected, and new text data needs to be tagged with parts of speech to complete subsequent information matching and searching.
[0071] A further embodiment of this embodiment performs natural language processing and natural language entity processing on the text, wherein in step 2, natural language processing is performed on the text data; wherein the natural language processing includes word segmentation, keyword extraction, topic extraction, summary, event extraction and semantic extraction;
[0072] Perform entity processing on text data after natural language processing; entity processing includes part-of-speech tagging, entity recognition, semantic disambiguation, role labeling, coreference resolution and entity alignment.
[0073] A further embodiment of this embodiment is to construct a knowledge graph for text data, such as Figure 7 As shown, in the step three, the text data is entity aligned, and the aligned text data is imported into the basic graph to generate a basic knowledge graph; wherein, the attributes of multiple entities in the text data are compared for similarity, and a similarity threshold is set. When the similarity reaches a certain similarity threshold, it is considered that the text data belongs to the same entity;
[0074] All atoms in the basic knowledge graph are quantified, and the quantified data are processed using the natural language translation algorithm, path learning algorithm, type constraint algorithm and heterogeneous information algorithm to generate a semantic knowledge graph.
[0075] Specifically, the data is aligned before being imported into the basic graph, because most plants and animals in agriculture have nicknames, and the entity alignment algorithm compares the similarities of the attributes of multiple entities. When the similarity reaches a certain threshold, they are considered to be the same entity.
[0076] The basic knowledge graph can be constructed by experts in the agricultural field. The advantage of constructing the basic knowledge graph by experts is that they have been deeply involved in the field for a long time and have a role in keeping up with the times in the knowledge of the field, which can effectively avoid some relationships that no longer exist or are rarely used. Since experts control the construction of the basic knowledge graph, the accuracy and authority of the graph can be guaranteed, which is conducive to the construction of the semantic graph. The knowledge graph is composed of countless triples. All atoms in the basic graph are quantified, and the quantified data is processed by the translation algorithm, path learning algorithm, type constraint algorithm, and heterogeneous information algorithm in natural language to generate a semantic knowledge graph. The essence of the semantic knowledge graph is to fuse heterogeneous knowledge graphs. Because animals and plants belong to heterogeneous knowledge graphs, although the knowledge of another field is mentioned in the leaves of the knowledge tree, the two are still scattered. The semantic knowledge graph integrates heterogeneous knowledge graphs through algorithms to form a closed loop of the field. The advantage of the closed loop of the field is that if accurate data is not found in the lower-level application, it can provide highly correlated information for reasoning based on the existing information. For example, if no relevant information about "pest and disease control in apple planting" is found at the beginning of the application search, the properties of apples will be used to infer the prevention and control of pests and diseases in fruits with similar properties, and then the discovered pests will be used to reversely infer the plants they will affect. It is possible to find that the pests also affect apples.
[0077] In a further embodiment of this embodiment, the method of processing text data is combined to construct three models to realize data output, such as Figure 4 , 5 As shown in 6, in the step 4, all parts of speech in the text data are marked, including word segmentation, part of speech tagging and role tagging of the text data;
[0078] Identify all entities in text data, including word segmentation, keyword extraction, entity recognition, and coreference resolution;
[0079] Determine the semantics expressed by text data, including summarizing, event extraction, semantic extraction, topic extraction, keyword extraction, and semantic disambiguation of text data.
[0080] Specifically, Figure 4This is the process of constructing the word segmentation and annotation model. The text is first segmented into words of varying lengths by a word segmentation algorithm. Then, the words are segmented by a part-of-speech tagging algorithm to determine the part-of-speech of all words. Then, additional information is added to some words through role tagging, such as location, time and other attributes. In the agricultural field, these words are all key information. Adding a role tagging algorithm can promote relational reasoning and information retrieval in applications. For example, the sentence "pests and diseases that need to be prevented and controlled for citrus in summer" will be segmented into "summer", "citrus", "prevention and control", and "pests and diseases". In the process of part-of-speech tagging, "summer" will be marked as a noun part-of-speech. At this time, it is necessary to input additional "time" information for this word through role tagging, because time plays an indispensable role in the agricultural field. Both plant planting and animal breeding are very sensitive to time.
[0081] Figure 5 This is the process of constructing the entity recognition model. The text is first segmented into words of varying lengths using a word segmentation algorithm, and the keyword extraction algorithm is used to filter out verbs, adjectives, etc. in the words. The filtering operation can shorten the processing time, and then all entity nouns in the text are confirmed through entity recognition, because in the agricultural field, words such as "planting" and "breeding" can be both verbs and nouns; or relational nouns such as "parent class" and "child class" can be distinguished through entity recognition. In the agricultural field, entities can only be "Saigon banana" and "teddy dog" with clear references. Then, entity alignment is performed through the reference resolution algorithm. The entity alignment of reference resolution is an alignment of entities and references. For example, "Apple sued Qualcomm, but Qualcomm did not respond", in which "Qualcomm" and "Qualcomm" are the relationship between entities and references, and the resolution algorithm forms an equivalent set of the two.
[0082] Figure 6 This is the process of constructing a semantic recognition model. The semantic recognition algorithm is mainly used for large text data. The text first passes through the summary and event extraction processing algorithms to "slim down" the data and extract key sentences. The key sentences are subjected to the semantic extraction algorithm to obtain all possible semantic sentences. The refined text is then subjected to topic extraction and keyword processing operations in order to use keywords, topics and other information to provide auxiliary analysis information for text semantics. The extracted keywords, topic information and all possible semantic sentences are subjected to the semantic disambiguation algorithm. The semantic disambiguation algorithm will score and rank all sentences, and the first ranked will be considered the most likely semantics.
[0083] A further embodiment of this embodiment implements iterative training and error correction of text data. In step 5, errors in existing data are modified, and behavioral data of the modification action is recorded, and the behavioral data is processed and converted into trainable data to implement iterative training of text data.
[0084] Realize downstream tasks such as supply and demand information matching, intelligent question answering, and patent literature retrieval in the agricultural field;
[0085] Test and annotate the generated text data;
[0086] Add, modify and delete the knowledge graph to complete the real-time update of the knowledge graph.
[0087] Specifically, it realizes specific downstream tasks such as supply and demand information matching, intelligent question and answer, and literature retrieval in the agricultural field. These basic functions can be used to process data testing and new data annotation, and then manually check the correctness of the data, which can improve production efficiency and reduce the consumption of human resources. Real-time modification of graph addition, modification, and deletion functions. Modify errors in existing data, and at the same time record the behavioral data of the modification action, which is converted into model-trainable data through data processing, and iteratively optimize the click model.
[0088] This embodiment also provides a semantic understanding system based on agricultural field text, such as Figure 2 As shown, to implement the above-mentioned semantic understanding method, the system includes a data processing layer and a data application layer, the data processing layer includes a data acquisition unit 110, a data processing unit 120 and a graph construction unit 130, and the data application layer includes a data combination unit 140 and a data application unit 150; its functions are specifically implemented as follows:
[0089] A data acquisition unit 110 is used to acquire text data in the agricultural field;
[0090] The data processing unit 120 is used to perform word segmentation and part-of-speech tagging on the text data, and perform entity processing on the text data after word segmentation and part-of-speech tagging according to the context information of the text data;
[0091] A graph construction unit 130, used to construct a basic knowledge graph for homologous text data and a semantic knowledge graph for heterologous text data;
[0092] A data combination unit 140, used to combine the method of processing text data by the data processing unit to form a word segmentation and annotation model, an entity recognition model and a semantic recognition model;
[0093] The data application unit 150 is used to iteratively update the text data and update the knowledge graph in real time.
[0094] In another embodiment of this embodiment, the data processing unit 120 includes a first processing unit and a second processing unit, which are specifically implemented as follows:
[0095] The first processing unit is used to perform natural language processing on the text data; wherein the natural language processing includes word segmentation, keyword extraction, topic extraction, summary, event extraction and semantic extraction;
[0096] The second processing unit is used to perform entity processing on the text data after natural language processing; wherein the entity processing includes part-of-speech tagging, entity recognition, semantic disambiguation, role labeling, reference resolution and entity alignment.
[0097] Specifically, the NLP processing of text data and NLP entity processing have been described in the above-mentioned semantic understanding method and will not be described here.
[0098] In another embodiment of this embodiment, the graph construction unit 130 includes a basic knowledge graph construction unit and a semantic knowledge graph construction unit, which are specifically implemented as follows:
[0099] The basic knowledge graph construction unit is used to perform entity alignment on the text data, import the aligned text data into the basic graph, and generate a basic knowledge graph; wherein the attributes of multiple entities in the text data are compared for similarity, a similarity threshold is set, and when the similarity reaches a certain similarity threshold, the text data is considered to belong to the same entity;
[0100] The semantic knowledge graph construction unit is used to quantify all atoms in the basic knowledge graph, and process the quantified data using the translation algorithm, path learning algorithm, type constraint algorithm and heterogeneous information algorithm in natural language to generate a semantic knowledge graph.
[0101] Specifically, the construction of the two knowledge graphs has been described in the above semantic understanding method and will not be described here.
[0102] In another embodiment of this embodiment, the word segmentation and tagging model is used to tag all parts of speech in the text data, including word segmentation, part of speech tagging and role tagging of the text data;
[0103] The entity recognition model is used to identify all entities in text data, including word segmentation, keyword extraction, entity recognition and coreference resolution of text data;
[0104] The semantic recognition model is used to determine the semantics expressed by text data, including summarizing, event extraction, semantic extraction, topic extraction, keyword extraction and semantic disambiguation of text data.
[0105] Specifically, the data combination unit 140 includes: a rule model and a matching model. The rule model includes a word segmentation and annotation model; the matching model includes an entity recognition model and a semantic recognition model. The functions and effects of the word segmentation and annotation model, the entity recognition model and the semantic recognition model have been described in the above-mentioned embodiment of the semantic understanding method and will not be described here.
[0106] In another embodiment of this embodiment, the data combination unit 140 further forms a click model, and the data application unit includes an AI interface, a basic function interface, a graph query interface, and an error correction and reasoning interface;
[0107] The error correction and reasoning interface is used to modify errors in existing data, record the behavior data of the modification action, process the behavior data and convert it into data that can be trained by the click model, and realize iterative training of the click model;
[0108] The AI interface includes word segmentation and annotation models, entity recognition models, and semantic recognition models, which are used to achieve downstream tasks such as supply and demand information matching, intelligent question answering, and patent document retrieval in the agricultural field;
[0109] Basic functional interface, used to test and annotate text data generated by the data processing unit;
[0110] The graph query interface is used to add, modify, and delete the knowledge graph in the graph construction unit to complete the real-time update of the knowledge graph.
[0111] Specifically, the click model in the data combination unit 140 is the source of behavior data. The click model records the operation behavior data in the form of a log. The behavior data is collected according to several dimensions such as id, user name, search statement, and click content. The search statement data is used as the optimization model data, and the remaining data is saved to the database in a regular manner. This part of the data can be used as the basic data for intelligent recommendation to analyze user behavior. The main function of the data application unit 150 is to open the function in the form of a web interface. The AI interface covers the rule model and matching model of the data combination unit 140, serving specific downstream tasks such as supply and demand information matching, intelligent question and answer, and patent document retrieval in agriculture. The basic function interface is to open all algorithms in the data processing unit 120 in the form of an interface. The advantage of opening it as a function is that the data test and the annotation of new data can be processed using these basic functions first, and then the data correctness can be checked manually, which can improve production efficiency and reduce the consumption of human resources. The graph query interface includes functions such as adding, modifying, and deleting the graph. The interface form is open for easy operation and can be modified in real time. The error correction and reasoning interface is a way of presenting the click model. The interface can not only modify errors in existing data, but also record the behavioral data of the modification action. After data processing, it is converted into data that can be trained by the model, and the click model is iteratively optimized.
[0112] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A semantic understanding method based on agricultural text, characterized in that: The method comprises the following steps: Step 1: Obtain text data in the agricultural field; Step 2: segment and tag the text data, and perform entity processing on the text data after segmentation and tagging according to the context information of the text data; Step three, constructing a basic knowledge graph of homologous text data and a semantic knowledge graph of heterogeneous text data; wherein, in said step three, entity alignment is performed on the text data, and the aligned text data is imported into the basic graph to generate a basic knowledge graph; wherein, the attributes of multiple entities in the text data are compared for similarity, and a similarity threshold is set, and when the similarity reaches a certain similarity threshold, the text data is considered to belong to the same entity; all atoms in the basic knowledge graph are quantified, and the quantified data is processed by a translation algorithm, a path learning algorithm, a type constraint algorithm and a heterogeneous information algorithm in natural language to generate a semantic knowledge graph; Step 4, combining the method for processing text data in step 2 to form a word segmentation and annotation model, an entity recognition model and a semantic recognition model; wherein, in step 4, all parts of speech in the text data are annotated, including word segmentation, part-of-speech tagging and role tagging of the text data; all entities in the text data are identified, including word segmentation, keyword extraction, entity recognition and reference resolution of the text data; the semantics expressed by the text data are determined, including summarization, event extraction, semantic extraction, topic extraction, keyword extraction and semantic disambiguation of the text data; Step five: iteratively update the text data and update the knowledge graph in real time.
2. A semantic understanding method based on agricultural field text according to claim 1, characterized in that: In the step 2, natural language processing is performed on the text data; wherein the natural language processing includes word segmentation, keyword extraction, topic extraction, summary, event extraction and semantic extraction; Perform entity processing on text data after natural language processing; entity processing includes part-of-speech tagging, entity recognition, semantic disambiguation, role labeling, coreference resolution and entity alignment.
3. The semantic understanding method based on agricultural field text according to claim 1 is characterized in that: In the step 5, the errors existing in the existing data are modified, and the behavior data of the modification action is recorded, and the behavior data is processed and converted into trainable data to achieve iterative training of the text data; Realize downstream tasks such as supply and demand information matching, intelligent question answering, and patent literature retrieval in the agricultural field; Test and annotate the generated text data; Add, modify and delete the knowledge graph to complete the real-time update of the knowledge graph.
4. A semantic understanding system based on agricultural text, characterized in that: To implement the semantic understanding method according to any one of claims 1 to 3, the system includes a data processing layer and a data application layer, the data processing layer includes a data acquisition unit, a data processing unit and a graph construction unit, and the data application layer includes a data combination unit and a data application unit, and its functions are specifically implemented as follows: A data acquisition unit, used for acquiring text data in the agricultural field; A data processing unit, used to perform word segmentation and part-of-speech tagging on text data, and perform entity processing on the text data after word segmentation and part-of-speech tagging according to context information of the text data; A graph construction unit is used to construct a basic knowledge graph for homologous text data and a semantic knowledge graph for heterogeneous text data; wherein, the graph construction unit includes a basic knowledge graph construction unit and a semantic knowledge graph construction unit, and its specific implementation is as follows: the basic knowledge graph construction unit is used to perform entity alignment on text data, import the aligned text data into the basic graph, and generate a basic knowledge graph; wherein, the attributes of multiple entities in the text data are compared for similarity, a similarity threshold is set, and when the similarity reaches a certain similarity threshold, the text data is considered to belong to the same entity; the semantic knowledge graph construction unit is used to quantify all atoms in the basic knowledge graph, and process the quantified data with a translation algorithm, a path learning algorithm, a type constraint algorithm, and a heterogeneous information algorithm in natural language to generate a semantic knowledge graph; A data combination unit, used to combine the method of processing text data by the data processing unit to form a word segmentation and annotation model, an entity recognition model and a semantic recognition model; wherein the word segmentation and annotation model is used to annotate all parts of speech in the text data, including word segmentation, part of speech tagging and role tagging of the text data; the entity recognition model is used to identify all entities in the text data, including word segmentation, keyword extraction, entity recognition and reference resolution of the text data; the semantic recognition model is used to determine the semantics expressed by the text data, including summarization, event extraction, semantic extraction, topic extraction, keyword extraction and semantic disambiguation of the text data; The data application unit is used to iteratively update text data and update the knowledge graph in real time.
5. A semantic understanding system based on agricultural field text according to claim 4, characterized in that: The data processing unit includes a first processing unit and a second processing unit, which are specifically implemented as follows: The first processing unit is used to perform natural language processing on the text data; wherein the natural language processing includes word segmentation, keyword extraction, topic extraction, summary, event extraction and semantic extraction; The second processing unit is used to perform entity processing on the text data after natural language processing; wherein the entity processing includes part-of-speech tagging, entity recognition, semantic disambiguation, role labeling, reference resolution and entity alignment.
6. A semantic understanding system based on agricultural field text according to claim 4, characterized in that: The data combination unit also forms a click model, and the data application unit includes an AI interface, a basic function interface, a graph query interface, and an error correction and reasoning interface; The error correction and reasoning interface is used to modify errors in existing data, record the behavior data of the modification action, process the behavior data and convert it into data that can be trained by the click model, and realize iterative training of the click model; The AI interface includes word segmentation and annotation models, entity recognition models, and semantic recognition models, which are used to achieve downstream tasks such as supply and demand information matching, intelligent question answering, and patent document retrieval in the agricultural field; Basic functional interface, used to test and annotate text data generated by the data processing unit; The graph query interface is used to add, modify, and delete the knowledge graph in the graph construction unit to complete the real-time update of the knowledge graph.
