Knowledge graph-based plant disease identification method and system

By processing and analyzing the plant disease pictures input by the user, combining the knowledge graph to calculate the entity correlation, the problem of users' difficulty in describing plant diseases is solved, and the accuracy and speed of plant diseases are improved.

CN118942078AActive Publication Date: 2024-11-12INST OF PLANT PROTECTION GANSU ACAD OF AGRI SCI
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410920167.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-10
Publication Date
2024-11-12
Estimated Expiration
2044-07-10

AI Technical Summary

Technical Problem

In the existing knowledge graph-based plant disease identification methods, it is difficult for users to accurately describe plant disease, resulting in a low diagnostic accuracy.

Method used

By obtaining the plant disease pictures input by the user, performing picture processing to extract the entity and the number of entities, combining the preset knowledge graph, calculate the correlation degree of each alternative matching entity, and then identifying plant disease. Specific steps include image segmentation, texture and color feature analysis, convolutional neural network extraction of entities, and calculating correlation based on entity recognition factors and knowledge graphs.

Benefits of technology

It reduces the difficulty of users to obtain plant disease characteristic entities, improves the accuracy of plant disease identification, and improves the extraction speed and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942078A_ABST
    Figure CN118942078A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of image data processing, in particular to a plant disease identification method and system based on a knowledge graph. The plant disease identification method based on the knowledge graph comprises the following steps: acquiring a to-be-identified plant disease picture input by a user; performing picture processing on the picture of the plant disease to be identified, and obtaining each entity and the number of each entity in the picture; based on each entity in the picture, the number of each entity and a preset knowledge graph, obtaining each alternative matching entity and the association degree of each entity; and based on each alternative matching entity and each entity association degree, obtaining a plant disease identification result. The objective of the invention is to solve the problem that a user is difficult to describe plant diseases accurately, so that the diagnosis accuracy of the plant diseases is low.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a plant disease identification method and system based on a knowledge graph. Background Art

[0002] Knowledge graph is a complex way of organizing information. It stores and displays rich entities and their relationships in the form of a graphical database. These entities and relationships form a huge network that allows the system to perform complex queries and reasoning, thereby providing a deeper level of semantic understanding and data relationships. Knowledge graphs are widely used. They can not only enhance semantic search capabilities and provide more accurate search results, but also provide rich background knowledge to assist decision making. Using knowledge graphs in plant disease identification systems can retrieve information entered by users in the knowledge graph and provide users with diagnostic results and prevention strategies.

[0003] Existing plant disease recognition methods based on knowledge graphs usually extract entities from data to construct knowledge graphs to describe different plant diseases, then extract entities based on user input for reasoning and output diagnosis results. However, it is very difficult for users to accurately describe plant diseases, which leads to low diagnostic accuracy of plant diseases. Summary of the invention

[0004] In order to solve the technical problem that it is difficult for users to accurately describe plant diseases, resulting in low diagnostic accuracy of plant diseases, the purpose of the present invention is to provide a plant disease identification method and system based on knowledge graph. The technical solution adopted is as follows:

[0005] A plant disease identification method based on knowledge graph, the method comprising:

[0006] Obtain the picture of the plant disease to be identified input by the user;

[0007] Performing image processing on the image of the plant disease to be identified to obtain each entity and the number of each entity in the image;

[0008] Based on each entity and the quantity of each entity in the image and a preset knowledge graph, obtaining each candidate matching entity and the association degree of each entity;

[0009] Based on each candidate matching entity and each entity association degree, a recognition result of the plant disease is obtained.

[0010] Preferably, the specific steps of performing image processing on the image of the plant disease to be identified and obtaining each entity and the number of each entity in the image include:

[0011] The image of the plant disease to be identified is segmented using a preset initial segmentation scale to determine the color features and texture features in each segmented area;

[0012] Determine a texture adjustment factor and a color adjustment factor in each segmented area based on the color features and texture features in each segmented area;

[0013] After adjusting the preset initial segmentation scale using the texture adjustment factor and the color adjustment factor in each segmented area, the image of the plant disease to be identified is segmented again to obtain a segmented plant disease image;

[0014] The convolutional neural network is used to extract the segmented plant disease images and obtain each entity and the number of each entity in the image.

[0015] Preferably, the specific steps of determining the texture adjustment factor and the color adjustment factor in each segmented area based on the color features and the texture features in each segmented area include:

[0016] Based on the texture features in each segmented area, the texture complexity of each segmented area is obtained;

[0017] Determining a texture adjustment factor within each segmented area according to the texture complexity of each segmented area;

[0018] Based on the color features in each segmented area, the main color tone of each segmented area and the main color tone of the segmented areas adjacent to it are obtained;

[0019] The color adjustment factor in each segmented area is determined according to the main color tone of each segmented area and the main color tone of the adjacent segmented areas.

[0020] Preferably, the specific steps of extracting the segmented plant disease picture using a convolutional neural network and obtaining each entity and the number of each entity in the picture include:

[0021] The segmented plant disease images are input into the input layer of the convolutional neural network, and the output layer of the convolutional neural network is used to obtain each entity and the number of each entity in the image.

[0022] Preferably, based on each entity and the number of each entity in the image and a preset knowledge graph, the specific steps of obtaining each candidate matching entity and the association degree of each entity include:

[0023] Based on each entity and the number of each entity in the image, obtaining an entity recognition factor for each entity;

[0024] According to the entity recognition factor of each entity and the preset knowledge graph, each candidate matching entity and each entity association degree are obtained.

[0025] Preferably, according to the entity recognition factor of each entity and the preset knowledge graph, the specific steps of obtaining each candidate matching entity and the association degree of each entity include:

[0026] According to the entity recognition factor of each entity and the preset knowledge graph, the entity matching index of each entity in the knowledge graph, each candidate matching entity, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity are obtained;

[0027] Based on the entity matching index of each entity in the knowledge graph, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity, the relevance degree of each entity is obtained.

[0028] Preferably, the specific steps of obtaining the identification result of plant diseases based on each candidate matching entity and each entity association degree include:

[0029] Sorting the relevance of each entity in descending order to obtain candidate matching entities in the sorted knowledge graph;

[0030] Based on the candidate matching entities in the sorted knowledge graph, the recognition results of plant diseases are obtained.

[0031] Preferably, the steps of constructing the preset knowledge graph specifically include:

[0032] Obtain historical plant disease data including text data and image data;

[0033] Performing data processing on text data in historical plant disease data to obtain processed text data;

[0034] Performing data processing on the image data in the historical plant disease data to obtain processed image data;

[0035] Build a preset knowledge graph based on the processed text data, processed image data and knowledge graph model.

[0036] Preferably, the specific steps of performing data processing on the image data in the historical plant disease data to obtain the processed image data include:

[0037] Gaussian filtering and denoising are performed on the image data in the historical plant disease data to obtain processed image data.

[0038] The present invention also provides a plant disease identification system based on knowledge graph, comprising:

[0039] A first acquisition module is used to acquire a picture of a plant disease to be identified input by a user;

[0040] A second acquisition module is used to process the picture of the plant disease to be identified, and obtain each entity and the number of each entity in the picture;

[0041] A third acquisition module, based on each entity and the number of each entity in the image and a preset knowledge graph, acquires each candidate matching entity and the association degree of each entity;

[0042] The recognition module is used to obtain the recognition result of the plant disease based on each candidate matching entity and each entity association degree.

[0043] The present invention has the following beneficial effects:

[0044] 1. The present invention processes the pictures of plant diseases to be identified input by the user, obtains each entity and the number of each entity in the picture, and then obtains each candidate matching entity and the association degree of each entity based on the preset knowledge graph, and then obtains the recognition result of the plant disease according to the association degree of each candidate matching entity and each entity. This reduces the difficulty for users to obtain the characteristic entities of plant diseases, and can improve the recognition accuracy of plant diseases by identifying photos.

[0045] 2. The present invention analyzes and processes the plant disease pictures input by the user, takes into account the texture features and color features of the pictures, and thus optimizes the picture segmentation, which not only improves the extraction accuracy, but also improves the extraction speed, while also improving the recognition accuracy of plant diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flow chart of a method for identifying plant diseases based on a knowledge graph provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of a plant disease identification method and system based on a knowledge graph proposed by the present invention, its specific implementation method, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0049] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0050] The existing plant disease diagnosis based on knowledge graph first constructs the knowledge graph by extracting entities from the acquired data. The acquired data is usually text data, describing the characteristics of various diseases, etc. Then, entities are extracted from the user's input, and the entities are linked to the entities in the knowledge graph through certain reasoning methods, and finally the diagnosis results are output. However, the existing methods usually only allow users to input text descriptions of disease characteristics, or let users select corresponding feature entities. It is difficult for ordinary users to extract accurate characteristics of plant diseases, resulting in low accuracy of diagnosis results.

[0051] The specific scheme of a plant disease identification method and system based on knowledge graph provided by the present invention is described in detail below with reference to the accompanying drawings.

[0052] See also Figure 1 , which shows a flow chart of a plant disease identification method based on a knowledge graph provided by an embodiment of the present invention, the method comprising the following steps:

[0053] S100: Obtaining a picture of a plant disease to be identified input by a user.

[0054] S200: performing image processing on the image of the plant disease to be identified to obtain each entity and the number of each entity in the image.

[0055] S300: Based on each entity and the quantity of each entity in the image and a preset knowledge graph, obtain each candidate matching entity and the association degree of each entity.

[0056] S400: Obtain plant disease recognition results based on each candidate matching entity and each entity association degree.

[0057] The present invention processes the picture of the plant disease to be identified input by the user, obtains each entity and the number of each entity in the picture, and then obtains each candidate matching entity and the association degree of each entity based on the preset knowledge graph, and then obtains the recognition result of the plant disease according to the association degree of each candidate matching entity and each entity. This reduces the difficulty of users obtaining the characteristic entity of the plant disease, and can improve the recognition accuracy of the plant disease by identifying the photo.

[0058] Specifically, the steps for constructing the preset knowledge graph include:

[0059] Historical plant disease data including text data and image data are obtained. In this embodiment, in order to construct a preset knowledge graph. First, data is obtained. Taking corn disease identification as an example, the present invention obtains corn disease data from the corn pest and weed occurrence law and control technology database. The obtained data includes text data such as the name of the disease, pathogens, morphological characteristics, symptom identification, occurrence and distribution, control technology and related literature, as well as image data of the disease.

[0060] The text data in the historical plant disease data is processed to obtain processed text data, wherein the collected text data is cleaned.

[0061] The image data in the historical plant disease data is processed to obtain processed image data. Specifically, the image data in the historical plant disease data is subjected to Gaussian filtering and denoising to obtain processed image data.

[0062] Based on the processed text data, processed image data and knowledge graph model, a preset knowledge graph is constructed. In this embodiment, in order to construct a knowledge graph of plant diseases, the processed text data needs to be further processed to perform word segmentation and remove stop words; the purpose is to remove certain redundant information in the text data and make the relationship between different entities in the text data easier to express.

[0063] Word segmentation refers to dividing a text string into smaller units, usually words or phrases. Word segmentation can help computers understand human language structure and more easily extract the relationship between words. For example, [I love artificial intelligence], the word segmentation result may be [I, love, artificial, intelligence]. The word segmentation result is related to the context, word segmentation granularity and word segmentation method. The same sentence may also produce different word segmentation results. This embodiment uses the jieba Chinese word segmentation tool, and the stop word list of Harbin Institute of Technology is used to remove stop words.

[0064] Then the BiLSTM-CRF sequence model is used for named entity recognition to obtain entities in the plant disease text data. Through the features between entities in the plant disease data, the CasRel model is used to extract the entity relationship of plant diseases, and a graph structure with entities and related attributes of entities as nodes and associations between different entities as edges is obtained. Then the TransD model is used to embed the triple data, map the triples to the vector space, and finally store the obtained graph structure in the Neo4j graph database. At the same time, the processed image data is used as the attribute of the plant disease entity. The image data can be stored locally. Using the link of the image data as the attribute of the plant disease entity can speed up the matching speed and reduce the storage space required for the knowledge graph. Thus, the construction of the preset knowledge graph is realized. Among them, the BiLSTM-CRF sequence model, the CasRel model, the TransD model and the Neo4j graph database are commonly used technologies for building knowledge graphs, and the specific process will not be repeated.

[0065] On the basis of the above embodiments, in order to identify the pictures of plant diseases to be identified input by the user, it is first necessary to extract entities from the pictures, which can be extracted using convolutional neural networks. However, there are environmental interference and background interference in plant disease pictures. For example, the pictures of corn diseases input by the user usually capture the entire leaf, including diseased leaves and parts, as well as normal leaves and parts; and there are usually backgrounds such as the sky and land in the pictures, and some photographers will use their hands to support the plants; resulting in non-target elements such as the sky, land, and hands also appearing in the image. The presence of non-target elements increases the complexity of image processing.

[0066] The specific steps of S200 include:

[0067] The image of the plant disease to be identified is segmented using a preset initial segmentation scale to determine the color features and texture features in each segmented area. In this embodiment, the preset initial segmentation scale can be determined by the size of the plant disease image, such as taking one tenth of the length and one tenth of the width of the plant disease image as the size of the initial segmentation scale.

[0068] The most notable features of diseased leaves and parts of plants compared to normal leaves and parts or other interfering factors are the differences in color and texture. For example, diseased corn leaves will have spots. Although these spots vary in shape and size, they are mostly gray, brown or white, which is significantly different from the green color of normal corn leaves. Diseased corn fruits usually appear green and black, which is also significantly different from the yellow color of corn fruits. Color characteristics can be used for analysis.

[0069] In addition, the texture of the diseased parts of the plant will also change. Spots on the leaves or mildew on the fruit will make the plant surface rougher, because the disease affects the normal growth of plant cells, resulting in an uneven surface. Some diseases will also reduce the clarity of leaf veins. Therefore, the segmentation range can also be determined by analyzing the texture changes of plant disease images.

[0070] Based on the color features and texture features in each segmented area, a texture adjustment factor and a color adjustment factor in each segmented area are determined. This step specifically includes:

[0071] Based on the texture features in each segmented region, the texture complexity of each segmented region is obtained. Specifically, since the diseased part of the plant will become rough and not smooth, the texture complexity in the texture features in each segmented region is greater than that of other normal parts.

[0072] According to the texture complexity of each segmented area, a texture adjustment factor within each segmented area is determined.

[0073] The gray level co-occurrence matrix in the texture complexity of each segmented region is obtained, and the product of the energy and entropy of each segmented region is calculated by the gray level co-occurrence matrix and recorded as the texture adjustment factor of the segmented region.

[0074] The larger the value of the texture adjustment factor, the more complex the texture of the segmented area is and the greater the possibility of disease. At this time, the segmentation scale should be reduced to eliminate environmental interference and obtain more accurate extraction results. On the contrary, if the segmentation scale is increased, the segmented area mainly contains environmental interference.

[0075] The segmentation scale is adjusted by the texture adjustment factor. The specific method is:

[0076] Specifically, the calculation formula of the texture adjustment factor is:

[0077] δ k =Asm k ·Ent k ;

[0078] When the adjustment scale is increased, the length of the adjusted segmentation scale = (1 + texture adjustment factor) × the length of the initial segmentation scale, as shown in the following formula:

[0079] L k =(1+δ k )·L;

[0080] The width of the adjusted segmentation scale = (1 + texture adjustment factor) × the width of the initial segmentation scale, as shown in the following formula:

[0081] W k =(1-δ k)·W;

[0082] When reducing the adjustment scale, the length of the adjusted segmentation scale = (1-texture adjustment factor) × the length of the initial segmentation scale, see the following formula:

[0083] L k =(1-δ k )·L

[0084] The width of the adjusted segmentation scale = (1-texture adjustment factor) × the width of the initial segmentation scale, as shown in the following formula:

[0085] W k =(1-δ k )·W;

[0086] Among them, δ k is the texture adjustment factor of the kth segmented area, Asm k is the energy of the kth segmentation area, Ent k is the entropy of the kth segmented region, L is the initial length of the kth segmented region, W is the initial width of the kth region, and L k is the adjusted length of the kth segmentation region, W k is the adjusted width of the kth segmented region.

[0087] At the same time, after the segmentation scale is adjusted by the texture adjustment factor, if there is overlap between the segmented areas, the method for processing the overlapping areas is: dividing the overlapping areas into areas with a larger texture adjustment factor.

[0088] Based on the color features in each segmented area, the main color tone of each segmented area and the main color tone of the segmented areas adjacent to the segmented area are obtained.

[0089] The color adjustment factor in each segmented area is determined according to the main color tone of each segmented area and the main color tone of the adjacent segmented areas.

[0090] In this step, some segmented regions contain plant diseases and have many extractable entities, while other segmented regions mainly contain environmental interference and entities unrelated to plant diseases may be extracted.

[0091] Therefore, the color adjustment factor can be obtained according to the color characteristics of different regions. A larger weight can be given to the segmented region containing plant diseases, and the entities of this segmented region can be mainly extracted. A smaller weight can be given to the region containing environmental interference, and fewer or no entities in this region can be extracted.

[0092] The main color of plant disease images is green, but the main color of each area after segmentation is not necessarily green. Only the part containing plants has a green main color, while the main color of the background far from the plants is not green. It should be noted that only the main color of leaf disease images is green, while the main color of other plant disease images, such as corn fruit disease images, is yellow.

[0093] The color adjustment factor is obtained by the main color tone of each segmented area and the main color tone of the segmented area adjacent to it, wherein the specific calculation method is:

[0094] The color of the most pixels in each segmented area is recorded as the main color tone, and the proportion of the main color tone pixels in each area is calculated, which is recorded as the main color tone weight of each area. Then the average of the main color tone weights of the four neighborhoods of the segmented area is calculated, which is recorded as the main color tone association weight. The sum of half of the main color tone association weight and the main color tone weight is recorded as the color tone weight of the area.

[0095] The hue weight represents the proportion of main hue pixels in a region and the proportion of main hue pixels in the neighborhood of the region. Specifically, taking the leaf disease image as an example, the larger the main hue weight of a segmented region, the more main hue pixels it has, and the more likely it is to contain plant parts. The greater the possibility that the area in the neighborhood of a segmented region contains plants, the greater the possibility that the segmented region contains plants, because plants are continuous in the image. Therefore, the larger the value of the main hue association weight, the greater the possibility that the segmented region contains plants.

[0096] The larger the value of the hue weight is, the more likely it is that the segmented area contains part of the plant and requires a larger weight, otherwise a smaller weight is given.

[0097] In addition, the proportion of non-main color pixels in each segmented area is calculated, and the ratio of the non-main color pixel proportion to the main color pixel proportion is recorded as the disease color weight; the product of the disease color weight and the color weight is recorded as the color adjustment factor. The calculation formula of the main color weight is:

[0098]

[0099] Specifically, the calculation formula of the color adjustment factor is:

[0100]

[0101] Among them, Z k is the number of main color pixels in the kth segmentation area, Z k is a positive integer, X k is the total number of pixels in the kth segmentation area, X k is a positive integer, S kis the main color weight of the kth segmented area; S N4(k) represents the main color weight of the four neighborhoods of the kth segmented area; AVG() represents the mean function, AVG(S N4(k) ) represents the mean value of the main color weights of the four neighborhoods of the kth segmented area, is the number of non-main color pixels in the kth segmentation area, μ k is the color adjustment factor of the kth segmented area.

[0102] It should be noted that in an area containing plants, the larger the proportion of non-main color pixels is, the more serious the plant disease situation in the area is, and the more entities it contains.

[0103] Based on the above analysis, the larger the value of the color adjustment factor, the more likely the segmented area is to be a plant part and the more entities it contains. At this time, a larger weight should be given to the area to obtain a more accurate extraction result; on the contrary, reducing the weight will reduce the number of entities extracted from this segmented area.

[0104] The specific steps of setting the extraction weight for each segmented region by the color adjustment factor are as follows: the color adjustment factor of each region is used as the extraction weight of the region, wherein the weight of the region whose weight is less than the third quartile of all weights is set to zero.

[0105] After adjusting the preset initial segmentation scale using the texture adjustment factor and the color adjustment factor in each segmented area, the image of the plant disease to be identified is segmented again to obtain a segmented plant disease image.

[0106] The segmented plant disease images are extracted using a convolutional neural network to obtain each entity and the number of each entity in the image. Specifically, the segmented plant disease images are input into the input layer of the convolutional neural network, and the output layer of the convolutional neural network is used to obtain each entity and the number of each entity in the image.

[0107] Specifically, step S300 specifically includes: based on each entity in the picture and the number of each entity, obtaining the entity recognition factor of each entity. In this step, each entity obtained is converted into an entity vector.

[0108] Semantic information refers to the content related to meaning, concept or intention contained in data or signals. In order to match each entity, it is necessary to obtain the semantic information of the entity. The preset knowledge graph has converted each entity into an entity vector through embedding technology during the construction process. Each entity obtained can be converted into an entity vector through a pre-trained word embedding model. The Word2Vec model is used in this embodiment.

[0109] Each entity in the preset knowledge graph is converted into an entity vector, and each entity extracted from the disease image input by the user is also converted into a vector. In order to match the entity vector extracted from the plant disease image input by the user with the entity vector in the knowledge graph, it is necessary to calculate the correlation between them. However, there are many types of entities extracted from the image, and each entity may have many entities. It is necessary to obtain a representative entity vector through the relationship between these entities.

[0110] In this embodiment, the entity recognition factor of each entity in the plant disease picture is calculated by the frequency of occurrence of each entity vector in the plant disease picture. The specific calculation steps are:

[0111] The ratio of the number of each entity in the image to the number of all entities in the image is used as the weight of the entity. The weighted sum of all entity vectors is taken and the average vector is calculated, which is recorded as the entity recognition factor of the entity.

[0112] The entity recognition factor is calculated based on the frequency of the entity appearing in the plant disease pictures. The more times the entity appears and the greater the frequency, the more obvious the feature of the entity in the picture is, that is, the more important the entity is when matching with the entities in the knowledge graph. Therefore, a larger weight is given to the entity vector, making the entity vector more important in the entity recognition factor.

[0113] In addition, the distance between different entities can also be considered, because on diseased plants, entities with significant characteristics are usually close to each other and appear clustered. For example, there are many spots on corn leaves with leaf spot disease, and the distance between these spots is very close. By calculating the relationship between the average distance and the number of entities, the smaller the average distance and the more entities there are, the more important the entity is, and the more weight it should occupy in the entity recognition factor. The calculation formula of the entity recognition factor is:

[0114]

[0115] Among them, N ei is the number of the i-th entity in the image, N is the number of all entities in the image, e i is the i-th entity in the image, davg() means finding the average distance from one entity to other entities, davg(e i ) represents the average distance from the i-th entity to other entities in the image, E i is the entity recognition factor of the i-th entity in the image.

[0116] The entity recognition factor of each entity obtained represents each entity extracted from the plant disease picture input by the user. Because the entities contained in a plant disease picture are closely related to each other, it takes a long time to match them one by one with the entities in the knowledge graph. Therefore, each entity recognition factor is calculated to represent the many entities in plant diseases. The more entities of a certain type in the picture, the larger its proportion in the picture, and the more likely it is to match the plant disease entity related to it.

[0117] According to the entity recognition factor of each entity and the preset knowledge graph, each candidate matching entity and each entity association degree are obtained. This step is specifically as follows: according to the entity recognition factor of each entity and the preset knowledge graph, the entity matching index of each entity, each candidate matching entity, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity are obtained.

[0118] In this embodiment, the entity matching index is calculated by the entity recognition factor of each entity and the entity vector in the knowledge graph, wherein the calculation method is: calculating the ratio of the cosine similarity and the Euclidean distance between the two, and normalizing the result to record it as the entity matching index.

[0119] Cosine similarity indicates the degree of similarity between two vectors in direction, while Euclidean distance indicates the spatial distance between two vectors. The smaller the Euclidean distance, the closer the positions of the two vectors in space are, that is, the more similar they are. The entity matching index measures the degree of matching between the entity recognition factor and the entity vector in the knowledge graph from two aspects. The larger its value, the greater the cosine similarity between the two entity vectors, and the smaller the Euclidean distance, that is, the more similar the two entity vectors are; otherwise, the more dissimilar they are.

[0120] Entities whose entity matching index in the knowledge graph is greater than the average value are recorded as candidate matching entities. The calculation formula of the entity matching index is:

[0121]

[0122] Among them, i is the i-th entity in the knowledge graph, E i is the entity recognition factor of the i-th entity in the image, cos() means calculating the cosine similarity, d() means calculating the Euclidean distance and normalization, norm() means normalization, P i is the entity matching index of the i-th entity in the knowledge graph. It should be noted that when d(E i ,ze i ) is 0, the preset adjustment parameter σ is used as the denominator to prevent the denominator from being 0, wherein the preset adjustment parameter is greater than 0, and is set to 0.01 in this embodiment, and can be adjusted by itself.

[0123] Based on the entity matching index of each entity, the degree of each candidate matching entity, and the number of entities connected to each candidate matching entity, the relevance of each entity is obtained. In this step, the entity relevance is calculated by the entity matching index of each entity in the knowledge graph and the degree of the entity in the knowledge graph, wherein the calculation steps are:

[0124] Calculate the degree of each entity in the knowledge graph, obtain the degree of the candidate matching entity, multiply the degree of the candidate matching entity by the degree of all entities connected to it, and sum them up. After normalization, add the entity matching index and record it as the entity relevance. The calculation formula for entity relevance is:

[0125]

[0126] In the above formula, GD i is the entity relevance of the i-th candidate matching entity in the knowledge graph, norm() indicates normalization, D i is the degree of the i-th candidate matching entity, D j is the degree of the jth entity connected to the ith candidate matching entity, and N is the number of entities connected to the ith candidate matching entity.

[0127] The degree in this embodiment refers to the number of other entities connected to an entity. A plant disease entity can be connected to multiple disease characteristic entities, and different plant disease entities may also be connected to the same disease characteristic entity. Therefore, it is necessary to consider other entities connected to the entity.

[0128] If a plant disease is more common, people will know more about it, and there will be more feature descriptions of it, which is reflected in the knowledge graph as more entities connected to it. Therefore, the more other entities connected to the disease entity, that is, the greater the degree of the plant disease entity, the more common the disease is. Therefore, this factor is taken into account in plant disease identification, and more common diseases are identified first.

[0129] The entity association not only considers the correlation between the entity recognition factor and the alternative matching entity, but also the degree of the alternative matching entity. The larger the entity association, the more similar the alternative matching entity is to the entity recognition factor, and the more entities are connected to the alternative matching factor, the easier it is to find the plant disease entity you want to identify, and the plant disease entities with high entity association are output first.

[0130] Specifically, step S400 specifically includes: sorting the relevance of each entity in order from large to small, and obtaining candidate matching entities in the sorted knowledge graph.

[0131] Based on the candidate matching entities in the sorted knowledge graph, the recognition results of plant diseases are obtained.

[0132] In this embodiment, the top three candidate matching entities can be selected and recorded as the first candidate matching entity, the second candidate matching entity and the third candidate matching entity, respectively. The disease entities corresponding to the top three candidate matching entities in the knowledge graph are obtained, and the plant disease data corresponding to these disease entities are output.

[0133] Among them, the final disease entity is not necessarily one. For example, the disease entity corresponding to the first candidate matching entity in the knowledge graph is the first disease entity, and the disease entity corresponding to the second and third candidate matching entities is another disease entity, recorded as the second disease entity.

[0134] Then, the first disease entity is the best recognition result, and the second disease entity is the candidate recognition result. Similarly, if the first candidate matching entity has two disease entities corresponding to it at the same time, then both disease entities are regarded as the best recognition results.

[0135] The plant disease data corresponding to the disease entity of the final output recognition result includes: the name of the disease, pathogen, morphological characteristics, symptom recognition, occurrence and distribution, prevention and control techniques and related literature, as well as pictures of the disease.

[0136] The present invention analyzes and processes the plant disease image data input by the user, takes into account the texture features and color features of the image, segments the image, and then assigns different extraction weights to each segmented area, which not only improves the extraction accuracy but also improves the extraction speed, thereby improving the recognition accuracy of plant diseases.

[0137] A plant disease identification system based on a knowledge graph provided by the present invention is described below. The plant disease identification system based on a knowledge graph described below and the plant disease identification method based on a knowledge graph described above can refer to each other.

[0138] The present invention provides a plant disease identification system based on knowledge graph, which includes a first acquisition module, a second acquisition module, a third acquisition module and an identification module.

[0139] The first acquisition module is used to acquire the picture of the plant disease to be identified input by the user.

[0140] The second acquisition module is used to process the image of the plant disease to be identified, and obtain each entity and the number of each entity in the image.

[0141] The third acquisition module acquires each candidate matching entity and the association degree of each entity based on each entity and the quantity of each entity in the image and a preset knowledge graph.

[0142] The recognition module is used to obtain the recognition result of the plant disease based on each candidate matching entity and each entity association degree.

[0143] The second acquisition module is specifically used for:

[0144] The image of plant diseases to be identified is segmented using a preset initial segmentation scale to determine the color and texture features in each segmented area.

[0145] Based on the color features and texture features in each segmented area, the texture adjustment factor and color adjustment factor in each segmented area are determined. Specifically:

[0146] Based on the texture features in each segmented region, the texture complexity of each segmented region is obtained.

[0147] According to the texture complexity of each segmented area, a texture adjustment factor in each segmented area is determined.

[0148] Based on the color features in each segmented area, the main color tone of each segmented area and the main color tone of the segmented areas adjacent to the segmented area are obtained.

[0149] The color adjustment factor in each segmented area is determined according to the main color tone of each segmented area and the main color tone of the adjacent segmented areas.

[0150] After adjusting the preset initial segmentation scale using the texture adjustment factor and the color adjustment factor in each segmented area, the image of the plant disease to be identified is segmented again to obtain a segmented plant disease image;

[0151] The segmented plant disease images are extracted using a convolutional neural network to obtain each entity and the number of each entity in the image. Specifically, the segmented plant disease images are input into the input layer of the convolutional neural network, and the output layer of the convolutional neural network is used to obtain each entity and the number of each entity in the image.

[0152] The third acquisition module is specifically used to: acquire the entity recognition factor of each entity based on each entity and the quantity of each entity in the picture.

[0153] According to the entity recognition factor of each entity and the preset knowledge graph, each candidate matching entity and each entity association degree are obtained. This step is specifically as follows: according to the entity recognition factor of each entity and the preset knowledge graph, the entity matching index of each entity in the knowledge graph, each candidate matching entity, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity are obtained.

[0154] Based on the entity matching index of each entity in the knowledge graph, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity, the relevance degree of each entity is obtained.

[0155] The recognition module is specifically used to sort the association degree of each entity in descending order to obtain candidate matching entities in the sorted knowledge graph.

[0156] Based on the candidate matching entities in the sorted knowledge graph, the recognition results of plant diseases are obtained.

[0157] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0158] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A plant disease identification method based on knowledge graph, It is characterized in that The method comprises: Obtain the picture of the plant disease to be identified input by the user; Performing image processing on the image of the plant disease to be identified to obtain each entity and the number of each entity in the image; Based on each entity and the quantity of each entity in the image and a preset knowledge graph, obtaining each candidate matching entity and the association degree of each entity; Based on each candidate matching entity and each entity association degree, a recognition result of the plant disease is obtained.

2. The method for identifying plant diseases based on knowledge graph according to claim 1, It is characterized in that The specific steps of performing image processing on the image of the plant disease to be identified and obtaining each entity and the number of each entity in the image include: The image of the plant disease to be identified is segmented using a preset initial segmentation scale to determine the color features and texture features in each segmented area; Determine a texture adjustment factor and a color adjustment factor in each segmented area based on the color features and texture features in each segmented area; After adjusting the preset initial segmentation scale using the texture adjustment factor and the color adjustment factor in each segmented area, the image of the plant disease to be identified is segmented again to obtain a segmented plant disease image; The convolutional neural network is used to extract the segmented plant disease images and obtain each entity and the number of each entity in the image.

3. The method for identifying plant diseases based on knowledge graph according to claim 2, It is characterized in that The specific steps of determining the texture adjustment factor and the color adjustment factor in each segmented area based on the color features and the texture features in each segmented area include: Based on the texture features in each segmented area, the texture complexity of each segmented area is obtained; Determining a texture adjustment factor within each segmented area according to the texture complexity of each segmented area; Based on the color features in each segmented area, the main color tone of each segmented area and the main color tone of the segmented areas adjacent to it are obtained; The color adjustment factor in each segmented area is determined according to the main color tone of each segmented area and the main color tone of the adjacent segmented areas.

4. The method for identifying plant diseases based on knowledge graph according to claim 2, It is characterized in that The specific steps of extracting the segmented plant disease picture by using a convolutional neural network and obtaining each entity and the number of each entity in the picture include: The segmented plant disease images are input into the input layer of the convolutional neural network, and the output layer of the convolutional neural network is used to obtain each entity and the number of each entity in the image.

5. The method for identifying plant diseases based on knowledge graph according to claim 1, It is characterized in that Based on each entity and the number of each entity in the image and the preset knowledge graph, the specific steps of obtaining each candidate matching entity and the relevance of each entity include: Based on each entity and the number of each entity in the image, obtaining an entity recognition factor for each entity; According to the entity recognition factor of each entity and the preset knowledge graph, each candidate matching entity and each entity association degree are obtained.

6. The method for identifying plant diseases based on knowledge graph according to claim 5, It is characterized in that According to the entity recognition factor of each entity and the preset knowledge graph, the specific steps of obtaining each candidate matching entity and the association degree of each entity include: According to the entity recognition factor of each entity and the preset knowledge graph, the entity matching index of each entity in the knowledge graph, each candidate matching entity, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity are obtained; Based on the entity matching index of each entity in the knowledge graph, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity, the relevance degree of each entity is obtained.

7. The method for identifying plant diseases based on knowledge graph according to claim 1, It is characterized in that The specific steps of obtaining the identification result of plant diseases based on each candidate matching entity and each entity association degree include: Sorting the relevance of each entity in descending order to obtain candidate matching entities in the sorted knowledge graph; Based on the candidate matching entities in the sorted knowledge graph, the recognition results of plant diseases are obtained.

8. The method for identifying plant diseases based on knowledge graph according to any one of claims 1 to 7, It is characterized in that The steps of constructing the preset knowledge graph specifically include: Obtain historical plant disease data including text data and image data; Performing data processing on text data in historical plant disease data to obtain processed text data; Performing data processing on the image data in the historical plant disease data to obtain processed image data; Build a preset knowledge graph based on the processed text data, processed image data and knowledge graph model.

9. The method for identifying plant diseases based on knowledge graph according to claim 8, It is characterized in that The specific steps of processing the image data in the historical plant disease data to obtain the processed image data include: Gaussian filtering and denoising are performed on the image data in the historical plant disease data to obtain processed image data.

10. A plant disease identification system based on knowledge graph, It is characterized in that include: The first acquisition module is used to acquire the picture of the plant disease to be identified input by the user; A second acquisition module is used to process the picture of the plant disease to be identified, and obtain each entity and the number of each entity in the picture; A third acquisition module, based on each entity and the number of each entity in the image and a preset knowledge graph, acquires each candidate matching entity and the association degree of each entity; The recognition module is used to obtain the recognition result of the plant disease based on each candidate matching entity and each entity association degree.

Citation Information

Patent Citations

  • Method for establishing medical image atlas based on image segmentation and convolutional neural network (CNN)

    CN108389614A

  • Data processing method and device and computer readable storage medium

    CN110795995A

  • Knowledge base document and knowledge graph entity association method and system

    CN113360665A

  • Knowledge graph and deep learning fused fruit tree disease and insect pest diagnosis method

    CN113516164A

  • Crop diagnosis method and device, coma diagnosis instrument and readable storage medium

    CN115375923A