Plant disease identification method and system based on knowledge graph

Abstract

The present invention relates to the field of image data processing technology, and in particular to a method and system for identifying plant diseases based on a knowledge graph. Among them, a method for identifying plant diseases based on a knowledge graph includes: obtaining a picture of a plant disease to be identified input by a user; performing image processing on the picture of the plant disease to be identified, obtaining each entity in the picture and the number of each entity; based on each entity in the picture and the number of each entity and a preset knowledge graph, obtaining each candidate matching entity and the degree of association of each entity; based on each candidate matching entity and the degree of association of each entity, obtaining the identification result of the plant disease. The purpose of the present invention is to solve the problem that it is difficult for users to accurately describe plant diseases, resulting in a low diagnostic accuracy rate for plant diseases.

Description

Technical Field

[0001] The present invention relates to the field of image data processing technology, and in particular to a plant disease identification method and system based on a knowledge graph. Background Art

[0002] A knowledge graph is a complex information organization method that stores and displays a rich set of entities and their relationships in the form of a graphical database. These entities and relationships form a vast network, enabling the system to perform complex queries and reasoning, thereby providing a deeper level of semantic understanding and data relationships. Knowledge graphs are widely used, not only enhancing semantic search capabilities and providing more accurate search results, but also providing rich background knowledge to aid decision-making. Using knowledge graphs in plant disease identification systems allows user-entered information to be retrieved from the knowledge graph, providing users with diagnostic results and prevention strategies.

[0003] Existing plant disease identification methods based on knowledge graphs typically first extract entities from data to construct a knowledge graph describing different plant diseases. These entities are then extracted based on user input for inference, resulting in a diagnostic result. However, accurately describing plant diseases is difficult for users, resulting in low diagnostic accuracy. Summary of the Invention

[0004] In order to solve the technical problem that users have difficulty accurately describing plant diseases, resulting in low diagnostic accuracy of plant diseases, the purpose of the present invention is to provide a plant disease identification method and system based on knowledge graph. The technical solutions adopted are as follows:

[0005] A plant disease identification method based on a knowledge graph, the method comprising:

[0006] Obtain the picture of the plant disease to be identified input by the user;

[0007] Performing image processing on the image of the plant disease to be identified to obtain each entity and the number of each entity in the image;

[0008] Based on each entity and the number of each entity in the image and a preset knowledge graph, obtaining each candidate matching entity and the association degree of each entity;

[0009] Based on each candidate matching entity and each entity association degree, a plant disease recognition result is obtained.

[0010] Preferably, the specific steps of performing image processing on the image of the plant disease to be identified and obtaining each entity and the number of each entity in the image include:

[0011] The image of the plant disease to be identified is segmented using a preset initial segmentation scale, and the color and texture features within each segmented area are determined;

[0012] Determining a texture adjustment factor and a color adjustment factor within each segmented area based on the color features and texture features within each segmented area;

[0013] After adjusting the preset initial segmentation scale using the texture adjustment factor and the color adjustment factor in each segmented area, the image of the plant disease to be identified is segmented again to obtain a segmented plant disease image;

[0014] Use convolutional neural networks to extract the segmented plant disease images and obtain each entity and the number of each entity in the image.

[0015] Preferably, the specific steps of determining the texture adjustment factor and the color adjustment factor in each segmented area based on the color features and the texture features in each segmented area include:

[0016] Based on the texture features in each segmented area, the texture complexity of each segmented area is obtained;

[0017] Determining a texture adjustment factor within each segmented area according to the texture complexity of each segmented area;

[0018] Based on the color features in each segmented area, the main color tone of each segmented area and the main color tone of the adjacent segmented areas are obtained;

[0019] A color adjustment factor in each segmented area is determined according to the main color hue of each segmented area and the main color hue of adjacent segmented areas.

[0020] Preferably, the specific steps of extracting the segmented plant disease image using a convolutional neural network and obtaining each entity and the number of each entity in the image include:

[0021] The segmented plant disease images are input into the input layer of the convolutional neural network, and the output layer of the convolutional neural network is used to obtain each entity and the number of each entity in the image.

[0022] Preferably, based on each entity and the number of each entity in the image and a preset knowledge graph, the specific steps of obtaining each candidate matching entity and the relevance of each entity include:

[0023] Based on each entity and the number of each entity in the image, obtaining an entity recognition factor for each entity;

[0024] According to the entity recognition factor of each entity and the preset knowledge graph, each candidate matching entity and each entity association degree are obtained.

[0025] Preferably, the specific steps of obtaining the relevance between each candidate matching entity and each entity according to the entity identification factor of each entity and the preset knowledge graph include:

[0026] According to the entity recognition factor of each entity and the preset knowledge graph, obtain the entity matching index of each entity in the knowledge graph, each candidate matching entity, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity;

[0027] Based on the entity matching index of each entity in the knowledge graph, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity, the relevance degree of each entity is obtained.

[0028] Preferably, the specific steps of obtaining the identification result of the plant disease based on each candidate matching entity and each entity association degree include:

[0029] Sort the relevance of each entity in descending order to obtain candidate matching entities in the sorted knowledge graph;

[0030] Based on the candidate matching entities in the sorted knowledge graph, the identification results of plant diseases are obtained.

[0031] Preferably, the steps of constructing the preset knowledge graph specifically include:

[0032] Obtain historical plant disease data including text data and image data;

[0033] Performing data processing on text data in historical plant disease data to obtain processed text data;

[0034] performing data processing on the image data in the historical plant disease data to obtain processed image data;

[0035] Build a preset knowledge graph based on the processed text data, processed image data and knowledge graph model.

[0036] Preferably, the specific steps of processing the image data in the historical plant disease data to obtain the processed image data include:

[0037] Gaussian filtering and denoising are performed on the image data in the historical plant disease data to obtain processed image data.

[0038] The present invention also provides a plant disease identification system based on a knowledge graph, comprising:

[0039] The first acquisition module is used to acquire a picture of a plant disease to be identified input by a user;

[0040] A second acquisition module is used to process the image of the plant disease to be identified to obtain each entity and the number of each entity in the image;

[0041] A third acquisition module acquires each candidate matching entity and the association degree of each entity based on each entity and the number of each entity in the image and a preset knowledge graph;

[0042] The recognition module is used to obtain the recognition result of the plant disease based on each candidate matching entity and the association degree of each entity.

[0043] The present invention has the following beneficial effects:

[0044] 1. This invention processes a user-input image of a plant disease to be identified, obtains each entity and its quantity in the image, then, based on a pre-set knowledge graph, obtains the association between each candidate matching entity and each entity. Furthermore, based on the association between each candidate matching entity and each entity, the plant disease identification result is obtained. This reduces the difficulty for users to obtain characteristic plant disease entities and improves the accuracy of plant disease identification through photo identification.

[0045] 2. The present invention analyzes and processes the plant disease pictures input by the user, taking into account the texture and color features of the pictures, and thus optimizes the picture segmentation, which not only improves the extraction accuracy, but also improves the extraction speed, while also improving the recognition accuracy of plant diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flow chart of a plant disease identification method based on a knowledge graph provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0048] To further illustrate the technical means and effectiveness of the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a knowledge graph-based plant disease identification method and system proposed by the present invention. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0049] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0050] Existing plant disease diagnosis methods based on knowledge graphs first construct a knowledge graph by extracting entities from acquired data. The acquired data is typically text data describing the characteristics of various diseases. Entities are then extracted from user input, linked to entities in the knowledge graph using certain reasoning methods, and finally a diagnosis result is output. However, existing methods typically only allow users to enter a text description of disease characteristics or select corresponding feature entities. This makes it difficult for ordinary users to extract accurate plant disease characteristics, resulting in low diagnostic accuracy.

[0051] The following describes in detail a plant disease identification method and system based on knowledge graph provided by the present invention in conjunction with the accompanying drawings.

[0052] See also Figure 1 , which shows a flow chart of a plant disease identification method based on a knowledge graph provided by an embodiment of the present invention, the method includes the following steps:

[0053] S100: Obtain a picture of a plant disease to be identified input by a user.

[0054] S200: Process the image of the plant disease to be identified to obtain each entity and the number of each entity in the image.

[0055] S300: Based on each entity and the number of each entity in the image and a preset knowledge graph, obtain each candidate matching entity and the association degree of each entity.

[0056] S400: Obtain plant disease recognition results based on each candidate matching entity and each entity association degree.

[0057] The present invention processes a user-input image of a plant disease to be identified, obtains each entity and its quantity in the image, then, based on a preset knowledge graph, obtains the association degree between each candidate matching entity and each entity. Furthermore, based on the association degree between each candidate matching entity and each entity, the plant disease identification result is obtained. This reduces the difficulty for users to obtain characteristic plant disease entities and improves the accuracy of plant disease identification through photo recognition.

[0058] Specifically, the steps for constructing the preset knowledge graph include:

[0059] Historical plant disease data, including text and image data, is obtained. In this embodiment, to construct a pre-set knowledge graph, data is first obtained. Taking corn disease identification as an example, the present invention obtains corn disease data from a database of occurrence patterns and control techniques for corn diseases, insect pests, and weeds. The obtained data includes text data such as the disease name, pathogen, morphological characteristics, symptom identification, occurrence and distribution, control techniques, and relevant literature, as well as image data of the disease.

[0060] Data processing is performed on the text data in the historical plant disease data to obtain processed text data. In particular, data cleaning is performed on the collected text data.

[0061] The image data in the historical plant disease data is processed to obtain processed image data. Specifically, the image data in the historical plant disease data is subjected to Gaussian filtering and denoising to obtain processed image data.

[0062] Based on the processed text data, processed image data, and knowledge graph model, a preset knowledge graph is constructed. In this embodiment, to construct a knowledge graph for plant diseases, the processed text data must first be further processed to perform word segmentation and remove stop words. This is to remove certain redundant information from the text data and make it easier to express the relationships between different entities in the text data.

[0063] Word segmentation refers to breaking a text string into smaller units, usually words or phrases. Word segmentation helps computers understand human language structure and more easily extract relationships between words. For example, the word segmentation result for "I love artificial intelligence" might be "I, love, artificial, intelligence." The word segmentation results depend on the context, word segmentation granularity, and word segmentation method. The same sentence can produce different word segmentation results. This example uses the Jieba Chinese word segmentation tool, and the Harbin Institute of Technology stop word list is used to remove stop words.

[0064] The BiLSTM-CRF sequence model is then used for named entity recognition to identify entities in the plant disease text data. Based on the features between entities in the plant disease data, the CasRel model is used to extract entity relationships within the plant disease data. This results in a graph structure with entities and their related attributes as nodes and relationships between different entities as edges. The TransD model is then used to embed the triples, mapping the triples into a vector space. Finally, the resulting graph structure is stored in a Neo4j graph database. The processed image data is used as attributes of the plant disease entities. Image data can be stored locally. Using links in the image data as attributes of the plant disease entities speeds up matching and reduces the storage space required for the knowledge graph. This allows the construction of the pre-defined knowledge graph. The BiLSTM-CRF sequence model, CasRel model, TransD model, and Neo4j graph database are commonly used technologies for building knowledge graphs, and the detailed process is not detailed here.

[0065] Based on the above examples, in order to identify the plant disease images input by the user, it is first necessary to extract entities from the images. This can be done using a convolutional neural network. However, plant disease images are subject to environmental and background interference. For example, user-input images of corn diseases typically capture the entire plant leaf, including both diseased leaves and parts as well as normal leaves and parts. Furthermore, the images often include background elements such as the sky and land, and some photographers may use their hands to support the plant, causing non-target elements such as the sky, land, and hands to appear in the image. The presence of non-target elements increases the complexity of image processing.

[0066] The specific steps of S200 include:

[0067] The image of the plant disease to be identified is segmented using a preset initial segmentation scale to determine the color and texture features within each segmented area. In this embodiment, the preset initial segmentation scale can be determined by the size of the plant disease image, such as using one-tenth of the length and one-tenth of the width of the plant disease image as the initial segmentation scale.

[0068] The most striking characteristics of diseased plant leaves and parts compared to healthy leaves and parts, or other interfering factors, are differences in color and texture. For example, diseased corn leaves will develop lesions. While these lesions vary in shape and size, they are generally gray, brown, or white, a stark contrast to the green color of normal corn leaves. Diseased corn berries often exhibit moldy green or black hues, which are also significantly different from the yellow berries. This color can be used for analysis.

[0069] In addition, the texture of diseased parts of plants will also change. Spots on leaves or mold on fruits will make the plant surface rougher. This is because the disease affects the normal growth of plant cells, resulting in an uneven surface. Some diseases can also reduce the clarity of leaf veins. Therefore, the segmentation range can also be determined by analyzing the texture changes of plant disease images.

[0070] Based on the color features and texture features in each segmented area, a texture adjustment factor and a color adjustment factor in each segmented area are determined. This step specifically includes:

[0071] Based on the texture features in each segmented region, the texture complexity of each segmented region is obtained. Specifically, since the diseased parts of the plant will become rough and not smooth, the texture complexity of the texture features in each segmented region is greater than that of other normal parts.

[0072] According to the texture complexity of each segmented area, a texture adjustment factor within each segmented area is determined.

[0073] The gray level co-occurrence matrix in the texture complexity of each segmented area is obtained, and the product of the energy and entropy of each segmented area is calculated by the gray level co-occurrence matrix and recorded as the texture adjustment factor of the segmented area.

[0074] The larger the value of the texture adjustment factor, the more complex the texture of the segmented area is and the greater the possibility of disease. At this time, the segmentation scale should be reduced to eliminate environmental interference and obtain more accurate extraction results. On the contrary, if the segmentation scale is increased, the segmented area will mainly contain environmental interference.

[0075] The segmentation scale is adjusted by the texture adjustment factor. The specific method is:

[0076] Specifically, the calculation formula of the texture adjustment factor is:

[0077] δ k =Asm k Ent k ;

[0078] When the adjustment scale is increased, the length of the adjusted segmentation scale = (1 + texture adjustment factor) × the length of the initial segmentation scale, as shown in the following formula:

[0079] L k =(1+δ k )·L;

[0080] The width of the adjusted segmentation scale = (1 + texture adjustment factor) × the width of the initial segmentation scale, as shown in the following formula:

[0081] W k =(1-δ k)·W;

[0082] When reducing the adjustment scale, the length of the adjusted segmentation scale = (1-texture adjustment factor) × the length of the initial segmentation scale, see the following formula:

[0083] L k =(1-δ k )·L

[0084] The width of the adjusted segmentation scale = (1-texture adjustment factor) × the width of the initial segmentation scale, as shown in the following formula:

[0085] W k =(1-δ k )·W;

[0086] Among them, δ k is the texture adjustment factor of the kth segmentation area, Asm k is the energy of the kth segmentation area, Ent k is the entropy of the kth segmentation region, L is the initial length of the kth segmentation region, W is the initial width of the kth region, L k is the adjusted length of the kth segmentation area, W k is the adjusted width of the kth segmented region.

[0087] At the same time, after the segmentation scale is adjusted by the texture adjustment factor, if there is overlap between the segmented areas, the method for processing the overlapping areas is: dividing the overlapping areas into areas with larger texture adjustment factors.

[0088] Based on the color features in each segmented area, the main color tone of each segmented area and the main color tones of the segmented areas adjacent to the segmented area are obtained.

[0089] A color adjustment factor in each segmented area is determined according to the main color hue of each segmented area and the main color hue of adjacent segmented areas.

[0090] In this step, some segmented regions contain plant diseases and have many extractable entities, while other segmented regions mainly contain environmental interference and may extract entities unrelated to plant diseases.

[0091] Therefore, the color adjustment factor can be obtained according to the color characteristics of different regions. A larger weight can be given to the segmented region containing plant diseases, and the entities of this segmented region can be mainly extracted. A smaller weight can be given to the region containing environmental interference, and fewer or no entities in this region can be extracted.

[0092] The dominant color in plant disease images is green, but after segmentation, the dominant color of each region is not necessarily green. Only the area containing the plant is green, while the background away from the plant is not green. It should be noted that only images of leaf diseases have a dominant color of green, while other plant disease images, such as images of corn pod diseases, have a dominant color of yellow.

[0093] The color adjustment factor is obtained by the main color tone of each segmented area and the main color tone of the adjacent segmented areas, wherein the specific calculation method is:

[0094] The color of the most numerous pixels in each segmented region is designated as the dominant hue. The percentage of pixels with the dominant hue in each region is calculated, which is the dominant hue weight for each region. The mean of the dominant hue weights of the four neighboring regions of the segmented region is then calculated, which is the dominant hue association weight. The sum of half the dominant hue association weight and the dominant hue weight is the hue weight for the region.

[0095] The hue weight represents the proportion of dominant hue pixels in a region and the proportion of dominant hue pixels in the region's neighborhood. Specifically, taking an image of leaf disease as an example, the larger the dominant hue weight of a segmented region, the more dominant hue pixels it contains, and the more likely it is to contain plant parts. Furthermore, the greater the likelihood that the areas within a segmented region's neighborhood contain plants, the greater the probability that the segmented region contains plants, because plants are continuous in the image. Therefore, the larger the dominant hue association weight, the greater the probability that the segmented region contains plants.

[0096] The larger the value of the hue weight is, the more likely the segmented area is to contain a plant part, and a larger weight is required. Otherwise, a smaller weight is given.

[0097] In addition, the proportion of non-main color pixels in each segmented area is calculated, and the ratio of the non-main color pixel proportion to the main color pixel proportion is recorded as the damage color weight; the product of the damage color weight and the color weight is recorded as the color adjustment factor. The calculation formula of the main color weight is:

[0098]

[0099] Specifically, the calculation formula of the color adjustment factor is:

[0100]

[0101] Among them, Z k is the number of main color pixels in the kth segmentation area, Z k is a positive integer, X k is the total number of pixels in the kth segmentation area, X k is a positive integer, S kis the main color weight of the kth segmentation area; S N4(k) represents the main color weight of the four neighborhoods of the kth segmentation area; AVG() represents the mean function, AVG(S N4(k) ) represents the mean of the main color weights of the four neighborhoods of the kth segmentation area, is the number of non-main color pixels in the kth segmentation area, μ k is the color adjustment factor of the kth segmented region.

[0102] It should be noted that in an area containing plants, the larger the proportion of non-main color pixels, the more serious the plant disease situation in the area is and the more entities it contains.

[0103] Based on the above analysis, the larger the value of the color adjustment factor, the more likely the segmented area is to be a plant part and the more entities it contains. In this case, a larger weight should be given to the area to obtain a more accurate extraction result; on the contrary, reducing the weight will reduce the number of entities extracted from this segmented area.

[0104] The specific steps of setting the extraction weight for each segmented region by the color adjustment factor are as follows: the color adjustment factor of each region is used as the extraction weight of the region, wherein the weight of the region whose weight is less than the third quartile of all weights is set to zero.

[0105] After adjusting the preset initial segmentation scale using the texture adjustment factor and the color adjustment factor in each segmented area, the image of the plant disease to be identified is segmented again to obtain a segmented plant disease image.

[0106] A convolutional neural network is used to extract the segmented plant disease images and obtain each entity and the number of each entity in the image. Specifically, the segmented plant disease image is input into the input layer of the convolutional neural network, and the output layer of the convolutional neural network is used to obtain each entity and the number of each entity in the image.

[0107] Specifically, step S300 includes: obtaining an entity recognition factor of each entity based on each entity and the number of each entity in the image. In this step, each entity obtained is converted into an entity vector.

[0108] Semantic information refers to the content related to meaning, concept, or intent contained in data or signals. To match each entity, semantic information about the entity is required. During the construction of the pre-set knowledge graph, each entity is converted into an entity vector using embedding technology. Each entity obtained can be converted into an entity vector using a pre-trained word embedding model. This example uses the Word2Vec model.

[0109] Each entity in the pre-set knowledge graph is converted into an entity vector. Simultaneously, each entity extracted from the user-entered disease image is also converted into a vector. To match the entity vectors extracted from the user-entered plant disease image with the entity vectors in the knowledge graph, the correlation between them must be calculated. However, there are many different entities extracted from the image, and each entity may have multiple instances. Therefore, a representative entity vector must be derived from the relationships between these entities.

[0110] In this embodiment, the entity recognition factor of each entity in the plant disease picture is calculated by the frequency of occurrence of each entity vector in the plant disease picture. The specific calculation steps are:

[0111] The ratio of the sum of the number of each entity in the image to the total number of entities in the image is used as the weight of the entity. The weighted sum of all entity vectors is taken and the average vector is calculated, which is recorded as the entity recognition factor of the entity.

[0112] The entity recognition factor is calculated based on the frequency of the entity appearing in the plant disease image. The more times the entity appears and the greater the frequency, the more obvious the feature of the entity in the image is, that is, the more important the entity is when matching with the entities in the knowledge graph. Therefore, a larger weight is given to the entity vector, making the entity vector more important in the entity recognition factor.

[0113] In addition, the distance between different entities can also be considered. This is because on diseased plants, entities with significant characteristics are often close together and clustered. For example, corn leaves with leaf spot disease have many spots that are very close together. By calculating the relationship between the average distance between different entities and their number, the smaller the average distance and the greater the number of entities, the more important the entity is and the greater the weight it should have in the entity recognition factor. The calculation formula for the entity recognition factor is:

[0114]

[0115] Among them, N ei is the number of entities in the image, N is the number of all entities in the image, e i is the i-th entity in the picture, davg() means finding the average distance from one entity to other entities, davg(e i ) represents the average distance from the i-th entity to other entities in the image, E i is the entity recognition factor of the i-th entity in the image.

[0116] The entity recognition factor obtained for each entity represents each entity extracted from the plant disease picture input by the user. Because the entities contained in a plant disease picture are closely related to each other, it takes a long time to match them one by one with the entities in the knowledge graph. Therefore, each entity recognition factor is calculated to represent the many entities in the plant disease. The more entities in the picture, the larger its proportion in the picture, and the more likely it is to match the plant disease entity related to it.

[0117] Based on each entity's entity recognition factor and the preset knowledge graph, obtain each candidate matching entity and each entity's association degree. This step specifically involves: based on each entity's entity recognition factor and the preset knowledge graph, obtain each entity's entity matching index, each candidate matching entity, each candidate matching entity's degree, and the number of entities connected to each candidate matching entity.

[0118] In this embodiment, the entity matching index is calculated by the entity recognition factor of each entity and the entity vector in the knowledge graph. The calculation method is: calculate the ratio of the cosine similarity and Euclidean distance between the two, and normalize the result to record it as the entity matching index.

[0119] Cosine similarity indicates the degree of directional similarity between two vectors, while Euclidean distance indicates the spatial distance between two vectors. A smaller Euclidean distance indicates that the two vectors are closer in space, meaning they are more similar. The entity matching index measures the degree of match between the entity recognition factor and the entity vectors in the knowledge graph from two perspectives. A larger value indicates a greater cosine similarity between the two entity vectors and a smaller Euclidean distance, meaning the two entity vectors are more similar; the opposite indicates a greater dissimilarity.

[0120] Entities with entity matching indexes greater than the average value in the knowledge graph are recorded as candidate matching entities. The calculation formula for the entity matching index is:

[0121]

[0122] Among them, i is the i-th entity in the knowledge graph, E i is the entity recognition factor of the i-th entity in the image, cos() represents the calculation of cosine similarity, d() calculates the Euclidean distance and normalization, norm() represents normalization, P i is the entity matching index of the i-th entity in the knowledge graph. It should be noted that when d(E i ,ze i ) is 0, the preset adjustment parameter σ is used as the denominator to prevent the denominator from being 0, wherein the preset adjustment parameter is greater than 0, and is set to 0.01 in this embodiment, and can be adjusted by itself.

[0123] Based on the entity matching index of each entity, the degree of each candidate matching entity, and the number of entities connected to each candidate matching entity, the relevance of each entity is obtained. In this step, the entity relevance is calculated based on the entity matching index of each entity in the knowledge graph and the degree of the entity in the knowledge graph. The calculation steps are:

[0124] Calculate the degree of each entity in the knowledge graph, obtain the degree of the candidate matching entity, multiply the degree of the candidate matching entity by the degree of all entities connected to it, and sum them up. After normalization, add the entity matching index and record it as the entity relevance. The calculation formula for entity relevance is:

[0125]

[0126] In the above formula, GD i is the entity relevance of the i-th candidate matching entity in the knowledge graph, norm() indicates normalization, D i is the degree of the i-th candidate matching entity, D j is the degree of the jth entity connected to the ith candidate matching entity, and N is the number of entities connected to the ith candidate matching entity.

[0127] In this example, the degree refers to the number of other entities connected to an entity. A plant disease entity can be connected to multiple disease characteristic entities, and different plant disease entities may also be connected to the same disease characteristic entity. Therefore, it is necessary to take into account the other entities connected to the entity.

[0128] If a plant disease is more common, people will know more about it, and thus more descriptions of its characteristics will be available. This is reflected in the knowledge graph as more entities connected to it. Therefore, the more entities connected to the disease entity, that is, the higher the degree of the plant disease entity, the more common the disease is. Therefore, this factor is taken into account in plant disease identification, and more common diseases are identified first.

[0129] The entity association not only considers the correlation between the entity recognition factor and the alternative matching entity, but also the degree of the alternative matching entity. The greater the entity association, the more similar the alternative matching entity is to the entity recognition factor, and the more entities connected to the alternative matching factor, the easier it is to find the plant disease entity to be identified, and the plant disease entities with high entity association are output first.

[0130] Specifically, step S400 specifically includes: sorting the relevance of each entity in descending order, and obtaining candidate matching entities in the sorted knowledge graph.

[0131] Based on the candidate matching entities in the sorted knowledge graph, the identification results of plant diseases are obtained.

[0132] In this embodiment, the top three candidate matching entities can be selected and recorded as the first candidate matching entity, the second candidate matching entity, and the third candidate matching entity, respectively. Disease entities corresponding to the top three candidate matching entities in the knowledge graph are obtained, and the plant disease data corresponding to these disease entities are output.

[0133] Among them, the final disease entity is not necessarily one. For example, the disease entity corresponding to the first alternative matching entity in the knowledge graph is the first disease entity, and the disease entity corresponding to the second and third alternative matching entities is another disease entity, recorded as the second disease entity.

[0134] Then, the first disease entity is the best recognition result, and the second disease entity is the alternative recognition result. Similarly, if the first alternative matching entity has two disease entities corresponding to it, then both disease entities are regarded as the best recognition results.

[0135] The plant disease data corresponding to the disease entity of the final output recognition result includes: the name of the disease, pathogen, morphological characteristics, symptom identification, occurrence and distribution, prevention and control techniques and related literature, as well as pictures of the disease.

[0136] The present invention analyzes and processes the plant disease image data input by the user, takes into account the texture and color features of the image, segments the image, and then assigns different extraction weights to each segmented area, which not only improves the extraction accuracy but also improves the extraction speed, thereby improving the recognition accuracy of plant diseases.

[0137] The following describes a plant disease identification system based on a knowledge graph provided by the present invention. The plant disease identification system based on a knowledge graph described below and the plant disease identification method based on a knowledge graph described above can refer to each other.

[0138] The present invention provides a plant disease identification system based on knowledge graph, which includes a first acquisition module, a second acquisition module, a third acquisition module and an identification module.

[0139] The first acquisition module is used to acquire a picture of the plant disease to be identified input by the user.

[0140] The second acquisition module is used to perform image processing on the image of the plant disease to be identified, and obtain each entity and the number of each entity in the image.

[0141] The third acquisition module acquires each candidate matching entity and the association degree of each entity based on each entity and the number of each entity in the image and a preset knowledge graph.

[0142] The recognition module is used to obtain the recognition result of the plant disease based on each candidate matching entity and the association degree of each entity.

[0143] The second acquisition module is specifically used for:

[0144] The image of plant diseases to be identified is segmented using a preset initial segmentation scale, and the color and texture features in each segmented area are determined.

[0145] Based on the color features and texture features of each segmented area, the texture adjustment factor and color adjustment factor of each segmented area are determined. Specifically:

[0146] Based on the texture features in each segmented region, the texture complexity of each segmented region is obtained.

[0147] A texture adjustment factor in each segmented area is determined according to the texture complexity of each segmented area.

[0148] Based on the color features in each segmented area, the main color tone of each segmented area and the main color tones of the segmented areas adjacent to the segmented area are obtained.

[0149] A color adjustment factor in each segmented area is determined according to the main color hue of each segmented area and the main color hue of adjacent segmented areas.

[0150] After adjusting the preset initial segmentation scale using the texture adjustment factor and the color adjustment factor in each segmented area, the image of the plant disease to be identified is segmented again to obtain a segmented plant disease image;

[0151] A convolutional neural network is used to extract the segmented plant disease images and obtain each entity and the number of each entity in the image. Specifically, the segmented plant disease image is input into the input layer of the convolutional neural network, and the output layer of the convolutional neural network is used to obtain each entity and the number of each entity in the image.

[0152] The third acquisition module is specifically configured to: acquire an entity recognition factor of each entity based on each entity and the number of each entity in the image.

[0153] Based on the entity identification factor of each entity and the preset knowledge graph, obtain each candidate matching entity and each entity association degree. This step specifically comprises: based on the entity identification factor of each entity and the preset knowledge graph, obtain the entity matching index of each entity in the knowledge graph, each candidate matching entity, the degree of each candidate matching entity, and the number of entities connected to each candidate matching entity.

[0154] Based on the entity matching index of each entity in the knowledge graph, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity, the relevance degree of each entity is obtained.

[0155] The recognition module is specifically used to sort the association degree of each entity in descending order to obtain the candidate matching entities in the sorted knowledge graph.

[0156] Based on the candidate matching entities in the sorted knowledge graph, the identification results of plant diseases are obtained.

[0157] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0158] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A plant disease identification method based on knowledge graph, characterized in that: The method comprises: Obtain the picture of the plant disease to be identified input by the user; Performing image processing on the image of the plant disease to be identified to obtain each entity and the number of each entity in the image; Based on each entity and the quantity of each entity in the image and a preset knowledge graph, obtaining each candidate matching entity and the association degree of each entity; Based on each candidate matching entity and each entity association degree, obtaining a plant disease recognition result; The specific steps of performing image processing on the image of the plant disease to be identified and obtaining each entity and the number of each entity in the image include: The image of the plant disease to be identified is segmented using a preset initial segmentation scale to determine the color features and texture features in each segmented area; Determine a texture adjustment factor and a color adjustment factor in each segmented area based on the color features and texture features in each segmented area; After adjusting the preset initial segmentation scale using the texture adjustment factor and the color adjustment factor in each segmented area, the image of the plant disease to be identified is segmented again to obtain a segmented plant disease image; Use convolutional neural networks to extract the segmented plant disease images and obtain each entity and the number of each entity in the image; The specific steps of determining the texture adjustment factor and the color adjustment factor in each segmented area based on the color features and the texture features in each segmented area include: Based on the texture features in each segmented area, the texture complexity of each segmented area is obtained; Determining a texture adjustment factor within each segmented area according to the texture complexity of each segmented area; Based on the color features in each segmented area, the main color tone of each segmented area and the main color tone of the segmented areas adjacent to it are obtained; The color adjustment factor in each segmented area is determined according to the main color tone of each segmented area and the main color tone of the adjacent segmented areas.

2. The plant disease identification method based on knowledge graph according to claim 1, characterized in that: The specific steps of extracting the segmented plant disease picture by using a convolutional neural network and obtaining each entity and the number of each entity in the picture include: The segmented plant disease images are input into the input layer of the convolutional neural network, and the output layer of the convolutional neural network is used to obtain each entity and the number of each entity in the image.

3. The plant disease identification method based on knowledge graph according to claim 1, characterized in that: Based on each entity and the number of each entity in the image and the preset knowledge graph, the specific steps of obtaining each candidate matching entity and the relevance of each entity include: Based on each entity and the number of each entity in the image, obtaining an entity recognition factor for each entity; According to the entity recognition factor of each entity and the preset knowledge graph, each candidate matching entity and each entity association degree are obtained; The calculation formula of entity recognition factor is: in, is the number of the i-th entity in the image, N is the number of all entities in the image, is the i-th entity in the image, davg() means finding the average distance from one entity to other entities, represents the average distance from the i-th entity to other entities in the image, is the entity recognition factor of the i-th entity in the image.

4. The method for identifying plant diseases based on knowledge graph according to claim 3, characterized in that: According to the entity recognition factor of each entity and the preset knowledge graph, the specific steps of obtaining each candidate matching entity and the association degree of each entity include: According to the entity recognition factor of each entity and the preset knowledge graph, the entity matching index of each entity in the knowledge graph, each candidate matching entity, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity are obtained; Based on the entity matching index of each entity in the knowledge graph, the degree of each candidate matching entity and the number of entities connected to each candidate matching entity, the relevance degree of each entity is obtained.

5. The plant disease identification method based on knowledge graph according to claim 1, characterized in that: The specific steps of obtaining the identification result of plant diseases based on each candidate matching entity and each entity association degree include: Sorting the relevance of each entity in descending order to obtain candidate matching entities in the sorted knowledge graph; Based on the candidate matching entities in the sorted knowledge graph, the recognition results of plant diseases are obtained.

6. The method for identifying plant diseases based on knowledge graph according to any one of claims 1 to 5, characterized in that: The steps of constructing the preset knowledge graph specifically include: Obtain historical plant disease data including text data and image data; Performing data processing on text data in historical plant disease data to obtain processed text data; Performing data processing on the image data in the historical plant disease data to obtain processed image data; Build a preset knowledge graph based on the processed text data, processed image data and knowledge graph model.

7. The method for identifying plant diseases based on knowledge graph according to claim 6, characterized in that: The specific steps of processing the image data in the historical plant disease data to obtain the processed image data include: Gaussian filtering and denoising are performed on the image data in the historical plant disease data to obtain processed image data.

8. A plant disease identification system based on knowledge graph, characterized in that: include: A first acquisition module is used to acquire a picture of a plant disease to be identified input by a user; A second acquisition module is used to process the picture of the plant disease to be identified, and obtain each entity and the number of each entity in the picture; The specific steps of performing image processing on the image of the plant disease to be identified and obtaining each entity and the number of each entity in the image include: The image of the plant disease to be identified is segmented using a preset initial segmentation scale to determine the color features and texture features in each segmented area; Determine a texture adjustment factor and a color adjustment factor in each segmented area based on the color features and texture features in each segmented area; After adjusting the preset initial segmentation scale using the texture adjustment factor and the color adjustment factor in each segmented area, the image of the plant disease to be identified is segmented again to obtain a segmented plant disease image; Use convolutional neural networks to extract the segmented plant disease images and obtain each entity and the number of each entity in the image; The specific steps of determining the texture adjustment factor and the color adjustment factor in each segmented area based on the color features and the texture features in each segmented area include: Based on the texture features in each segmented area, the texture complexity of each segmented area is obtained; Determining a texture adjustment factor within each segmented area according to the texture complexity of each segmented area; Based on the color features in each segmented area, the main color tone of each segmented area and the main color tone of the segmented areas adjacent to it are obtained; Determine a color adjustment factor in each segmented area according to the main color tone of each segmented area and the main color tone of the segmented areas adjacent to the segmented area; A third acquisition module, based on each entity and the number of each entity in the image and a preset knowledge graph, acquires each candidate matching entity and the association degree of each entity; The recognition module is used to obtain the recognition result of the plant disease based on each candidate matching entity and each entity association degree.

Images