Diagnostic device and diagnostic system

Abstract

To enable accurate diagnosis of plant disorders.SOLUTION: A processor (11) of a diagnostic device (10) disclosed herein is configured to acquire multiple captured images of one plant from a user device, and diagnose disorders of the plant from the multiple images using a diagnostic model machine-trained on correlation relationships between disorders and images of affected parts of plants suffering from disorders that are stored in a storage device (14).SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to a diagnostic apparatus and a diagnostic system.

Background Art

[0002] Conventionally, inventions for automatically diagnosing disorders such as pests occurring in plants are known (for example, Patent Document 1 and Patent Document 2). The invention described in Patent Document 1 diagnoses plant pests using an image diagnosis model constructed by machine learning. The invention described in Patent Document 2 extracts feature data from an image and diagnoses plant pests based on the extracted feature data.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the methods described in Patent Document 1 and Patent Document 2, since diagnosis is performed using a single image, there is a risk that the diagnostic accuracy may be low.

[0005] One aspect of the present disclosure aims to accurately diagnose plant disorders.

Means for Solving the Problems

[0006] In order to solve the above problems, a diagnostic apparatus according to an aspect of the present disclosure is a diagnostic apparatus capable of communicating with a user device and diagnosing a disorder occurring in a plant, and includes a processor and a storage device. The processor acquires a plurality of images of one plant from the user device, and diagnoses the disorder of the plant from the plurality of images using a diagnostic model obtained by machine learning the correlation between an image of an affected part of a plant in which the disorder has occurred and the disorder, which is stored in the storage device.

[0007] In order to solve the above problems, a diagnostic system according to an aspect of the present disclosure is a diagnostic system including a user device and a diagnostic apparatus capable of communicating with the user device and diagnosing a disorder occurring in a plant. The diagnostic apparatus includes a processor and a storage device. The processor acquires a plurality of images of one plant from the user device, and diagnoses the disorder of the plant from the plurality of images using a diagnostic model obtained by machine learning the correlation between an image of an affected part of a plant in which the disorder has occurred and the disorder, which is stored in the storage device.

[0008] The diagnostic apparatus according to each aspect of the present disclosure may be implemented by a computer. In this case, a control program for implementing the diagnostic apparatus by operating a computer as each part (software element) included in the diagnostic apparatus, and a computer-readable recording medium on which the program is recorded also fall within the scope of the present disclosure.

Advantages of the Invention

[0009] According to one aspect of the present disclosure, it is possible to accurately diagnose a disorder of a plant.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

BEST MODE FOR CARRYING OUT THE INVENTION

[0011] 〔Embodiment 1〕 Hereinafter, the diagnostic system 100 according to Embodiment 1 of the present disclosure will be described in detail with reference to the drawings. In the description of the drawings, the same reference numerals are given to the same parts and the description thereof will be omitted.

[0012] (Outline of Diagnostic System 100) As shown in FIG. 1, the diagnostic system 100 includes a diagnostic apparatus 10 and user apparatuses 30, 30A, 30B respectively held by a plurality of users 20, 20A, 20B. In FIG. 1, three users are shown, but the number is not limited to three, and a large number of users may exist.

[0013] The diagnostic apparatus 10 is configured to be able to communicate with each of the user apparatuses 30, 30A, 30B via a communication network 40. The communication network 40 is, for example, the Internet, but is not limited thereto, and other wireless communication methods may be adopted.

[0014] The user devices 30, 30A, and 30B are installed with a dedicated application program for posting and viewing information related to plants, such as plant cultivation and harvesting. This application program also has a function for diagnosing plant disorders.

[0015] In this embodiment, "plant disorder" refers to at least any one or a combination thereof among diseases, pests, and physiological disorders occurring in plants. Examples of "plant disorders" include powdery mildew, leaf spot disease, bacterial wilt, aphids, whiteflies, thrips, spider mites, etc.

[0016] The "plant" in this embodiment may be an edible plant such as a vegetable or fruit tree, or an ornamental plant such as a flower.

[0017] The diagnostic device 10 is a device for operating and managing services using the application program, and is, for example, a server. The installation location of the diagnostic device 10 is not particularly limited. For example, the diagnostic device 10 may be installed in the management center of an operator who operates and manages services using the application program. The "services using the application program" include a plurality of services. In this embodiment, one of the services, namely the plant disorder diagnosis service, will be taken up and described. The diagnostic device 10 can provide the plant disorder diagnosis service to users 20, 20A, and 20B through communication via the communication network 40.

[0018] Hereinafter, for the sake of convenience of explanation, among the users 20, 20A, 20B and the user devices 30, 30A, 30B, the user 20 and the user device 30 will be taken up and described, and the description of the users 20A, 20B and the user devices 30A, 30B, which have the same components as the user 20 and the user device 30, will be omitted.

[0019] (An example of a scene where the plant disorder diagnosis service is used) FIG. 2 is a diagram showing an example of a usage scenario of a plant disorder diagnosis service provided by the diagnostic device 10. The plant 50 is a plant cultivated by the user 20. Suppose the user 20 discovers a mutation in the plant 50 that is predicted to be caused by a disorder. At this time, the user 20 attempts to use the disorder diagnosis service to obtain a diagnosis result of what disorder is occurring in the plant 50.

[0020] As an example of a method for obtaining a diagnosis result, the user 20 activates an application program installed in the user device 30 to open a diagnosis page. The application program requests the user 20 to photograph the plant 50 to be diagnosed. In accordance with this request, as shown in FIG. 2, the user 20 photographs the plant 50 to obtain an image of the plant 50. In the present embodiment, the application program requests the user 20 to obtain a plurality of images of the plant 50 to be diagnosed.

[0021] The plurality of images of the plant 50 photographed by the user 20 are transmitted from the user device 30 to the diagnostic device 10 via the communication network 40. The diagnostic device 10 diagnoses the disorder of the plant 50 using the plurality of images of the plant 50. Hereinafter, an example of the diagnosis method will be described together with the hardware configuration of the diagnostic device 10 and the like. Note that the plant 50 to be diagnosed has been described as being cultivated by the user 20, but it is not limited thereto, and it may not be cultivated by the user 20.

[0022] (Hardware Configuration of Diagnostic Device 10) Next, an example of the hardware configuration of the diagnostic device 10 will be described with reference to FIG. 3. FIG. 3 is a block diagram showing the hardware configuration of the diagnostic device 10.

[0023] As shown in FIG. 3, the diagnostic device 10 includes a processor 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage device 14, and a communication I / F 15. Each component is connected to be communicable with each other via a bus 16.

[0024] The processor 11 is an arithmetic unit that executes various programs, such as a CPU (Central Processing Unit). The processor 11 reads a program from the ROM 12 or the storage device 14 and executes various programs using the RAM 13 as a working area. In FIG. 3, one processor 11 is shown, but it is not limited thereto, and a plurality of processors 11 may be provided.

[0025] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores a program or data as a working area. Note that the computer-readable recording medium is not limited to the ROM 12 and the RAM 13, and may include an EPROM (Erasable Programmable ROM), an EEPROM (registered trademark) (Electrically Erasable Programmable ROM), and the like.

[0026] The storage device 14 is composed of an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, etc., and stores various programs and various data. FIG. 3 shows an example in which a diagnostic model database 141 is stored in the storage device 14. The diagnostic model database 141 stores a diagnostic model for diagnosing the disorders of the plant 50.

[0027] The "diagnostic model" mentioned here is, for example, a learned model obtained by machine learning the correlation between the affected part images showing various disorders of the plant and the disorders. The "affected part image of the plant" is an image taken of the part where the disorder has occurred. The machine learning method is not particularly limited, and for example, a well-known convolutional neural network may be used. The convolutional neural network can extract complex patterns and features due to its multi-layer structure and construct a diagnostic model suitable for image recognition. Note that the diagnostic model may be a learned model that outputs a diagnostic result indicating that there is "no disorder" depending on the affected part image.

[0028] In order to construct a diagnostic model, it is necessary to machine-learn a large number of images. The method for acquiring a large number of images is not particularly limited. For example, if a bot is used, a large number of images can be automatically collected. A bot is an application program that executes automated tasks on the Internet. When machine-learning the collected images, as preprocessing of the images, processes such as enlargement / reduction, rotation, noise removal, and partial cropping by plant detection may be performed. Such preprocessing enables efficient machine learning.

[0029] The communication I / F 15 is implemented as hardware such as a network adapter, communication software, and combinations thereof, and communicates with the user device 30 via the communication network 40.

[0030] (Hardware configuration of the user device 30) Next, an example of the hardware configuration of the user device 30 will be described with reference to FIG. 4. FIG. 4 is a block diagram showing the hardware configuration of the user device 30. The user device 30 is a portable terminal that can be carried by the user 20, and is, for example, a smartphone, a tablet terminal, a wearable device, or the like. In the present embodiment, as an example, the user device 30 will be described as a smartphone.

[0031] As shown in FIG. 3, the user device 30 includes a processor 31, a ROM 32, a RAM 33, a storage device 34, a communication I / F 35, a camera 36, and a display 37. Each component is connected to be communicable with each other via a bus 38.

[0032] The configurations of the processor 31, the ROM 32, the RAM 33, the storage device 34, and the communication I / F 35 are the same as those of the processor 11, the ROM 12, the RAM 13, the storage device 14, and the communication I / F 15 of the diagnostic device 10 described above, and thus the description thereof will be omitted.

[0033] The memory device 34 has an application program 341 installed therein, which is used for the plant disorder diagnosis service. Hereinafter, the application program 341 will be simply referred to as "app 341". The app 341 functions when the processor 31 reads and executes a dedicated application program from the memory device 34.

[0034] The camera 36 has an imaging element such as a CCD (charge - coupled device) or a CMOS (complementary metal oxide semiconductor). As an example of the use of the camera 36, photographing the plant 50 to be diagnosed can be mentioned. The image of the plant 50 photographed by the camera 36 is stored in the memory device 34 and transmitted to the diagnostic device 10 by the function of the app 341.

[0035] The display 37 is composed of a liquid crystal display or an organic EL display, etc., and displays various information such as the diagnostic result. Further, the display 37 is provided with a capacitive touch sensor and also functions as an input device that receives the touch operation of the user 20 as an input operation.

[0036] (Function of the processor 11 of the diagnostic device 10) Next, with reference to FIG. 5, an example of the function of the processor 11 of the diagnostic device 10 will be described. FIG. 5 is a functional block diagram of the processor 11 of the diagnostic device 10. As shown in FIG. 5, in the diagnostic device 10 of the present embodiment, the processor 11 functions as an image acquisition unit 111, a diagnostic unit 112, and an output unit 113 by executing a specific program.

[0037] The image acquisition unit 111 acquires a plurality of images of the plant 50 photographed by the user 20 from the user device 30 via the communication network 40. The image acquisition unit 111 outputs the acquired plurality of images of the plant 50 to the diagnostic unit 112.

[0038] The diagnosis unit 112 inputs the plurality of images acquired from the image acquisition unit 111 into the diagnosis model stored in the diagnosis model database 141 to diagnose the disorders of the plant 50. The diagnosis unit 112 outputs the diagnosis result to the output unit 113.

[0039] The output unit 113 transmits the diagnosis result diagnosed by the diagnosis unit 112 to the user device 30. As a result, the diagnosis result is displayed on the app 341, and the user 20 can know the diagnosis result.

[0040] (An example of the diagnosis method) Next, with reference to the sequence chart of FIG. 6, an example of a method for diagnosing the disorders of the plant 50 will be described. The scene of the sequence chart in FIG. 6 is not particularly limited, but here, as described with reference to FIG. 2, it will be described as a scene in which the user 20 is trying to obtain the diagnosis result of what kind of disorder has occurred in the plant 50.

[0041] In step S101, when the user 20 taps the icon of the app 341 displayed on the display 37, the app 341 is launched.

[0042] The process proceeds to step S102, and the app 341 requests the user 20 to take an image of the plant 50 to be diagnosed. More specifically, the app 341 requests the user 20 to take an image of the affected part where a disorder has occurred for each part of the plant 50. The "parts of the plant 50" here includes at least five parts: the growth point of the plant 50, upper leaves, middle leaves, lower leaves, and fruits. Additionally, the "parts of the plant 50" may include the stem of the plant 50. In FIG. 6, the description is based on the premise that disorders have occurred in all parts of the plant 50. However, it is not limited to this premise. Examples not meeting this premise will be described later.

[0043] As an example of a method for requesting to capture images for each part, the application 341 may request the user 20, by voice, "Please capture an image of the growth point of the plant 50", "Next, please capture an image of the upper leaves of the plant 50", "Next, please capture an image of the middle leaves of the plant 50", "Next, please capture an image of the lower leaves of the plant 50", "Next, please capture an image of the fruit of the plant 50". Also, instead of voice, the application 341 may display character information on the application 341 and request to capture images for each part.

[0044] The user 20 captures images of the growth point, upper leaves, middle leaves, lower leaves, and fruit of the plant 50 according to the request of the application 341. As a result, a plurality of images with different parts are obtained.

[0045] In step S103, the "image of the growth point of the plant 50", "image of the upper leaves of the plant 50", "image of the middle leaves of the plant 50", "image of the lower leaves of the plant 50", and "image of the fruit of the plant 50" captured by the user 20 are transmitted from the user device 30 to the diagnostic device 10. Here, it is assumed that there is one image for each part. Therefore, the number of images transmitted from the user device 30 to the diagnostic device 10 is five.

[0046] In step S104, the processor 11 of the diagnostic device 10 acquires a plurality of images transmitted from the user device 30.

[0047] In step S105, the processor 11 of the diagnostic device 10 diagnoses the disorders of the plant 50 by using the plurality of images acquired in the process of step S104 and the diagnostic model stored in the diagnostic model database 141. Specifically, first, the processor 11 of the diagnostic device 10 inputs the "image of the growth point of the plant 50" into the diagnostic model to diagnose the disorder of the growth point. In this process, the "disorder occurring at the growth point" and the "probability of that disorder" are obtained. Here, as an example of the "disorder occurring at the growth point", it is assumed that four disorders, thrips, spider mites, leaf spot disease, and bacterial wilt, are diagnosed. Also, it is assumed that the probability of thrips is A (%), the probability of spider mites is B (%), the probability of leaf spot disease is C (%), and the probability of bacterial wilt is D (%).

[0048] Next, the processor 11 of the diagnostic device 10 inputs the "image of the upper leaves of the plant 50" into the diagnostic model to diagnose the disorder of the upper leaves. In this process, the "disorder occurring at the upper leaves" and the "probability of that disorder" are obtained. As the "disorder occurring at the upper leaves", it is assumed that the same four disorders, thrips, spider mites, leaf spot disease, and bacterial wilt, as those occurring at the growth point are diagnosed. Also, it is assumed that the probability of thrips is A1 (%), the probability of spider mites is B1 (%), the probability of leaf spot disease is C1 (%), and the probability of bacterial wilt is D1 (%).

[0049] Next, the processor 11 of the diagnostic device 10 inputs the "image of the middle leaves of the plant 50" into the diagnostic model to diagnose the disorder of the middle leaves. In this process, the "disorder occurring at the middle leaves" and the "probability of that disorder" are obtained. As the "disorder occurring at the middle leaves", it is assumed that the same four disorders, thrips, spider mites, leaf spot disease, and bacterial wilt, as those occurring at the growth point are diagnosed. Also, it is assumed that the probability of thrips is A2 (%), the probability of spider mites is B2 (%), the probability of leaf spot disease is C2 (%), and the probability of bacterial wilt is D2 (%).

[0050] Next, the processor 11 of the diagnostic device 10 inputs the "image of the lower leaves of the plant 50" into the diagnostic model to diagnose the disorders of the lower leaves. In this process, the "disorders occurring in the lower leaves" and "the probability of those disorders" are obtained. Suppose that as the "disorders occurring in the lower leaves", the same four disorders as those in the "disorders occurring at the growing points", i.e., thrips, spider mites, leaf spot disease, and bacterial wilt, are diagnosed. Also, suppose that the probability of thrips is diagnosed as A3 (%), the probability of spider mites is B3 (%), the probability of leaf spot disease is C3 (%), and the probability of bacterial wilt is D3 (%).

[0051] Next, the processor 11 of the diagnostic device 10 inputs the "image of the fruits of the plant 50" into the diagnostic model to diagnose the disorders of the fruits. In this process, the "disorders occurring in the fruits" and "the probability of those disorders" are obtained. Suppose that as the "disorders occurring in the fruits", the same four disorders as those in the "disorders occurring at the growing points", i.e., thrips, spider mites, leaf spot disease, and bacterial wilt, are diagnosed. Also, suppose that the probability of thrips is diagnosed as A4 (%), the probability of spider mites is B4 (%), the probability of leaf spot disease is C4 (%), and the probability of bacterial wilt is D4 (%).

[0052] Next, the processor 11 of the diagnostic device 10 calculates the average of the probabilities of the disorders diagnosed for each part of the plant 50. For thrips, the processor 11 of the diagnostic device 10 calculates the average of A, A1, A2, A3, and A4. The averages are calculated in the same way for spider mites, leaf spot disease, and bacterial wilt. The processor 11 of the diagnostic device 10 can diagnose the disorder with the highest calculated average probability as the disorder occurring in the plant 50. Here, assume that the disorder with the highest calculated average probability is thrips.

[0053] The process proceeds to step S106, and the processor 11 of the diagnostic device 10 transmits the information indicating the diagnostic result to the user device 30.

[0054] In step S107, the application 341 acquires the information indicating the diagnostic result from the diagnostic device 10.

[0055] The process proceeds to step S108, and the application 341 displays the diagnostic result obtained in the process of step S107 on the application 341.

[0056] The method of displaying the diagnostic result is not particularly limited. For example, the application 341 may display character information such as "Isn't it thrips?" on the application 341. If the pest with the next highest average probability after thrips is spider mites, character information such as "Isn't it thrips?" may be displayed while character information such as "Other possible symptoms: spider mites" is also displayed. As another display example, the average of the four diagnosed pests and their probabilities may be displayed. By displaying the diagnostic result in this way, the user 20 can know the pest occurring in the plant 50.

[0057] In addition, in FIG. 6, an example of diagnosing a pest using five images with different parts has been described, but the number of images used for diagnosis is not limited to five, and five or more images may be used. Also, a limit may be set on the number of images used in one diagnosis, or such a limit may not be set and it may be unlimited.

[0058] In addition, in FIG. 6, an example in which all the parts are different in the five images has been described, but it is not limited to this. For example, in the five images, all may be the same part, or a combination such as four being the same part and the remaining one being a different part may be used. Alternatively, a combination such as three being the same part and the remaining two being different parts may be used, or other combinations may be used.

[0059] As a specific example, all five may be images of upper leaves, or four of the five may be images of upper leaves and the remaining one may be an image of middle leaves. Three of the five may be images of upper leaves and the remaining two may be images of middle leaves. Also, three of the five may be images of upper leaves, one of the remaining two may be an image of middle leaves, and the other one may be an image of lower leaves. However, when using multiple images of the same part, it is necessary to photograph the corresponding part from different angles. This is because even if there are multiple images of the same part taken from the same angle, it does not contribute to improving the diagnostic accuracy.

[0060] When acquiring images of the same part, the application 341 may request the user 20 to, for example, "take pictures of the upper leaves at different angles". Even when diagnosing using multiple images of the same part, it can be diagnosed in the same way as the above-described method. For example, assume that all 5 images are of the upper leaves. In this case, the processor 11 of the diagnostic device 10 inputs each image into the diagnostic model to diagnose the disorder of the upper leaves. As a result, for each image, the disorder and its probability can be obtained. Thus, similar to the above-described method, the average of the probabilities may be calculated, and the disorder with the highest calculated average probability may be diagnosed as the disorder occurring in the plant 50.

[0061] Also, as described above, the disorder does not necessarily occur in all parts of the plant 50. Depending on the part, there may be a case where there are parts where the disorder occurs and parts where the disorder does not occur. As a specific example, while no disorder occurs at the growth point, a disorder may occur in the upper leaves. In this case, for example, even if the user 20 is requested to "take a picture of the growth point", from the perspective of the user 20 who recognizes that there is no disorder at the growth point and it is normal, there is little need to take a picture of the normal growth point. Therefore, in this case, the user 20 may omit taking a picture of the growth point. For example, the user 20 can omit taking a picture of the growth point by tapping an icon indicating skip on the application 341.

[0062] The flow of the process in the sequence chart shown in FIG. 6 is an example, and steps may be deleted, new steps may be added, or the processing order may be changed within the scope not departing from the gist.

[0063] (Function and effect) As described above, according to the diagnostic system 100 according to Embodiment 1, the following function and effect can be obtained.

[0064] The diagnostic device 10 is capable of communicating with the user device 30 and is a device that diagnoses disorders occurring in the plant 50, and includes a processor 11 and a storage device 14. The processor 11 of the diagnostic device 10 acquires a plurality of images of one plant 50 from the user device 30, and uses a diagnostic model obtained by machine learning the correlation between the affected part image of the plant with a disorder and the disorder stored in the storage device 14 to diagnose the disorder of the plant 50 from the plurality of images.

[0065] According to the above configuration, by diagnosing the disorder of the plant 50 using a plurality of images, the diagnostic accuracy can be improved.

[0066] Further, the processor 11 of the diagnostic device 10 may acquire a plurality of images of different parts of one plant 50 from the user device 30, and diagnose the disorder of the plant 50 using different diagnostic models for each part.

[0067] In FIG. 6, an example of diagnosing a disorder using a predetermined diagnostic model has been described, but it is not limited thereto. For example, dedicated diagnostic models optimized for each part of the plant 50 may be prepared, and the dedicated diagnostic models may be applied to each part. As dedicated diagnostic models for each part, dedicated diagnostic models such as a "diagnostic model for the growth point", a "diagnostic model for upper leaves", a "diagnostic model for middle leaves", a "diagnostic model for lower leaves", and a "diagnostic model for fruits" may be prepared in the diagnostic model database 141.

[0068] If the image is of the growth point, the processor 11 of the diagnostic device 10 can diagnose the disorder of the growth point by inputting the image into the diagnostic model for the growth point. The same applies to upper leaves, middle leaves, lower leaves, and fruits. By diagnosing the disorder using dedicated diagnostic models optimized for each part in this way, the diagnostic accuracy can be improved.

[0069] Further, the processor 11 of the diagnostic device 10 may output information indicating the diagnostic result of diagnosing the disorder of the plant 50 to the user device 30.

[0070] According to the above configuration, the user 20 can know at a glance the disorders occurring in the plant 50 by checking the diagnosis result displayed on the application 341.

[0071] 〔Embodiment 2〕 Next, referring to FIGS. 7 to 9, Embodiment 2 will be described.

[0072] What is different between Embodiment 2 and Embodiment 1 is that the processor 11 of the diagnostic device 10 corrects the diagnosis result using the interrogation information regarding the interrogation. For the configurations overlapping with those of Embodiment 1, the same reference numerals are used and the description thereof is omitted. Hereinafter, the description will focus on the differences.

[0073] FIG. 7 is a block diagram showing the hardware configuration of the diagnostic device 10 according to Embodiment 2. As a configuration different from that of Embodiment 1, in Embodiment 2, an interrogation database 142 is stored in the storage device 14. The "interrogation database 142" is a database in which the answers to the interrogation of the plant 50 and the parameters for correcting the diagnosis result are associated.

[0074] In Embodiment 2, after the user 20 photographs the plant 50, the application 341 requests the user 20 to input the interrogation of the plant 50. Regarding the condition of "the user 20 photographs the plant 50", the application 341 may determine that the condition is satisfied by using, as a trigger, that an image has been acquired by the camera 36. After the user 20 photographs the plant 50, the application 341 changes the screen and displays a screen for inputting the interrogation, and requests the user 20 to input the interrogation. Note that the order of photographing and interrogation input may be reversed. After the user 20 inputs the interrogation, the application 341 may request the user 20 to photograph the plant 50.

[0075] (An example of the correction method using interrogation information) Next, an example of a method for correcting a diagnostic result using medical interview information will be described with reference to Figs. 8 and 9. Fig. 8 is a sequence chart illustrating an example of the operation of diagnostic device 10 and user device 30 according to embodiment 2. Fig. 9 is a table illustrating parameters for correcting a diagnostic result according to a response to a medical interview by user 20.

[0076] The processing in steps S201 and S202 in FIG. 8 is similar to the processing in steps S101 and S102 shown in FIG. 6, and therefore description thereof will be omitted.

[0077] In step S203 of Fig. 8, after the user 20 photographs the plant 50, the app 341 requests the user 20 to input a medical question about the plant 50. Examples of the medical question about the plant 50 include multiple items such as "how the damage spreads," "where the symptoms appear," and "what the symptoms are," but here, "how the damage spreads" will be taken as an example and described. That is, the app 341 displays a screen for inputting the medical question and requests the user 20 to input "how the damage spreads."

[0078] When the user 20 is requested to input the questionnaire, the user 20 answers by selecting one of the options displayed on the application 341. As shown in Fig. 9, examples of the options include "only a few plants", "multiple occurrences in one place", "multiple occurrences mainly around the ventilation opening", and "scattered occurrences throughout the field". The user 20 answers by selecting one of these options.

[0079] The process proceeds to step S204, where the multiple images captured by the user 20 and medical interview information, which is the answer to the medical interview, are transmitted from the user device 30 to the diagnostic device .

[0080] In step S205, the processor 11 of the diagnostic device 10 acquires the multiple images and interview information transmitted from the user device 30.

[0081] In step S206, the processor 11 of the diagnostic device 10 diagnoses the disorders of the plant 50 by using the plurality of images acquired in the process of step S205 and the diagnostic model stored in the diagnostic model database 141. Similar to the first embodiment, it is assumed that in the second embodiment, four disorders, i.e., thrips, spider mites, leaf spot disease, and bacterial wilt, are diagnosed by this process. Also, it is assumed that the calculation of the average of the probabilities of disorders for each part is the same as in the first embodiment.

[0082] The process proceeds to step S207, and the processor 11 of the diagnostic device 10 corrects the diagnostic result according to the interview information acquired in the process of step S205. Since the diagnostic result is indicated by the average probability, "correcting the diagnostic result" means correcting the average probability. Hereinafter, an example of the method for correcting the diagnostic result will be described.

[0083] The processor 11 of the diagnostic device 10 compares the user 20's answer to the interview with the interview database 142 to obtain parameters for correcting the diagnostic result. As shown in FIG. 9, when the user 20's answer is "only a few plants", the parameter for correcting the average probability related to thrips, spider mites, and leaf spot disease is zero, and the parameter for correcting the average probability related to bacterial wilt is +20%. Therefore, the processor 11 of the diagnostic device 10 does not correct the average probability for thrips, spider mites, and leaf spot disease, but for bacterial wilt, it adds 20% to the average probability for correction.

[0084] Also, when the user 20's answer is "multiple occurrences concentrated in one place", as shown in FIG. 9, the parameter for correcting the average probability related to thrips is +10%, the parameters for correcting the average probabilities related to spider mites and bacterial wilt are +20%, and the parameter for correcting the average probability related to leaf spot disease is zero. Therefore, the processor 11 of the diagnostic device 10 does not correct the average probability for leaf spot disease, but for thrips, it adds 10% to the average probability for correction, and for spider mites and bacterial wilt, it adds 20% to the average probability for correction.

[0085] Also, when the answer of user 20 is "frequently occurring mainly around the ventilation openings", as shown in FIG. 9, the parameter for correcting the average probability related to thrips and spider mites is +30%, and the parameter for correcting the average probability related to leaf spot disease and bacterial wilt is zero. Therefore, the processor 11 of the diagnostic device 10 adds 30% to the average probability and corrects it for thrips and spider mites, while does not correct the average probability for leaf spot disease and bacterial wilt.

[0086] Also, when the answer of user 20 is "occurring variably throughout the field", as shown in FIG. 9, the parameter for correcting the average probability related to thrips and spider mites is zero, the parameter for correcting the average probability related to leaf spot disease is +20%, and the parameter for correcting the average probability related to bacterial wilt is -10%. Therefore, the processor 11 of the diagnostic device 10 does not correct the average probability for thrips and spider mites, but adds 20% to the average probability and corrects it for leaf spot disease, and subtracts 10% from the average probability and corrects it for bacterial wilt.

[0087] By correcting the average probability in this way, the processor 11 of the diagnostic device 10 can diagnose the disorder with the highest corrected average probability as the disorder occurring in the plant 50.

[0088] In the example of FIG. 9, the method of correcting the average probability by addition and subtraction has been described, but it is not limited thereto, and the average probability may be corrected by multiplication or division.

[0089] Also, the parameter for correcting the diagnostic result may be a fixed value or a variable value. When the parameter for correcting the diagnostic result is treated as a variable, the value may be changed, for example, monthly or seasonally. Also, since there are differences in the susceptibility of pests and diseases to occur in each region, the parameter may be changed for each region.

[0090] The processes of steps S208 to S210 in FIG. 8 are the same as the processes of steps S106 to S108 shown in FIG. 6, and thus the description thereof is omitted.

[0091] The processing flow in the sequence chart shown in FIG. 8 is an example, and steps may be deleted, new steps may be added, or the processing order may be changed within the scope not departing from the gist. For example, the processing order of step S202 and step S203 may be reversed.

[0092] (Function and effect) As described above, according to the diagnostic system 100 according to Embodiment 2, the following function and effect can be obtained.

[0093] The processor 11 of the diagnostic device 10 may acquire inquiry information regarding the inquiry of the plant from the user device 30 and diagnose the plant's disorder from a plurality of images using the diagnostic model and the inquiry information.

[0094] According to the above configuration, by correcting the diagnostic result using the inquiry information, the accuracy of the diagnostic result can be improved.

[0095] Further, the processor 11 of the diagnostic device 10 may calculate the average value of the probability that a disorder has occurred for each of the plurality of images, and correct the average value using a correction value based on the inquiry information to diagnose the plant's disorder. The "correction value" here means a parameter for correcting the diagnostic result.

[0096] According to the above configuration, by correcting the average value of the probability that a disorder has occurred using the inquiry information, the accuracy of the diagnostic result can be improved.

[0097] In Embodiment 2, an example of correcting the average probability that is the diagnosis result using the interview information has been described, but the method of using the interview information is not limited to this. For example, using the interview information, an image that may be noise among a plurality of images may be excluded. When it is presumed from the interview information that the disorder of the plant 50 is a disease that occurs only in the stem and roots, it is considered that the growth point and the upper leaves are healthy without the occurrence of the disease. Therefore, since using the images of the growth point and the upper leaves may lead to a decrease in the accuracy of the average probability that is the diagnosis result, the average probability may be calculated by excluding the images of the growth point and the upper leaves. Alternatively, using the interview information, the above-mentioned probabilities A, B, C, and D in the image of the growth point and the above-mentioned probabilities A1, B1, C1, and D1 in the image of the upper leaves may be corrected to 0%. Thereby, it becomes possible to exclude the influence of the images of the growth point and the upper leaves.

[0098] 〔Other Embodiments〕 The storage device 14 of the diagnostic device 10 may store map data of a predetermined area where the user 20 cultivates the plant 50. The "predetermined area" here means a place where the plant 50 can be cultivated, and includes a paddy field, a field, an agricultural land, a greenhouse, an orchard, a home garden made in the yard of one's own house, a planter installed on the veranda of one's own house, a rooftop garden made on the rooftop of a building, and the like. A usage example of the map data of the predetermined area will be described.

[0099] When the user device 30 is provided with a GPS receiver, by using the GPS, the position information of the shooting location can be associated with the image taken by the user 20. The processor 11 of the diagnostic device 10 can identify where in the predetermined area the plant 50 to be diagnosed is cultivated by comparing the position information associated with the image with the map data of the predetermined area. As the classification of the cultivation location, it can be classified into the center of the ridge, the edge of the ridge, near the ventilation opening, and the like.

[0100] Parameters for correcting the diagnostic results according to the cultivation location may be prepared, and the diagnostic results may be corrected using these parameters. That is, when diagnosing the damage of the plant 50 from a plurality of images, the processor 11 of the diagnostic device 10 may diagnose the damage of the plant 50 using parameters related to the cultivation location. Suppose that, based on the position information associated with the image, it is specified that the cultivation location of the plant 50 is near the ventilation opening. Since the area near the ventilation opening is susceptible to the influence of pests that have entered through the ventilation opening, the processor 11 of the diagnostic device 10 may add a predetermined parameter to the diagnostic result.

[0101] Also, suppose that, based on the position information associated with the image, it is specified that the cultivation location of the plant 50 is at the end of the ridge. Since pests are likely to enter at the end of the ridge, the processor 11 of the diagnostic device 10 may add a predetermined parameter to the diagnostic result.

[0102] By thus specifying the cultivation location of the plant 50 in a predetermined area and correcting the diagnostic result using parameters related to the cultivation location, the accuracy of the diagnostic result can be improved.

[0103] When correcting the diagnostic result using parameters related to the cultivation location, the above-mentioned inquiry information may also be taken into account to correct the diagnostic result. In other words, the processor 11 of the diagnostic device 10 may correct the diagnostic result of diagnosing the damage of the plant 50 from a plurality of images using the diagnostic model, using the inquiry information and parameters related to the cultivation location, and diagnose the damage of the plant 50. Thereby, the accuracy of the diagnostic result can be further improved.

[0104] 〔Example of implementation by software〕 The functions of the diagnostic device 10 can be realized by a program for causing a computer to function as the diagnostic device 10, and by programs for causing a computer to function as each control block of the diagnostic device 10.

[0105] In this case, the diagnostic device 10 includes a computer having at least one device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing a program. When the computer executes the program, each function described in each embodiment is realized.

[0106] The program may be recorded on one or more computer-readable recording media, rather than temporarily. This recording medium may or may not be included in the diagnostic device 10. In the latter case, the program may be supplied to the diagnostic device 10 via any wired or wireless transmission medium.

[0107] Also, part or all of the functions of each control block can also be realized by a logic circuit. For example, an integrated circuit in which a logic circuit functioning as each control block is formed is also included in the scope of the present disclosure. In addition to this, for example, it is also possible to realize the functions of each control block by a quantum computer.

[0108] Also, each process described in each embodiment may be executed by AI (Artificial Intelligence). In this case, the AI may operate in the diagnostic device 10, or may operate in another device (e.g., an edge computer or a cloud server).

[0109] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present disclosure.

Explanation of Reference Numerals

[0110] 100 Diagnostic system, 10 Diagnostic device, 11 Processor, 14 Storage device, 30 User device

Claims

1. A diagnostic device capable of communicating with a user device and diagnosing a disorder occurring in a plant, comprising: a processor; a storage device, and the processor is configured to: obtain a plurality of images of one plant captured from the user device; diagnose the disorder of the plant from the plurality of images using a diagnostic model obtained by machine learning the correlation between the affected part image of the plant with the disorder and the disorder stored in the storage device.

2. The processor is further configured to: obtain inquiry information regarding an inquiry about the plant from the user device; diagnose the disorder of the plant from the plurality of images using the diagnostic model and the inquiry information. The diagnostic device according to claim 1.

3. The processor is further configured to: calculate an average value of the probability of occurrence of the disorder for each of the plurality of images; correct the average value using a correction value based on the inquiry information to diagnose the disorder of the plant. The diagnostic device according to claim 2.

4. The processor is further configured to: obtain a plurality of images of different parts of the one plant captured from the user device; diagnose the disorder of the plant for each part using a different diagnostic model. The diagnostic device according to any one of claims 1 to 3.

5. The processor outputs information indicating a diagnostic result of diagnosing the disorder of the plant to the user device. The diagnostic device according to claim 1.

6. The storage device stores map data of a predetermined area where the plant is cultivated, and the processor is configured to: identify the cultivation location of the plant in the predetermined area by comparing the position information of the image with the map data; diagnose the disorder of the plant using parameters related to the cultivation location when diagnosing the disorder of the plant from the plurality of images. The diagnostic device according to claim 1 or 2.

7. A diagnostic system including a user device, and a diagnostic device capable of communicating with the user device and diagnosing a disorder occurring in a plant, wherein the diagnostic device comprises a processor and a storage device, and the processor is configured to: obtain a plurality of images of one plant captured from the user device; diagnose the disorder of the plant from the plurality of images using a diagnostic model obtained by machine learning the correlation between the affected part image of the plant with the disorder and the disorder stored in the storage device.

Images