Image diagnosis device, image diagnosis method, program, and storage medium

Abstract

To provide a diagnostic imaging device, a diagnostic imaging method, a program, and a storage medium, which are not easily affected by unnecessary matters other than a diagnosis object part existing in an image extracted from a crop image when acquiring a growth index and the resolution of the crop image.SOLUTION: A diagnostic imaging device includes an object pixel determination unit configured to divide a crop image into a plurality of divided images and determine object pixels including at least a part of the crop with each of the plurality of divided images as an input, and a feature quantity diagnostic unit configured to output a feature quantity relating to the growth state of the crop using the object pixels.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an image diagnostic device, an image diagnostic method, a program, and a storage medium for diagnosing the growth state of crops using images of the crops. [Background technology]

[0002] In recent years, methods have been proposed for acquiring growth indicators, such as the color of flowers and fruits, used to control quality in crop cultivation, using image recognition such as deep learning. Patent Document 1 discloses a method for diagnosing the growth stage from images of flowers, fruits, etc. extracted from images of crops using a convolutional neural network (CNN). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-83745 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method using CNN in Patent Document 1 cannot correctly diagnose the growth stage if the extracted image contains unnecessary objects other than the target part such as fruits or flowers (for example, the background, or leaves or stems covering the fruits or flowers).In addition, because CNN uses shape as a judgment criterion, it is necessary to increase the resolution of the crop image.

[0005] The present invention aims to provide an image diagnostic device, an image diagnostic device, a program, and a storage medium that are less susceptible to unwanted objects other than the diagnostic target area present in an image extracted from a crop image and to the influence of the resolution of the crop image when obtaining growth indices. [Means for solving the problem]

[0006] An image diagnostic device according to one aspect of the present invention includes a determination unit that divides an image of a crop into a plurality of segmented images and determines a target pixel including at least a part of the crop from each of the plurality of segmented images, and a diagnosis unit that outputs a feature amount related to the growth state of the crop using the target pixel, wherein the determination unit includes a first trained model obtained by machine learning using the image as training data and determines the target pixel by inputting the plurality of segmented images to the first trained model, and the diagnosis unit includes a second trained model obtained by machine learning using pixel data in a one-dimensional array as training data and outputs the feature amount by inputting the target pixel to the second trained model. The diagnosis unit executes an extraction process for extracting a plurality of pixels in the vicinity of the target pixel, a rearrangement process for rearranging the plurality of pixels into a one-dimensional array according to their respective brightness values, and an acquisition process for acquiring a feature amount relating to the growth state using a signal output by the rearrangement process. It is characterized by: [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an image diagnostic device, an image diagnostic method, a program, and a storage medium that, when obtaining growth indices, are less susceptible to unwanted objects other than the diagnostic target area present in the image extracted from the crop image, and are less affected by the resolution of the crop image. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of an image diagnostic system including an image diagnostic apparatus according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating a hardware configuration of an imaging diagnostic apparatus. [Figure 3] FIG. 2 is a diagram illustrating the software configuration of the diagnostic imaging apparatus. [Figure 4] 1 is a flowchart showing an imaging diagnostic method. [Figure 5] 1 is a conceptual diagram of the growth index acquisition unit. [Figure 6] 10 is a flowchart illustrating an example of a method for determining a target pixel. [Figure 7] FIG. 1 is a conceptual diagram of a learning model for target pixel detection. [Figure 8] 10 is a flowchart illustrating an example of a feature acquisition method. [Figure 9]FIG. 2 is a conceptual diagram of a neighboring pixel extractor, a rearranger, and a feature amount acquirer. [Figure 10] FIG. 1 is a conceptual diagram of a learning model for feature acquisition. [Figure 11] FIG. 10 is a diagram illustrating an example of an image of paddy rice. [Figure 12] FIG. 1 is a diagram showing the diagnosis results in Example 1. [Figure 13] FIG. 10 is a diagram illustrating visualization of the diagnosis results of the target pixel determination unit. [Figure 14] FIG. 10 is a diagram showing an example of an image of rice grains after threshing. [Figure 15] FIG. 10 is a diagram showing the diagnosis results in Example 2. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same components, and redundant explanations will be omitted.

[0010] 1 is a configuration diagram of an image diagnostic system 100 including an image diagnostic apparatus according to an embodiment of the present invention. The image diagnostic system 100 includes an image acquisition device 110, the Internet 120, a data collection server 130, and an image diagnostic apparatus 300.

[0011] The image acquisition device 110 acquires images of crops. In this embodiment, crops refer to, for example, plants such as rice, wheat (barley and wheat), tomatoes, and grapes, fungi such as mushrooms, crystals, etc. The image acquisition device 110 may be any device capable of acquiring images of crops, and an imaging device or a smartphone may be used as the image acquisition device 110, for example. The crop images acquired by the image acquisition device 110 are stored in the data collection server 130 via the Internet 120. The image diagnosis device 300 reads the crop images stored in the data collection server 130 via the Internet 120 and diagnoses the growth state of the crops using the read crop images.

[0012] In this embodiment, the diagnostic imaging device 300 is configured as a separate unit from the image acquisition device 110, but may be provided within the image acquisition device 110.

[0013] 2 is a diagram showing the hardware configuration of the image diagnostic apparatus 300. The image diagnostic apparatus 300 includes a system bus 301 for transferring data between the components, a CPU 302, a ROM 303, a RAM 304, a HDD 305, a NIC 306, an input unit 307, a display unit 308, and a GPU 309.

[0014] FIG. 3A is a software configuration diagram of the image diagnostic device 300. The image diagnostic device 300 includes an image input unit 310, a diagnosis unit 320, and a growth index display unit 330. The image input unit 310 inputs a crop image. The image is input by loading an image previously stored in a storage device such as a ROM 303 or HDD 305 onto the RAM 304, or by loading an image stored on a network via the NIC 306 onto the RAM 304. The diagnosis unit 320 functions as a target pixel determination unit 321, a feature amount diagnosis unit 322, and a growth index acquisition unit 323 when the CPU 302 executes a program stored in the storage device. The diagnosis unit 320 acquires a growth index using the crop image loaded onto the RAM 304. The growth index display unit 330 can display the growth index on the display unit 308.

[0015] The image diagnostic method of this embodiment using the image diagnostic apparatus 300 will be described below with reference to Fig. 4. Fig. 4 is a flowchart showing the image diagnostic method.

[0016] In step S10, the image input unit 310 inputs an image of a crop.

[0017] In step S20, the target pixel determination unit (determination unit) 321 divides the crop image input by the image input unit 310 into multiple segmented images and determines target pixels that include at least a portion of the crop (diagnosis target portion) from each of the multiple segmented images. In this step, determining target pixels where only the diagnosis target portion exists reduces the impact of unwanted objects other than the diagnosis target portion on growth indicators. Furthermore, robust diagnosis can be performed even if the diagnosis target portion, such as fruit or flowers, is hidden by unwanted objects such as leaves or stems. Furthermore, since there is no need to individually recognize each grain, robust diagnosis is possible even for crops such as rice and wheat, where the individual grains are small and difficult to individually recognize. The diagnosis target portion is not limited to fruit or flowers, but may be other parts such as leaves or stems.

[0018] In step S30, the feature amount diagnosis unit (diagnosis unit) 322 acquires feature amounts related to the growth state using the target pixel determined by the target pixel determination unit 321, and outputs the acquired feature amounts related to the growth state to the growth index acquisition unit 323. The feature amounts related to the growth state are, for example, information related to the color of the crop.

[0019] In step S40, the growth index acquisition unit 323 acquires a growth index using the feature amount related to the growth state acquired by the feature amount diagnosis unit 322. The processing of the growth index acquisition unit 323 is performed by the CPU 302. For example, when identifying the ripeness color of rice grains, the growth index acquisition unit 323 acquires the color ratio of rice grains, which is a growth index, using the binary data of yellow or green output by the feature amount diagnosis unit 322, as shown in FIG.

[0020] By implementing such a method, when a growth index is obtained using an image of a crop, it is possible to eliminate the influence of unnecessary objects other than the part to be diagnosed and the influence of the resolution of the image of the crop.

[0021] In this embodiment, when at least one processor executes a program stored in at least one memory, it functions as the target pixel determination unit 321 and the feature amount diagnosis unit 322. Specifically, at least one processor executes a process of determining a target pixel and a process of outputting a feature amount.

[0022] The spectrum of light irradiated on the diagnostic target area, which is directly irradiated by sunlight, is different from that of the diagnostic target area located in a shadow. Therefore, when acquiring a growth index using information about the color of the crop, as in this embodiment, determining a pixel included in the diagnostic target area in a shadow as the target pixel may result in an error in the growth index. Therefore, as shown in FIG. 3(b), the target pixel determination unit 321 preferably includes a removal processing unit 321a, a target pixel detection unit 321b, and a noise processing unit 321c to remove shadows and reduce such errors. FIG. 6 is a flowchart showing a method for determining a target pixel by removing the shadow portion of the diagnostic target area, which is an example of a method for determining a target pixel by the target pixel determination unit 321.

[0023] In step S21, the removal processing unit 321a removes shadow areas from the diagnostic target area. In this step, pixels that do not satisfy a predetermined condition may be removed as shadow areas. For example, a mask may be generated by removing pixels whose brightness values ​​are equal to or less than a threshold. In this case, for color correction, it is recommended that a reflector with a known reflectance in the detection wavelength range is reflected in the image when capturing the image. Shadow areas can be removed more accurately by inputting an image that has undergone color correction based on the value of the reflector reflected in the image. However, the method for removing shadow areas is not limited to the method of this embodiment. For example, other methods, such as segmentation of shadow areas using a convolutional neural network (CNN), may also be used.

[0024] In step S22, the target pixel detection unit 321b detects the target pixel. The target pixel detection unit 321b may detect the target pixel using, for example, a trained model generated by CNN. Information regarding the structure and various parameters of the trained model is stored in advance in the ROM 303 or the HDD 305. When detecting the target pixel in this step, the GPU 309 is used in addition to the CPU 302. Since the GPU 309 can perform efficient calculations by processing data in parallel, it is effective to use the GPU 309 when performing CNN calculation processing. As the trained model, for example, as shown in FIG. 7(a), a CNN may be used in which the input data is an image of a paddy rice crop (crop image) and the output data is a detected image of rice grains (diagnosis target area). Note that the target pixel detection unit 321b is not limited to machine learning, and may also detect the target pixel using pattern recognition techniques such as pattern matching or local features such as SIFT features.

[0025] An example of learning using CNN will be explained below with reference to Figure 7(b). The input data is an image of paddy rice, and the training data (training sample) is a labeled image in which the rice grains and other areas are labeled with different values. For example, as shown in Figure 7(b), an image in which the rice grain areas are labeled with 1 and the areas other than the rice grains are labeled with 0 can be used as a labeled image. Furthermore, the shaded areas can be removed by labeling the rice grains in the shaded areas of the training data at this point as 0.

[0026] In Figure 7(b), first, the error corresponding to the difference between the output data output from the CNN output layer in response to the input data input to the input layer and the training data is calculated using a loss function. Next, based on the obtained error, the connection weight coefficients between CNN nodes are updated using, for example, the backpropagation method so that the error is reduced. The backpropagation method is a method of adjusting the connection weight coefficients between CNN nodes so that the error is reduced.

[0027] In step S23, the noise processing unit 321c performs noise processing to reduce errors due to target pixels erroneously detected by the target pixel detection unit 321b and chromatic aberration of the input image caused by the camera's performance. The noise processing unit 321c may, for example, perform contraction processing on the image output from the target pixel detection unit 321b. Contraction processing refers to a process in which, if there is at least one non-target pixel surrounding a target pixel, the target pixel is replaced with a non-target pixel. However, the noise processing unit 321c is not limited to contraction processing and may also perform other noise processing such as threshold processing using the values ​​of the crop image.

[0028] When diagnosing growth conditions using image recognition with machine learning such as deep learning, differences between the capture conditions of the training image and the recognition image can sometimes prevent accurate feature diagnosis. For example, when diagnosing rice ripeness color, it is necessary to capture subtle color changes. However, rice ripeness color changes over time after harvest due to changes in the rice's moisture content, making accurate identification impossible. Therefore, as shown in Figure 3(c), the feature diagnosis unit 322 preferably includes an environmental condition correction unit 322a, a neighboring pixel extraction unit 322b, a rearranger 322c, and a feature acquisition unit 322d to correct differences between the training image and the recognition image. Figure 8 is a flowchart showing an example of a feature acquisition method. Figure 9 is a conceptual diagram of the neighboring pixel extraction unit 322b, the rearranger 322c, and the feature acquisition unit 322d.

[0029] In step S31, the environmental condition corrector 322a performs processing to align the input with the environmental conditions at the time of learning.

[0030] In step S32, the neighboring pixel extractor 322b extracts a plurality of neighboring pixels in the vicinity of the target pixel determined by the target pixel determiner 321.

[0031] In step S33, the rearranger 322c rearranges the multiple neighboring pixels extracted by the neighboring pixel extractor 322b into a one-dimensional array according to their luminance values. This causes a trend to appear in the disordered pixel array, allowing efficient learning by the neural network (NN) used in the next step S34.

[0032] In step S34, the feature acquisition unit 322d acquires features using the signal output from the rearranger 322c. The feature acquisition unit 322d may acquire features using, for example, a trained model generated by a neural network (NN) that receives a one-dimensional array as input. Information regarding the structure and various parameters of the trained model is stored in advance in the ROM 303 or the HDD 305. When acquiring features in this step, the GPU 309 is used in addition to the CPU 302. For example, when diagnosing the ripeness color of rice grains, as shown in FIG. 10(a), a neural network may be used that uses pixel values ​​extracted from target pixels containing rice grains in an image of paddy rice as input data and outputs binary labels of yellow or green representing the ripeness color. In this case, the input data may be sorted by the tendency of brightness values ​​in step S33 and converted into a one-dimensional array. The trained model generated by the neural network may be generated in a manner similar to the trained model used by the target pixel detection unit 321b in step S22 of FIG. 6, as shown in FIG. 10(b).

[0033] In this way, the feature amount diagnosis unit 322 can accurately acquire feature amounts relating to the growth state by correcting the environmental conditions and rearranging the extracted pixels according to the situation.

[0034] Note that the processes from step S32 to step S34 may be executed multiple times to acquire multiple feature amounts, provided that the same pixel is not extracted in step S32. Since the GPU 309 is used to acquire feature amounts, parallel processing by the GPU 309 can be used to efficiently perform calculations even when acquiring multiple feature amounts. Furthermore, the order of the process of step S31 may be reversed as long as it is performed after the process of step S20 in FIG. 4 and before the process of step S34.

[0035] In this embodiment, when at least one processor executes a program stored in at least one memory, it functions as the neighboring pixel extraction unit 322b, the rearranger 322c, and the feature acquisition unit 322d. In other words, at least one processor performs an extraction process that extracts multiple pixels neighboring a target pixel, a rearrangement process that rearranges the multiple pixels into a one-dimensional array according to their brightness values, and an acquisition process that acquires features related to the growth state using signals output by the rearrangement process.

[0036] Furthermore, the feature amount diagnosis unit 322 can correct differences between the learning images and the recognition images due to subtle color differences between the varieties by performing the processing flow in Fig. 8. This eliminates the need to prepare a trained model for each variety to be used by the feature amount acquisition unit 322d. [Example]

[0037] In this embodiment, a method for diagnosing the rice ripeness ratio (the ratio of green rice grains to the total rice grains, green rice grain rate) using an image of paddy rice in a field by an image diagnostic device 300 will be described.

[0038] In this embodiment, the image input unit 310 inputs an image of paddy rice as the crop image. The diagnosis unit 320 uses the paddy rice image loaded onto the RAM 304 to obtain the ripeness color ratio as information on the growth state of the crop. The growth index display unit 330 displays the ripeness color ratio on the display unit 308 as necessary.

[0039] A method for diagnosing the ripeness color ratio of paddy rice using an image of the paddy rice by the image diagnostic device 300 will be described below along the flow of FIG.

[0040] In step S10, the image input unit 310 inputs an image of paddy rice.

[0041] In step S20, the target pixel determination unit 321 divides the image of paddy rice into a plurality of divided images, and determines rice grain detection pixels (target pixels) that include rice grains, which are the diagnosis target portions, using each of the plurality of divided images as an input.

[0042] In this embodiment, the target pixel determination unit 321 determines the rice grain detection pixels to be input to the feature amount diagnosis unit 322 in accordance with the flow of FIG.

[0043] In step S21, the removal processing unit 321a removes the shadowed portions of the rice grains by processing (threshold processing, segmentation) an input image in which the brightness values ​​of the shadowed portions are lowered relative to the sunny portions using the value of the reflector reflected in the image of Fig. 11.

[0044] In step S22, the target pixel detection unit 321b detects rice grain detection pixels using a trained model generated by CNN. Although the image in FIG. 11 is displayed in monochrome, in this embodiment an RGB image is used as the image of paddy rice. When detecting rice grain detection pixels, a monochrome image of the chloroplast absorption wavelength band, such as a wavelength of 700 nm, may be used, but since using color information improves detection accuracy, it is preferable to use an image containing wavelength information of two or more bands.

[0045] In step S23, the noise processing unit 321c performs noise processing to reduce errors due to target pixels erroneously detected by the target pixel detection unit 321b, chromatic aberration of the input image caused by the performance of the camera, etc. In this embodiment, the noise processing unit 321c performs contraction processing for three pixels, taking into account the performance of the camera.

[0046] In step S30, the feature amount diagnosis unit 322 acquires feature amounts related to the growth state using the target pixel in accordance with the flow of Fig. 8. In this embodiment, information on whether the rice grain is green or yellow is acquired as the feature amount related to the growth state.

[0047] In step S40, the growth index acquisition unit 323 acquires the grain ripeness color ratio, which is a growth index, using the feature amount acquired in step S30.

[0048] Figure 12 shows the results of diagnosing the ripeness color ratio (green rice rate) using images of paddy rice in a field. The horizontal axis of Figure 12 is the date, and the vertical axis is the green rice rate. As shown in Figure 12, the diagnostic values ​​are close to the correct values, confirming the effectiveness of the image diagnosis method of the present invention.

[0049] The acquired rice ripeness color ratio may be stored in ROM 303 or HDD 305, or may be stored in an external storage device (not shown) connected to the network via NIC 306. As shown in Fig. 13, growth index display unit 330 may display an image showing the rice grain positions used in the diagnosis on display unit 308 to visualize the diagnosis. Furthermore, growth index display unit 330 may use colors according to the rice grain positions and display an image on display unit 308 that shows which parts have been determined to be green rice grains or yellow rice grains.

[0050] 8 as many times as possible without extracting the same pixel, the feature diagnostic unit 322 outputs the feature multiple times, and the growth index acquisition unit 323 performs statistical processing and outputs the growth index. This averages the detection errors of the feature and reduces the standard error of the final output.

[0051] In this embodiment, an RGB image is used as the input image, and both the target pixel determination unit 321 and the feature amount diagnosis unit 322 use RGB values ​​to determine the target pixel and acquire the feature amount, but the input image in this embodiment is not limited to an RGB image. A multiband image with four or more bands containing a large amount of information may also be used as the input image. This allows for more accurate diagnosis. Furthermore, the target pixel determination unit 321 may use an RGB image as input data, and the feature amount diagnosis unit 322 may input values ​​of different wavelengths, such as RGB and values ​​from the fourth band or higher. This makes it possible to easily perform feature amount diagnosis even in wavelength ranges with little training data, such as infrared. [Example]

[0052] In this embodiment, a method for diagnosing the ripeness color of rice grains using images of threshed rice grains taken by an image diagnostic device 300 will be described. The diagnostic method of this embodiment is basically the same as the diagnostic method of embodiment 1. In this embodiment, processes that differ from embodiment 1 will be described, and detailed descriptions of processes that are the same as those of embodiment 1 will be omitted.

[0053] In this embodiment, the image input unit 310 inputs an image of threshed rice grains shown in FIG. 14 as an image of the crop. Unlike the image of paddy rice in FIG. 11, the image of threshed rice grains in FIG. 14 has sufficient resolution to recognize individual grains. Although the image in FIG. 14 is displayed in monochrome, in this embodiment an RGB image is used as the image of rice grains. When detecting rice grain detection pixels, a monochrome image of the chloroplast absorption wavelength band, such as a wavelength of 700 nm, may be used; however, using color information improves detection accuracy, so it is preferable to use an image containing wavelength information in two or more bands.

[0054] In this embodiment, in step S22 of FIG. 6, the target pixel detection unit 321b performs individual recognition of the diagnosis target area. Individual recognition can be achieved by using an instance segmentation method such as MASK R-CNN. MASK R-CNN is a method for individually recognizing objects and classifying them, so in principle, it can simultaneously classify yellow and green rice grains. However, in a method using a CNN such as MASK R-CNN, the judgment criteria are highly dependent on the shape of the object. In other words, when classifying objects with similar shapes, such as yellow and green rice grains, the judgment criteria can easily be affected by the influence of unnecessary objects such as the background or subtle changes in the arrangement of the rice grains. As a result, it is difficult for MASK R-CNN to perform classification into yellow and green rice grains. Therefore, in this embodiment, the target pixel detection unit 321b performs individual recognition of the rice grains, and the diagnosis unit 320 performs classification. In this case, the pixel extraction performed by the neighboring pixel extraction unit 322b can be performed by randomly extracting pixels from the rice grain area that has been recognized as a single rice grain. In this embodiment, the neighboring pixel extraction unit 322b extracts 300 pixels. In this embodiment, the environmental conditions of the learning image and the rice grain image are almost the same, so no processing is performed by the environmental condition correction unit 322a.

[0055] Figures 15(a) to 15(c) show the results of diagnosing the ripeness color ratio (green rice rate) of rice grains using images of rice grains taken over four days from August to September in fields 1 to 3, respectively. The horizontal axis of Figure 15 represents the date, and the vertical axis represents the green rice rate. As shown in Figure 15, the diagnostic values ​​are close to the correct values, confirming that the image diagnosis method of the present invention is useful even when the resolution is such that objects such as grains can be sufficiently recognized.

[0056] In this embodiment, a smartphone is used as the image acquisition device 110, allowing accurate and hassle-free diagnosis by taking a picture with the smartphone immediately after threshing in the field. Even if there are many fields located far away, measurements can be made on the same day by using multiple smartphones.

[0057] Furthermore, in this embodiment, an RGB image is used as the input image, and both the target pixel determination unit 321 and the feature amount diagnosis unit 322 use RGB values ​​to determine the target pixel and acquire the feature amount, but the input image in this embodiment is not limited to an RGB image. A multiband image with a large amount of information, having four or more bands, may also be used as the input image. This allows for more accurate diagnosis. Furthermore, the target pixel determination unit 321 may use an RGB image as input data, and the feature amount diagnosis unit 322 may input values ​​of different wavelengths, such as RGB and values ​​from the fourth band or higher. This makes it possible to easily perform feature amount diagnosis even in wavelength ranges with little training data, such as infrared. [Example]

[0058] In this embodiment, a method for diagnosing the ripeness of tomatoes using tomato images obtained by an image diagnostic device 300 will be described. If the resolution of the tomato image is sufficient, individual fruit recognition may be performed as in Example 2, or the shape of the calyx may be recognized to diagnose the shape of the calyx. In addition, growth indices for the ripeness of the fruit and the shape of the calyx may be output. By outputting two growth indices, the visibility of the judgment criteria can be improved.

[0059] By using a smartphone as the image acquisition device 110, the present invention can also be applied to crops grown in home gardens by users other than farmers who lack specialized knowledge. (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0060] Although the preferred embodiments and examples of the present invention have been described above, the present invention is not limited to these embodiments and examples, and various combinations, modifications, and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0061] 300 Diagnostic Imaging Devices 321 Target pixel determination unit (determination unit) 322 Feature Diagnosis Unit (Diagnosis Unit)

Claims

1. a determination unit that divides an image of the crop into a plurality of divided images and determines a target pixel including at least a portion of the crop from each of the plurality of divided images; a diagnosis unit that outputs a feature amount related to the growth state of the crop using the target pixel, the determination unit includes a first trained model obtained by machine learning using images as training data, and determines the target pixel by inputting the plurality of segmented images into the first trained model; the diagnosis unit includes a second trained model obtained by machine learning using pixel data of a one-dimensional array as training data, and outputs the feature amount by inputting the target pixel into the second trained model; The diagnostic unit performs an extraction process to extract multiple pixels in the vicinity of the target pixel, a rearrangement process to rearrange the multiple pixels into a one-dimensional array according to their respective brightness values, and an acquisition process to acquire features related to the growth state using signals output by the rearrangement process.

2. The image diagnostic device according to claim 1 , wherein the diagnostic unit matches environmental conditions when the image of the crop is acquired with environmental conditions when the image is learned.

3. 3. The image diagnostic apparatus according to claim 1, wherein the extraction process, the rearrangement process, and the acquisition process are performed a plurality of times while changing the pixels extracted by the extraction process.

4. 4. The image diagnostic device according to claim 1, further comprising an acquisition unit that acquires a growth index of the crop using the feature amount related to the growth state.

5. The image diagnostic device according to claim 4 , wherein the acquisition unit acquires the growth index using a plurality of feature quantities related to the growth state.

6. 6. The image diagnostic apparatus according to claim 1, wherein the determining unit performs a removal process to remove pixels that do not satisfy a predetermined condition.

7. 7. The image diagnostic apparatus according to claim 6, wherein the removal process removes pixels whose luminance values ​​are equal to or less than a threshold value.

8. 8. The image diagnostic apparatus according to claim 1, wherein the determination unit performs noise processing for removing noise from the target pixel.

9. 9. The diagnostic imaging apparatus according to claim 1, wherein the crop is a plant.

10. a first step of dividing an image of a crop into a plurality of segmented images and determining a target pixel including at least a portion of the crop from each of the plurality of segmented images; a second step of outputting a feature amount relating to the growth state of the crop using the target pixel; In the first step, the target pixel is determined by inputting the plurality of divided images into a first trained model obtained by machine learning using images as training data; In the second step, the target pixel is input to a second trained model obtained by machine learning using pixel data of a one-dimensional array as training data, thereby outputting the feature amount; The second step of the image diagnostic method includes performing an extraction process to extract a plurality of pixels in the vicinity of the target pixel, a rearrangement process to rearrange the plurality of pixels into a one-dimensional array according to their respective brightness values, and an acquisition process to acquire features related to the growth state using signals output by the rearrangement process.

11. A program causing a computer to execute the image diagnosis method according to claim 10.

12. A storage medium storing the program according to claim 11.

Images