Information processing apparatus

Abstract

Provided is an information processing apparatus that calculates an index value related to a plant growth state by a neural network, i.e., an information processing apparatus that completes training of a neural network with a small number of images. The information processing apparatus includes: first to Nth image analysis sections that calculate a state index value, which is an index value of a plant growth state, in a cultivation area by analyzing a cultivation area image that has captured the cultivation area based on a neural network, the first to Nth image analysis sections each including a neural network that has been trained by a plurality of cultivation area images classified into first to Nth tree vigor index value divisions by a prescribed tree vigor index value; and a selection section that receives input of a cultivation area image for which the state index value is to be calculated, and causes an image analysis section, among the first to Nth image analysis sections, that has been trained by a plurality of cultivation area images classified into the same tree vigor index value division as the cultivation area image to analyze the input cultivation area image.

Description

Technical Field

[0001] The present invention relates to an information processing device for calculating an index value related to a plant growth state through AI image analysis. Background Art

[0002] Index values ​​related to plant growth status are calculated by AI (Artificial Intelligence) image analysis (see, for example, Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2018-29568 Summary of the Invention

[0006] To calculate indicators related to plant growth using AI image analysis, a neural network, such as a CNN (Convolutional Neural Network), must be trained by inputting multiple images of a specified size. However, since the appearance of plants changes significantly as they grow, training the neural network required to calculate these indicators requires a large number of images.

[0007] The present invention has been made in view of the above-mentioned situation and provides an information processing device for obtaining an index value related to plant growth status through a neural network. The information processing device completes the training of the neural network using a relatively small number of images.

[0008] An information processing device according to one aspect of the present invention comprises: first to Nth image analysis units, which are first to Nth (≥2) image analysis units for calculating status index values ​​as index values ​​of plant growth status in the cultivation area by performing neural network-based analysis on a cultivation area image captured of a plant cultivation area, and each of which includes a neural network trained by a plurality of cultivation area images classified into first to Nth tree vigor index values ​​according to a prescribed tree vigor index value; and a selection unit for receiving an input of a cultivation area image for which the status index value is to be calculated, and causing the image analysis units, among the first to Nth image analysis units, which have been trained by a plurality of cultivation area images classified into the same tree vigor index value as the cultivation area image, to analyze the input cultivation area image.

[0009] Specifically, this information processing device includes a neural network (image analysis unit) for calculating a status index value, which is an indicator of the plant growth status in a cultivation area, divided by tree vigor index value. Training a neural network to accurately calculate the status index value independently of the plant's vigor requires a large number of images. However, by configuring the neural networks according to tree vigor index value, each neural network can be trained without using images that are inappropriate for training the neural network. Therefore, in the information processing device having the above configuration, the neural network of each image analysis unit can be trained using a relatively small number of images.

[0010] The information processing device may be configured as follows: "The neural network included in the i-th image analysis unit (i is an integer from 1 to N) is to convert the number of vertical pixels H i , horizontal pixel number W i The neural network #i is input with an image of the cultivation area to be analyzed, and the i-th image analysis unit (i is an integer from 1 to N) extracts a plurality of images with an aspect ratio of H from the cultivation area image to be analyzed. i :W i and input each extracted image to the neural network #i after adjusting the number of pixels. In addition, since the height of plants generally increases continuously, the information processing device may also adopt the following configuration: "The k+1th tree potential index value division (k is each integer value from 1 to N-1) is a division with better tree potential than the kth tree potential index value division, and the input image size of the neural networks #i to #N is determined so that for each integer value i from 1 to N-1, H i+1 / W i+1 >H i / W i Established".

[0011] The tree vigor index value may be a representative value of plant height in the cultivation area (e.g., a mode or average value of plant height). The representative value may be provided separately from the cultivation area image or may be obtained by analyzing the cultivation area image.

[0012] An information processing device according to another aspect of the present invention (hereinafter also referred to as a second information processing device) calculates a state index value as an indicator value of a plant growth state, the information processing device comprising: a neural network for calculating the state index value, the neural network being trained by extracting and inputting images of sizes corresponding to the tree vigor index value divisions to which the tree vigor of the plant in the cultivation area image belongs from each of a plurality of cultivation area images taken of the plant cultivation area; and a calculation unit for calculating the state index value by extracting, from the cultivation area image for which the state index value is to be calculated, images of sizes corresponding to the tree vigor index value divisions to which the tree vigor of the plant in the cultivation area image belongs, and inputting the images into the neural network.

[0013] Specifically, by extracting an image of a size corresponding to the tree vigor index value category of the plant vigor within each of the multiple cultivation area images captured, and inputting it into a neural network such as an FCN, it is possible to train the neural network without using images that are unsuitable for training. Consequently, in this second information processing device, the neural network of each image analysis unit can be trained using a relatively small number of images.

[0014] The second information processing device may adopt the following structure: "The k+1th tree potential index value partition (k is an integer value from 1 to N-1) is a partition with a better tree potential than the kth tree potential index value partition, and the number of vertical pixels and the number of horizontal pixels of the size corresponding to the i-th partition are expressed as H respectively. i 、W i When the sizes corresponding to the first to Nth partitions are determined, H i+1 / W i+1 >H i / W i Established".

[0015] According to the present invention, an information processing device is provided that obtains an index value related to a plant growth state through a neural network, and completes the training of the neural network using a relatively small number of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is an explanatory diagram of the configuration and usage of the information processing device according to the first embodiment of the present invention.

[0017] Figure 2 It is an explanatory diagram of the configuration of the image analysis unit included in the information processing device according to the first embodiment.

[0018] Figure 3This is a flowchart of analysis target selection processing performed by the analysis control unit of the information processing device according to the first embodiment.

[0019] Figure 4 It is an explanatory diagram of the configuration and usage of the information processing device according to the second embodiment of the present invention.

[0020] Figure 5 This is a flowchart of tree potential index value classification and determination processing performed by the analysis control unit of the information processing device according to the second embodiment. DETAILED DESCRIPTION

[0021] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0022] First Implementation Method

[0023] Figure 1 The configuration and usage of the information processing device 10 according to the first embodiment of the present invention are described.

[0024] The information processing device 10 of this embodiment is a device that calculates an index value related to the plant growth status in each user's plant cultivation area based on the results of photographing the plant cultivation area transmitted from each user's user terminal 30 via the Internet 50. As shown in the figure, the information processing device 10 includes multiple image analysis units 12, an analysis control unit 13, and a cultivation area management database 14. Furthermore, the information processing device 10 is programmed so that a high-performance computer, such as one or more high-performance GPUs (Graphics Processing Units), functions as a device including the multiple image analysis units 12, analysis control unit 13, and cultivation area management database 14. Therefore, the detailed description of the hardware configuration of the information processing device 10 is omitted. Hereinafter, the multiple image analysis units 12 included in the information processing device 10 will also be referred to as the first to Nth image analysis units 12 (N being the total number of image analysis units 12 included in the information processing device 10).

[0025] The cultivation area management database 14 is a database for managing information related to the cultivation area of ​​each user. Authentication information for each user (eg, user ID and password) is also set in the cultivation area management database 14.

[0026] The i-th (i=1 to N) image analysis unit 12 is a unit that calculates an index value of the plant growth status (hereinafter referred to as a status index value) in the cultivation area by analyzing the image capture result of the plant cultivation area (hereinafter referred to as the cultivation area image) based on the neural network #i.

[0027] More specifically, if Figure 2As schematically shown, the i-th (i=1 to N) image analysis unit 12 includes a pre-processing unit #i, a neural network ("NN") #i, a post-processing unit #i, and a learning processing unit #i.

[0028] Neural network #i is the input vertical pixel number H i , horizontal pixel number W i The input image size of neural network #1 to #N is determined as follows: for each integer value j from 1 to N-1, H j+1 / W j+1 >H j / W j . Furthermore, when neural networks #1 to #N are implemented using one or more GPUs, it is preferable to pre-set the input image size (total number of pixels) of neural networks #1 to #N to a size that fully utilizes the GPU's performance. Examples of such input image sizes include H1 = 256, W1 = 256, H2 = 128, W2 = 512, H3 = 64, and W3 = 1028 when N = 3.

[0029] The pre-processing unit #i (i=1 to N) is a unit that extracts a plurality of images with an aspect ratio H from the cultivation area image provided by the analysis control unit 13 (described in detail later). i :W i The image is taken as the processing object, and each extracted image is converted into a vertical pixel number H i , horizontal pixel number W i The post-processing unit #i is a unit that stores the average value of the index value group output from the neural network #i for each image input from the pre-processing unit #i as a state index value for the processing target area in the cultivation area management database 14.

[0030] The learning processing unit #i (i=1 to N) is a unit that trains the neural network #i by using a plurality of cultivation area images classified into i-th tree vigor index value categories and a state index value to be obtained from each cultivation area image as training data.

[0031] Here, the tree vigor index value refers to a value indicating the degree (strength) of the tree vigor of plants in the cultivation area. As the tree vigor index value, for example, the representative height of the plants in the cultivation area (the average value or the most frequent value of the plant height) is used. The first to Nth tree vigor index value divisions refer to tree vigor index value divisions that are predetermined to increase the tree vigor index value (the tree vigor becomes stronger) as the division number (the value i in the i-th tree vigor index value division) increases. In addition, "training the neural network #i" means "determining the weight value and bias value of each node of the neural network #i so that the prescribed evaluation function is minimized." The learning processing unit #i is similar to the pre-processing unit #i and is configured to extract a plurality of tree vigor index values ​​with an aspect ratio of H from the cultivation area image. i :W i The extracted images are converted into the vertical pixel number H i , horizontal pixel number W i The image is then input to the neural network #i.

[0032] The analysis control unit 13 is a unit that receives the cultivation area image sent from the user terminal 30 as a cultivation area image (hereinafter referred to as processing target image) regarding the cultivation area for which the state index value is to be calculated (hereinafter referred to as processing target area) through processing accompanying user authentication.

[0033] When receiving the input of the processing target image, the analysis control unit 13 executes Figure 3 The analysis destination selection process of the steps shown.

[0034] That is, the analysis control unit 13 that receives the input of the processing object image first calculates the tree vigor index value (average value and mode value of plant height) by analyzing the processing object image (step S101). Next, the analysis control unit 13 determines which of the first to Nth tree vigor index value divisions the calculated tree vigor index value is classified into (step S102). Furthermore, the analysis control unit 13 supplies the processing object image to the image analysis unit 12 corresponding to the determination result (step S103). That is, when the tree vigor index value is classified into the kth tree vigor index value division, the analysis control unit 13 supplies the processing object image to the kth image analysis unit 12, thereby causing the kth image analysis unit 12 to calculate the status index value of the processing object image. Furthermore, the analysis control unit 13 that has completed the processing of step S103 ends the analysis destination selection processing.

[0035] As described above, the information processing apparatus 10 of the present embodiment is provided with a neural network (the image analysis section 12) for calculating a state index value (an index value of the plant growth state in the cultivation area) with respect to the cultivation area, divided by the vigor index value. Further, the information processing apparatus 10 has a configuration in which the neural network in which the stronger the cultivation area of the calculation target region of the state index value is, the longer the input image size is in the vertical direction is used. In order to train one neural network in a manner that the state index value can be accurately calculated regardless of the vigor of the plant, a large number of images are required, but if the neural network in which the input image size is changed in the above-described manner according to the division of the vigor index value is provided, each neural network can be trained without using images that are not suitable for the training of the neural network. Therefore, in the information processing apparatus 10 of the present embodiment, the training of the neural network of each image analysis section 12 is completed with a relatively small number of images.

[0036] <Second Embodiment>

[0037] Figure 4 The configuration and the usage of the information processing apparatus 20 of the second embodiment of the present application are shown. In addition, the information processing apparatus 20 of the present embodiment, like the information processing apparatus 10, is an apparatus that calculates an index value related to the plant growth state in the plant cultivation area of each user based on the photographed result of the plant cultivation area transmitted from the user terminal 30 of each user via the Internet 50.

[0038] As shown, the information processing apparatus 20 is provided with an image analysis section 22, an analysis control section 23, and a cultivation area management database 24.

[0039] The cultivation area management database 24 is a database for managing information related to the cultivation area of each user. In the cultivation area management database 24, authentication information (for example, a user ID and a password) of each user is also set.

[0040] The image analysis section 22 is a unit that calculates an index value of the plant growth state (hereinafter referred to as a state index value) in the plant cultivation area by analyzing the photographed result of the plant cultivation area (hereinafter referred to as a cultivation area image). As shown, the image analysis section 22 is provided with a preprocessing section 22a, an FCN (Fully Convolutional Network) 22b, a post-processing section 22c, and a learning processing section 22d.

[0041] The FCN 22b is a neural network that can input images of various sizes, which is entirely composed of convolution layers. In addition, in the information processing apparatus 20 of the present embodiment, the FCN 22b is implemented by a plurality of GPUs.

[0042] The pre-processing unit 22a is a unit that extracts a plurality of images with an aspect ratio H from the cultivation area image inputted together with the division number i (1≤i≤N) from the analysis control unit 23 or the learning processing unit 22d. i :W i The extracted images are converted into the vertical pixel number H i , horizontal pixel number W i The image is then input to FCN22b.

[0043] The segment number i, input to the pre-processing unit 22a along with a particular cultivation area image, indicates to which of the first to Nth tree vigor index value segments the tree vigor index value for that cultivation area image belongs (the value of i in the i-th tree vigor index value segment). Furthermore, the tree vigor index value indicates the degree (strength) of tree vigor of plants in the cultivation area. For example, the representative height of plants in the cultivation area (the average or mode of plant height) is used as the tree vigor index value.

[0044] The first to Nth tree vigor index value divisions are predetermined so that the tree vigor index value increases (the tree vigor becomes stronger) as the division number increases. In addition, if the number of vertical pixels and the number of horizontal pixels of the image extracted from the cultivated area image of the i-th (1≤i≤N) tree vigor index value division are expressed as H, respectively i 、W i , the size of the image extracted from the cultivated area image divided by the first to Nth tree potential index values ​​is determined as follows: for each integer value j from 1 to N-1, H j+1 / W j+1 >H j / W j In addition, H i 、W i (i=1~N) is also determined to be independent of the value of i and “H i ×W i " is a fixed value.

[0045] The post-processing unit 22c is a unit that stores the average value of the index value group output from the FCN 22b as a status index value for a certain cultivation area in the cultivation area management database 24 while the pre-processing unit 22a processes the cultivation area image for a certain cultivation area input from the analysis control unit 23.

[0046] The learning processing unit 22d is a unit that trains the FCN 22b using a plurality of cultivated area images and the state index values ​​to be determined from each cultivated area image as training data. Here, "training the FCN 22b" means determining the weight value and bias value of each node of the FCN 22b so as to minimize a predetermined evaluation function. In addition, the learning processing unit 22d is configured to train the FCN 22b by combining the tree potential index value classification and discrimination processing ( Figure 5 ) In the same steps, the division number of the tree vigor index value division to which each cultivation area image belongs is determined, and the determination result is supplied to the pre-processing unit 22a together with each cultivation area image, thereby training the FCN 22b.

[0047] The analysis control unit 23 is a unit that receives the cultivation area image sent from the user terminal 30 as a cultivation area image (hereinafter referred to as processing target image) regarding the cultivation area for which the state index value is to be calculated (hereinafter referred to as processing target area) through processing accompanied by user authentication.

[0048] When receiving the input of the processing target image, the analysis control unit 23 executes Figure 5 The steps shown are tree potential index value classification and determination processing.

[0049] Specifically, upon receiving the input of the processing target image, the analysis control unit 23 first analyzes the processing target image to determine the tree vigor index value (average and mode of plant height) (step S201). Next, the analysis control unit 23 determines into which of the first to Nth tree vigor index value categories the determined tree vigor index value falls (step S202). Furthermore, the analysis control unit 23 supplies the determination result (tree vigor index value category number) and the processing target image to the image analysis unit 22 (preprocessing unit 22a) (step S203), causing the image analysis unit 22 to calculate the status index value for the processing target image. Upon completing step S203, the analysis control unit 23 terminates the tree vigor index category determination process.

[0050] Information processing device 20 of this embodiment has the configuration described above. Furthermore, by extracting an image of a size corresponding to the tree vigor index value category to which the tree vigor of the plants within the cultivation area image belongs from each of a plurality of cultivation area images captured of the plant cultivation area and inputting it into FCN 22b, FCN 22b can be trained without using images unsuitable for training. Therefore, in information processing device 20 of this embodiment, the training of the neural network (FCN 22b) of each image analysis unit can be completed using a relatively small number of images.

[0051] Modification

[0052] The information processing devices 10 and 20 described above can be modified in various ways. For example, if more images are required for training each neural network than for the information processing device 10, the information processing device 10 of the first embodiment can be modified so that the input images of the neural networks of the image analysis units 12 have the same size.

[0053] Information processing device 20 can also be modified to calculate the state index using a neural network including one (one type of) convolutional layer and multiple dense layers. Furthermore, while information processing devices 10 and 20 calculate the tree vigor index by analyzing cultivated area images, information processing device 10 can also be modified to receive cultivated area images and tree vigor index values ​​from user terminal 30.

[0054] Postscript

[0055] An information processing device (10), characterized by comprising:

[0056] The first to Nth image analysis units (12) are configured to calculate state index values, which are index values ​​of plant growth states, in the cultivation area by performing neural network-based analysis on a cultivation area image obtained by photographing the plant cultivation area, and the first to Nth (≥2) image analysis units (12) are respectively configured to include a neural network trained on a plurality of cultivation area images classified into first to Nth tree vigor index value groups according to predetermined tree vigor index values; and

[0057] The selection unit (13) receives an input of a cultivation area image for which the state index value is to be calculated, and causes the image analysis unit (12) among the first to Nth image analysis units, which is trained by a plurality of cultivation area images classified as having the same tree vigor index value as that of the cultivation area image, to analyze the input cultivation area image.

[0058] An information processing device (20) is characterized in that it calculates a state index value as an index value of a plant growth state, and the information processing device (20) comprises:

[0059] a neural network (22b) for calculating the state index value, the neural network (22b) being trained by extracting, from each of a plurality of cultivation area images obtained by capturing the plant cultivation area, images of sizes corresponding to the tree vigor index value divisions to which the tree vigor of the plant in the cultivation area image belongs, from first to Nth (≥2) tree vigor index value divisions and inputting the images; and

[0060] The calculation unit (22a, 23) calculates the state index value by extracting an image of a size corresponding to the tree vigor index value division to which the tree vigor of the plant in the cultivation area image belongs from the cultivation area image for which the state index value is to be calculated and inputting the image into the neural network (22b).

[0061] Explanation of symbols

[0062] 10, 20 Information processing device

[0063] 12, 22 Image Analysis Department

[0064] 13, 23 Analysis and Control Department

[0065] 14, 24 Cultivation area management database

[0066] 30 user terminals

[0067] 50 Internet

Claims

1. An information processing device, characterized in that have: First to N-th image analysis units calculate a state index value, which is an index value of a plant growth state, in the cultivation area by performing a neural network-based analysis on a cultivation area image captured by taking a picture of the plant cultivation area, each including a neural network trained on a plurality of cultivation area images classified into first to N-th tree vigor index values ​​by a predetermined tree vigor index value, where N ≥ 2; as well as The selection unit receives an input of a cultivation area image for which the state index value is to be calculated, and causes the image analysis units among the first to Nth image analysis units, which are trained using a plurality of cultivation area images classified as having the same tree vigor index value as the cultivation area image, to analyze the input cultivation area image. The neural network included in the i-th image analysis unit is to calculate the vertical pixel number H i , horizontal pixel number W i The image of is taken as input to the neural network #i, The i-th image analysis unit extracts a plurality of images with an aspect ratio H from the cultivated area image to be analyzed. i :W i The extracted images are converted into the number of vertical pixels H i , horizontal pixel number W i The image is then input into the neural network #i, i is an integer value from 1 to N.

2. The information processing device according to claim 1, wherein Determine the input image size of neural networks #i to #N so that for each integer value i from 1 to N-1, H i+1 / W i+1 >H i / W i Established.

3. The information processing device according to claim 1 or 2, characterized in that The tree vigor index value is a representative value of the plant height in the cultivation area.

4. The information processing device according to claim 1 or 2, characterized in that The tree vigor index value is a representative value of plant heights in the cultivation area obtained by analyzing the cultivation area image.

5. An information processing device, characterized in that Calculating a state index value as an index value of plant growth state, with: a neural network for calculating the state index value, the neural network being trained by extracting, from each of a plurality of cultivation area images of the cultivation area of ​​the plant, images of sizes corresponding to the tree vigor index value divisions to which the tree vigor of the plant in the cultivation area image belongs, and inputting the images, wherein N is greater than or equal to 2; as well as The calculation unit calculates the state index value by extracting an image of a size corresponding to the tree vigor index value division to which the tree vigor of the plant in the cultivation area image belongs from the cultivation area image for which the state index value is to be calculated, and inputting the image into the neural network, The number of vertical pixels and the number of horizontal pixels corresponding to the size of the i-th division are expressed as H i 、W i When the sizes corresponding to the first to Nth partitions are determined, H i+1 / W i+1 >H i / W i Established, k is an integer value from 1 to N-1.

Images