Information processing system, information processing method, and program

Abstract

To solve the problem that the dry weight method requires extracting and drying a plant to be measured, resulting in considerable labor and time consumption.SOLUTION: An information processing system acquires a first captured image, the first captured image being an image in which a plant in a first region is captured. The information processing system acquires a second captured image, the second captured image being an image in which a plant in a second region is captured. The information processing system determines a degree of plant vegetation in the first region based on the first captured image and the second captured image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] The dry weight method is a method for evaluating how large a plant has grown (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-130146 A Summary of the Invention [Problem to be solved by the invention]

[0004] The dry weight method requires the plants to be measured to be removed and dried, which is time-consuming and laborious. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, there is provided an information processing system. The information processing system acquires a first captured image. The first captured image is a captured image of plants in a first region. The information processing system acquires a second captured image. The second captured image is a captured image of plants in a second region. The information processing system calculates a degree of growth of plants in the first region based on the first captured image and the second captured image. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram illustrating an example of a hardware configuration of a mobile terminal device. [Figure 2] FIG. 2 is a diagram showing an example of screen transition when capturing an image of a plant. [Figure 3] FIG. 3 is a diagram illustrating an example of a guide object. [Figure 4] FIG. 4 is a flowchart illustrating an example of information processing executed by the mobile terminal device. [Figure 5] FIG. 5 is a diagram (part 1) showing an example of the data input screen. [Figure 6] FIG. 6 is a diagram (part 2) showing an example of the data input screen. [Figure 7] FIG. 7 is a diagram showing an example of a data input screen according to the first modification. [Figure 8] FIG. 8 is a diagram showing an example of a data input screen according to the second modification. DETAILED DESCRIPTION OF THE INVENTION

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0008] <Embodiment 1> 1. Hardware Configuration 1 is a diagram showing an example of the hardware configuration of mobile terminal device 100. Mobile terminal device 100 includes, as its hardware configuration, a control unit 110, a storage unit 120, an imaging unit 130, an input / output unit 140, and a communication unit 150.

[0009] The control unit 110 is a CPU (Central Processing Unit) or the like, and controls the entire mobile terminal device 100 .

[0010] The storage unit 120 is any one of a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), etc., or any combination thereof, and stores programs, data used when the control unit 110 executes processing based on the programs, etc. The storage unit 120 is an example of a storage medium. Examples of data used when the control unit 110 executes processing based on the programs include captured images, blank data, and measurement data, which will be described later.

[0011] In the specification, the data used when control unit 110 executes processing based on a program is described as being stored in storage unit 120, but the data may also be stored in a storage unit of another device that can communicate with mobile terminal device 100. In other words, the data may be stored in a storage unit of any device as long as it can be referenced or acquired by control unit 110. When control unit 110 executes processing based on a program stored in storage unit 120, the functions of mobile terminal device 100 and the processing of the flowchart shown in Fig. 4 described below are realized.

[0012] The imaging unit 130 is a camera or the like, and captures an image of a subject. The input / output unit 140 is a touch panel display or the like, and inputs operation information from the user and outputs images and the like under the control of the control unit 110. Here, the user refers to the operator who operates the mobile terminal device 100.

[0013] The communication unit 150 connects the mobile terminal device 100 to a network and controls communication with other devices. The communication unit 150 may include a communication unit that performs mobile data communication with other devices using a wireless communication system such as 4G or 5G, and a communication unit that connects to the Internet via an access point using a wireless communication technology such as Wi-Fi (registered trademark) and communicates with other devices.

[0014] An example of the mobile terminal device 100 is a smartphone. However, the mobile terminal device 100 is not limited to a smartphone and may be a tablet-type terminal device. The mobile terminal device 100 may be any device that can display a screen described below and capture an image of a subject such as a plant.

[0015] Here, the information processing system described in the claims may be composed of multiple devices or may be composed of a single device. When the information processing system described in the claims is composed of a single device, an example of that device is the mobile terminal device 100. When the information processing system described in the claims is composed of a plurality of devices, an example of the plurality of devices is the mobile terminal device 100 and a server device. Part of the processing of the mobile terminal device 100 described below may be executed by a server device that can communicate with the mobile terminal device 100 via wireless communication or the like.

[0016] 2. Information Processing The information processing of the first embodiment will be described below.

[0017] (1) Overview of the process The control unit 110 acquires a first captured image. The first captured image is an image of plants in a first region. The control unit 110 also acquires a second captured image. The second captured image is an image of plants in a second region. The control unit 110 calculates the degree of luxuriance of plants in the first region based on the first captured image and the second captured image. Here, the degree of luxuriance is a value indicating how lush the plants are compared to a comparison target.

[0018] This configuration eliminates the need to remove or dry the plants to be measured, allowing the degree of plant growth to be determined efficiently and quickly. Furthermore, this configuration also allows for the determination of quantitative and uniform evaluation values.

[0019] For example, the first area and the second area are different areas. For example, the first area is an area to which a predetermined pesticide or fertilizer has been supplied. The second area is an area to which the predetermined pesticide or fertilizer has not been supplied. As another example, the first region and the second region may be regions with different environments such as different sunlight conditions or different regions. The first region and the second region may be regions with different plant growth conditions. As another example, the first area and the second area may be the same area. For example, the area of ​​a field at a certain date and time may be the first area, and the same area of ​​the same field after a predetermined period of time (e.g., one week) has passed may be the second area.

[0020] However, for the sake of simplicity, the following explanation will be given assuming that the first area and the second area are different areas, with the first area being an area to which a specified pesticide or fertilizer has been supplied, and the second area being an area to which the specified pesticide or fertilizer has not been supplied. In the following, the first region is also referred to as the measurement region. That is, the measurement region is a region to which a predetermined pesticide or fertilizer is supplied. The second region is also referred to as the blank region. That is, the blank region is a region to which a predetermined pesticide or fertilizer is not supplied. The first captured image is also referred to as the measurement image. The second captured image is also referred to as the blank image.

[0021] (2) Details of the processing (2-1) Imaging and storage processing FIG. 2 is a diagram showing an example of screen transition when capturing an image of a plant. When the control unit 110 receives a predetermined operation, it controls the input / output unit 140 to display a screen 210. The screen 210 includes at least a blank button 215 and a measurement button 220. The screen 210 is an example of a second screen. As shown in FIG. 2, the screen 210 is configured to allow the user to select whether the plant to be photographed is a plant in the measurement area or a plant in the blank area.

[0022] When it is detected that the blank button 215 has been selected, the control unit 110 controls the input / output unit 140 to display a screen 225. The screen 225 is a screen that is displayed when capturing an image of a plant. The screen 225 is an example of a third screen. The screen 225 includes at least a subject display area 230 and an image capture button 235. The subject display area 230 is an area where the subject captured by the imaging unit 130 is displayed. In the example of FIG. 2, the subject display area 230 also displays an image 260 indicating that a blank image will be captured. The user places a plant in the blank area in the subject display area 230 within the imaging range and selects the imaging button 235. Upon detecting that the imaging button 235 has been selected, the control unit 110 controls the imaging unit 130 to capture an image of the plant in the blank area, which is the subject. The control unit 110 acquires the image captured via the screen 225 as a blank image and stores it in a predetermined area of ​​the storage unit 120. That is, the control unit 110 acquires, as a blank image, an image captured after the plant in the blank area is selected via the screen 210.

[0023] When it is detected that the measurement button 220 has been selected, the control unit 110 controls the input / output unit 140 to display a screen 240. The screen 240 is a screen that is displayed when capturing an image of a plant. The screen 240 is also an example of a third screen. The screen 240 includes at least a subject display area 245 and an image capture button 250. The subject display area 245 is an area where the subject captured by the imaging unit 130 is displayed. In the example of FIG. 2, an image 270 indicating that a measurement image will be captured is displayed in the subject display area 245. The user places the plant in the measurement area within the imaging range in the subject display area 245 and selects the imaging button 250. Upon detecting that the imaging button 250 has been selected, the control unit 110 controls the imaging unit 130 to capture an image of the plant in the measurement area, which is the subject. The control unit 110 acquires the image captured via the screen 240 as a measurement image and stores it in a predetermined area of ​​the memory unit 120. That is, the control unit 110 acquires, as a measurement image, an image captured after the plant in the measurement area is selected via the screen 210.

[0024] (2-2) Guide object display processing Here, when the user captures images of the plants in the blank area and the measurement area, the control unit 110 may display a guide object 320 in the subject display area 310. That is, the control unit 110 may display the following guide object 320 in the subject display area 230 and the subject display area 245 in FIG.

[0025] FIG. 3 is a diagram showing an example of a guide object. Screen 300 is a screen displayed when capturing an image of a plant. Screen 300 is also an example of a third screen. Screen 300 includes at least a subject display area 310, a guide object 320, and an image capture button 330. Subject display area 310 is similar to subject display area 230 and subject display area 245. Image capture button 330 is similar to image capture button 235 and image capture button 250. Guide object 320 is an image that guides the user in adjusting the angle of view when capturing an image of a plant. Note that the guide object may be a transparent image that does not obstruct the user's view of the plant. A transparent image is an image through which the background is visible. An example of an image through which the background is visible is an image with a transparent background. More specifically, a transparent image is an image that has been processed to increase the transparency, which is the degree to which the image is transparent and allows the background color to show through, compared to the original image. The original image is the image before the transparency-increasing process is applied.

[0026] The image that guides the user in adjusting the angle of view when capturing an image of a plant is not limited to guide object 320 in FIG. 3, but may be, for example, an image of a plant that was previously captured and selected by the user as a reference captured image. Furthermore, for example, when a measurement image is about to be captured after a blank image has been captured, the control unit 110 may display the blank image as a transparent image in the subject display area. Similarly, when a blank image is about to be captured after a measurement image has been captured, the control unit 110 may display the measurement image as a transparent image in the subject display area. By performing such processing, the angles of view of the plants in the measurement image and the blank image can be aligned. Furthermore, if the captured image includes an object other than a plant (for example, a planter), the planter may also be displayed as a transparent image in the subject display area. By performing such processing, the user can align the size of the planter in the transparent image with the planter surrounding the plant that is currently being captured. As a result, guidance can be provided to align the angle of view in both images. By aligning the angle of view, the accuracy of the calculation of the degree of luxuriance, which will be described later, can be improved.

[0027] (2-3) Calculation of growth level FIG. 4 is a flowchart showing an example of information processing executed by the mobile terminal device 100. As shown in FIG. In step S410, control unit 110 accepts an operation from the user via input / output unit 140 to display a data entry screen. In step S415, the control unit 110 generates a data entry screen. In step S420, control unit 110 displays the generated data entry screen on input / output unit 140. The processes of steps S415 and S420 are an example of a process of controlling to display the first screen.

[0028] 5 is a diagram (part 1) showing an example of a data input screen 500. The data input screen 500 is an example of a first screen. The data input screen 500 includes a first area 510 and a second area 550. The first area 510 includes at least a pull-down menu 520, a spin box 530, and a measurement start button 540.

[0029] The pull-down menu 520 is configured to allow selection of either blank data input or measured data input. Blank data input means inputting blank data. Blank data is data relating to an area to which a specified pesticide or fertilizer has not been applied. An area is the range of a specified plot. For example, an area is the range of one planter if plants are planted in a planter, or the range of one field if plants are planted in a field. Similarly, measured data input means inputting measured data. Measured data is data relating to an area to which a specified pesticide or fertilizer has been applied.

[0030] In step S425, control unit 110 determines whether "blank data input" or "measurement data input" has been selected in pull-down menu 520. If control unit 110 determines that "blank data input" has been selected in pull-down menu 520, it proceeds to step S430. On the other hand, if control unit 110 determines that "measurement data input" has been selected in pull-down menu 520, it proceeds to step S440.

[0031] In step S430, control unit 110 switches the display of data entry screen 500 to a display for blank data entry. For example, control unit 110 displays "Number of Blank Data Items" near spin box 530. Control unit 110 also switches the display of second area 550 to a display for blank data. The display of second area 550 in FIG. 5 is an example of a display when "Blank Data Entry" is selected in pull-down menu 520.

[0032] 5 is a box for inputting the number of data selected in the pull-down menu 520. In the example of FIG. 5, blank data is selected in the pull-down menu 520, so the number of blank data is input in the spin box 530. For the sake of simplicity, the following description will be given assuming that 1 is input in the spin box 530.

[0033] The measurement start button 540 is a button selected when the user instructs calculation of the degree of growth.

[0034] The second area 550 is an area where the user inputs the data selected in the pull-down menu 520. In the example of Fig. 5, blank data is selected in the pull-down menu 520, so the second area 550 in Fig. 5 is an area where the user inputs the blank data. The second area 550 in Fig. 5 includes an image selection area 560, a seeding number input area 570, and a germination number input area 580.

[0035] Image selection area 560 is an area for selecting a blank image. For example, control unit 110 acquires an image that is dragged and dropped into image selection area 560 as a blank image. Note that dragging and dropping is an example of a process for selecting an image, and is not limited to this. For example, when control unit 110 detects that image selection area 560 has been selected, it may display blank images stored in storage unit 120 or the like of mobile terminal device 100 in second area 550 or the like so that the blank image can be selected, and acquire a blank image selected from the displayed images via input / output unit 140. That is, the control unit 110 selects a blank image based on the user's image selection operation via the image selection area 560.

[0036] The seed number input area 570 is an area where the user inputs the number of seeds to be sown for the plants captured in the blank image selected in the image selection area 560 . The germination number input area 580 is an area where the user inputs the number of germinations of the plant captured in the blank image selected in the image selection area 560 . That is, the data input screen 500 is configured so that the number of seeds sown and the number of germinated plants can be input.

[0037] In step S435, the control unit 110 receives blank data via the data input screen 500 and stores it in a predetermined area or the like of the storage unit 120. The blank data is data including at least a blank image selected via the image selection area 560, the number of seeds input via the seeding number input area 570, and the number of germinations input via the germination number input area 580. The process of step S435 is an example of a process of acquiring germination data of plants included in the blank image as second germination data. The second germination data includes the number of seeds sown and the number of germinated plants included in the blank image. The process of step S435 is also an example of a process of acquiring second germination data input via the data input screen 500.

[0038] In step S440, the control unit 110 switches the display of the data input screen 500 to one for inputting measurement data. FIG. 6 is a diagram (part 2) showing an example of the data input screen 500. For example, the control unit 110 displays "Number of measurement data" or the like near the spin box 530. The control unit 110 also switches the display of the second area 550 to a display for measurement data. The display of the second area 550 in FIG. 6 is an example of the display when "Input measurement data" is selected in the pull-down menu 520.

[0039] The second area 550 in FIG. 6 includes an image selection area 610, a seeding number input area 620, and a germination number input area 630. The image selection area 610 is an area for selecting a measurement image. For example, the control unit 110 acquires an image that is dragged and dropped into the image selection area 610 as the measurement image. Note that dragging and dropping is an example of a process for selecting an image, and the process is not limited to this. For example, when the control unit 110 detects that the image selection area 610 has been selected, it may display measurement images stored in the storage unit 120 or the like of the mobile terminal device 100 on the input / output unit 140 in a selectable manner, and acquire a measurement image selected from the displayed images via the input / output unit 140. That is, the control unit 110 selects a measurement image based on the user's image selection operation via the image selection area 610.

[0040] The seed number input area 620 is an area where the user inputs the number of seeds to be sown for the plant captured in the measurement image selected in the image selection area 610. The germination number input area 630 is an area where the user inputs the number of germinations of the plant captured in the measurement image selected in the image selection area 610.

[0041] In step S445, the control unit 110 receives the measurement data via the data input screen 500 and stores it in a predetermined area or the like of the memory unit 120. The measurement data is data including at least the measurement image selected via the image selection area 610, the number of seeds input via the seeding number input area 620, and the number of germinations input via the germination number input area 630. The process of step S445 is an example of a process for acquiring germination data of plants included in the measurement image as first germination data. The first germination data includes the number of seeds sown and the number of germinated seeds of plants included in the measurement image. For simplicity of explanation, the following description will be given assuming that the number of seeds sown in the blank image and the number of seeds sown in the measurement image are the same. The process of step S445 is also an example of a process of acquiring the first germination data input via the data input screen 500.

[0042] In step S450, control unit 110 determines whether blank data and measurement data have been received via the screen. If control unit 110 determines that blank data and measurement data have been received, it proceeds to step S455. If control unit 110 determines that blank data and measurement data have not been received, it returns to step S425.

[0043] In step S455, the control unit 110 displays the measurement start button 540 as selectable on the data input screen 500. The process of displaying the measurement start button 540 as selectable on the data input screen 500 is an example of a process of enabling the measurement start button 540.

[0044] In step S460, control unit 110 determines whether or not measurement start button 540 has been selected. If control unit 110 determines that measurement start button 540 has been selected, control unit 110 proceeds to step S465. If control unit 110 determines that measurement start button 540 has not been selected, control unit 110 repeats the processing of step S460.

[0045] In step S465, the control unit 110 calculates the area of ​​the leaves of the plants in the measurement region included in the measurement image as the first area based on the measurement image. The control unit 110 calculates the area of ​​the leaves of the plants in the blank region included in the blank image as the second area based on the blank image. The control unit 110 calculates the degree of plant growth in the measurement region based on the first area and the second area. More specifically, the control unit 110 calculates the area of ​​the leaves of the plants in the measurement region included in the measurement image as the first area based on the pixel values ​​of the measurement image. The control unit 110 calculates the area of ​​the leaves of the plants in the blank region included in the blank image as the second area based on the pixel values ​​of the blank image.

[0046] More specifically, the control unit 110 calculates the germination rate S (%) of the plants included in the measurement image based on the first germination data. The control unit 110 calculates the germination rate T (%) of the plants included in the blank image based on the second germination data. Here, the germination rate is (number of germinated seeds / number of seeds sown)×100. The control unit 110 calculates the degree of growth based on the following (Equation 1), (Equation 2), and (Equation 3). N = {first area * (100 / S)} / (total area of ​​the measured image) (Equation 1) M = {area of ​​the second image * (100 / T)} / (total area of ​​the blank image) (Equation 2) Flourishing degree = N / M (Formula 3) Note that the control unit 110 can compare the first area and the second area under the same conditions by reflecting information on the number of seeds sown and / or the number of germinated seeds in the area.

[0047] In step S470, the control unit 110 outputs the determined degree of growth. The control unit 110 may output the degree of growth by writing it in a file of a predetermined format and storing it in the storage unit 120 or the like, or may output the degree of growth by displaying it on the input / output unit 140, or may output the degree of growth by transmitting it to another device with which it can communicate via a network or the like.

[0048] According to the process of the first embodiment, it is possible to evaluate how large a plant has grown without having to remove or dry the plant. Furthermore, according to the process of the first embodiment, it is possible to perform quantitative and uniform evaluation.

[0049] <Variation 1> Modification 1 is a modification of Embodiment 1. Modification 1 is included in Embodiment 1, and configurations, processes, etc. that are not described in Modification 1 are the same as those in Embodiment 1. (Select trimming range) The control unit 110 of Modification 1 (hereinafter simply referred to as the control unit 110) controls the input / output unit 140 to display a data input screen 500 of Modification 1. FIG. 7 is a diagram showing an example of the data input screen 500 of Modification 1. The data input screen 500 of Modification 1 (hereinafter simply referred to as the data input screen 500) is an example of a fourth screen. Compared to the data input screen 500 shown in FIG. 5, the data input screen 500 includes an image display area 710 that displays an image selected in an image selection area. The image display area 710 is configured to be able to accept a user's selection operation of a trimming range. The trimming range is an example of a processing range. When the control unit 110 accepts a user's selection operation of a trimming range in the image display area 710, it sets the trimming range in accordance with the selection operation and superimposes an image indicating the trimming range on the image display area 710. A frame image 720 is an example of an image indicating the trimming range. When the user specifies the trimming range and performs a predetermined operation, control unit 110 determines the trimming range and performs control so that a blank image cut out within the trimming range is displayed in preview area 730. In preview area 730, the user can check the image cut out within the trimming range. Although the blank image has been used as an example in the explanation of FIG. 7, the same applies to the measurement image.

[0050] The control unit 110 calculates the area of ​​the leaves of the plants in the measurement area included in the measurement image as the first area based on the measurement image of the trimming range. The control unit 110 calculates the area of ​​the leaves of the plants in the blank area included in the blank image as the second area based on the blank image of the trimming range. The control unit 110 calculates the degree of plant growth in the measurement area based on the first area and the second area. These processes are an example of a process for calculating the degree of plant growth in the measurement area based on the selection area selected as the processing area of ​​the measurement image selected by a selection operation received via the data input screen 500 and the selection area selected as the processing area of ​​the blank image selected by a selection operation received via the data input screen 500.

[0051] More specifically, the control unit 110 calculates the area of ​​the leaves of the plant in the measurement area included in the measurement image as the first area based on the pixel values ​​of the measurement image in the trimming range, and calculates the area of ​​the leaves of the plant in the blank area included in the blank image as the second area based on the pixel values ​​of the blank image in the trimming range.

[0052] More specifically, the control unit 110 calculates the germination rate S (%) of the plants included in the measurement image based on the first germination data. The control unit 110 calculates the germination rate T (%) of the plants included in the blank image based on the second germination data. The control unit 110 then calculates the degree of growth based on (Equation 1), (Equation 2), and (Equation 3) shown in the first embodiment. That is, in the first modification, the control unit 110 also uses the total area of ​​the measurement image as the denominator value of (Equation 1), and the total area of ​​the blank image as the denominator value of (Equation 2).

[0053] According to the first modification, the user can specify the trimming range.

[0054] <Variation 2> Modification 2 is a modification of Embodiment 1. Modification 2 is included in Embodiment 1, and configurations, processes, etc. not described in Modification 2 are the same as those in Embodiment 1 or Modification 1. (Color and size selection (part 1)) The control unit 110 of the second modification (hereinafter simply referred to as the control unit 110) controls the input / output unit 140 to display a data input screen 500 of the second modification. Fig. 8 is a diagram showing an example of the data input screen 500 of the second modification. The data input screen 500 of the second modification (hereinafter simply referred to as the data input screen 500) further includes a pull-down menu 810 and a pull-down menu 820 in the second area 550, as compared to the data input screen 500 shown in Fig. 7.

[0055] The pull-down menu 810 is configured to allow the user to select the leaf size of the target plant. When it is detected that the pull-down menu 810 has been selected, the control unit 110 displays information indicating multiple leaf sizes in the pull-down menu 810. The user can select the leaf size of the target plant in the pull-down menu 810. The information indicating the leaf size may be the linear distance from the base of the leaf to the tip in centimeters (for example, 5 cm or less, 5 to 10 cm, 10 to 15 cm, 15 cm or more, etc.), or may be a size ranking such as large, medium, or small.

[0056] The pull-down menu 820 is configured to allow the user to select the leaf color of the target plant. When it is detected that the pull-down menu 820 has been selected, the control unit 110 displays information indicating multiple leaf colors in the pull-down menu 820. The user can select the leaf color of the target plant in the pull-down menu 820. The information indicating the leaf color may be a written representation of the color, such as light green or green, or a display of the actual color. Although the blank image has been used as an example in the explanation of FIG. 8, the same applies to the measurement image.

[0057] When the trimming range is specified, the size of the leaves of the target plant is selected in pull-down menu 810, and the color of the leaves of the target plant is selected in pull-down menu 820, control unit 110 extracts objects (i.e., plant leaves) of the selected size and color from among the objects within the trimming range, and controls display of the objects in preview area 830. Control unit 110 controls display of the extracted objects in preview area 830 so that they can be distinguished from other objects. In the example of FIG. 8, the extracted plant leaves are shown in white, but this is not limited to this. Any mode may be used as long as the extracted objects can be distinguished from other objects. By performing such processing, the user can confirm that the desired plant has been selected.

[0058] The control unit 110 extracts a plant in the measurement area from the measurement image based on the size of the object included in the measurement image and the pixel values ​​of the object area. Here, the size of the object is the size of the leaf of the plant selected in the pull-down menu 810. Furthermore, the pixel values ​​of the object area are the pixel values ​​of the leaf color of the plant selected in the pull-down menu 820. The control unit 110 calculates the area of ​​the leaf of the plant in the extracted measurement area as the first area. Similarly, control unit 110 extracts a plant in a blank area from the blank image based on the size of the object included in the blank image and the pixel value of the object area. The size of the object is the size of the leaf of the plant selected in pull-down menu 810. The pixel value of the object area is the pixel value of the leaf color of the plant selected in pull-down menu 820. Control unit 110 calculates the area of ​​the leaf of the plant in the extracted blank area as the second area.

[0059] According to the second modification, by specifying the size and color of the target plant, the target plant can be extracted from the captured image and the degree of growth can be calculated.

[0060] (Color and size selection (part 2)) As another example, in Modification 2, the control unit 110 has been described as displaying a screen configured to allow selection of the size and color of leaves of target plants. However, the control unit 110 may also be configured to display a screen configured to allow selection of the size and color of leaves of plants that are not targets, i.e., plants that are not to be processed in image processing as weeds, etc. In such a configuration, the control unit 110 may exclude leaves of plants with the size, color, etc. selected via the screen from targets for processing as weeds, and calculate the leaf area of ​​the target plants.

[0061] By selecting the size and color of weeds and excluding them from the processing target, it is also possible to extract the target plants from the captured image and determine the degree of growth.

[0062] (Leaf shape selection) As another example, the control unit 110 may display a pull-down menu on the data input screen 500 from which the user can select the type of leaf shape of the target plant. The control unit 110 may determine the leaf area of ​​the target plant by regarding leaves of a shape selected from such a pull-down menu as leaves of the target plant, i.e., by excluding leaves of any other shape as weeds from the target plant. Examples of leaf shapes include egg-shaped, inverted egg-shaped, oval, heart-shaped, palm-shaped, narrow-legged, needle-shaped, and scale-shaped. The pull-down menu may display text such as egg-shaped or inverted egg-shaped, or may display sample images of leaf shapes such as egg-shaped or inverted egg-shaped.

[0063] Furthermore, by making it possible to select the type of leaf, it is possible to more accurately extract the target plant from the captured image and determine the degree of luxuriance.

[0064] <Variation 3> Modification 3 is a modification of Embodiment 1. Modification 3 is included in Embodiment 1, and the configuration, processing, etc. not described in Modification 3 are the same as those in Embodiment 1. In Modification 3, removal of overlapping leaves will be described. (AI removes overlaps) The control unit 110 of Modification 3 (hereinafter simply referred to as the control unit 110) inputs the measured image into the trained model. The trained model is trained using a captured image of the plant as input data and the area of ​​each of the plant's multiple leaves, with overlapping leaves removed, as output data. The control unit 110 obtains the area of ​​each of the multiple leaves of the plant in the measurement region included in the measured image from the trained model as a first area. Similarly, the control unit 110 inputs the blank image to the trained model. The control unit 110 acquires the area of ​​each of the multiple leaves of the plant in the blank region included in the blank image from the trained model as the second area.

[0065] According to the third modification, overlapping leaves can be removed and the degree of growth can be calculated based on the accurate area of ​​each of the multiple leaves of the plant.

[0066] (Shake to remove overlaps) As another example, the control unit 110 may output guidance information on a screen displayed when capturing an image of a plant, encouraging the user to capture at least two blank images and a measurement image. For example, the guidance information may be, "Take images before and during vibration!" For example, after the blank button 215 is selected on the screen 210 shown in FIG. 2, the control unit 110 controls the input / output unit 140 to display a screen including a pre-vibration button and a vibrating button.

[0067] The control unit 110 acquires and stores an image captured when the measurement button 220 is selected and then the pre-vibration button is selected as a measurement vibration image. The control unit 110 acquires and stores an image captured when the blank button 215 is selected and then the vibration button is selected as a measurement vibration image. The pre-measurement vibration image is an example of a 1A captured image. The measurement vibration image is an example of a 1B captured image. The plant included in the measurement vibration image is a plant to which vibration has been applied compared to the plant included in the pre-measurement vibration image. The control unit 110 compares the pre-measurement vibration image with the measurement vibration image to determine the area of ​​the plant leaves (first area) with overlapping plant leaves in the measurement image removed. This process is an example of a process of determining the area of ​​the plant leaves in the measurement region included in the pre-measurement vibration image and the measurement vibration image as the first area based on the measurement images, i.e., the pre-measurement vibration image and the measurement vibration image.

[0068] Similarly, the control unit 110 acquires and stores an image captured when the blank button 215 is selected and then the pre-vibration button is selected as a blank pre-vibration image. The control unit 110 acquires and stores an image captured when the blank button 215 is selected and then the during-vibration button is selected as a blank during-vibration image. The blank pre-vibration image is an example of a 2A captured image. The blank during-vibration image is an example of a 2A captured image. The plants included in the blank during-vibration image are plants to which vibration has been applied compared to the plants included in the blank pre-vibration image. The control unit 110 obtains the area of ​​the plant leaves (second area) by, for example, comparing the blank pre-vibration image with the blank during-vibration image, removing overlapping plant leaves in the blank image. This processing is an example of processing for obtaining the area of ​​the plants in the blank regions included in the blank pre-vibration image and the blank during-vibration image as the second area based on the blank pre-vibration image and the blank during-vibration image, which are blank images.

[0069] The control unit 110 determines the degree of vegetation in the measurement area based on the first area and the second area.

[0070] By shaking the plant and taking an image, overlapping leaves can also be removed, and the degree of growth can be determined based on the accurate area of ​​each of the plant's multiple leaves. It should be noted that the term "image before vibration" may be read as "image (still) after vibration." In other words, the same effect can be obtained by comparing an image during vibration with an image still after vibration. Furthermore, the image during vibration is not limited to one image, and for example, when capturing an image during vibration, the control unit 110 may capture multiple still images in continuous shooting mode based on one imaging instruction. In such a configuration, the control unit 110 may recognize overlapping of leaves by comparing multiple captured images captured in continuous shooting mode, rather than comparing an image captured when the plant is not vibrating with an image captured when the plant is vibrating, and may calculate the accurate area of ​​each of the multiple leaves of the plant.

[0071] <Variation 4> Modification 4 is a modification of Embodiment 1. Modification 4 is included in Embodiment 1, and the configuration, processing, etc. not described in Modification 4 are the same as those in Embodiment 1.

[0072] 3, the control unit 110 has been described as displaying a guide object 320 on the screen when capturing an image of a plant, the guide object 320 providing guidance for adjusting the angle of view when capturing an image of the plant. However, the control unit 110 of Modification 4 (hereinafter simply referred to as the control unit 110) may perform image processing such as enlargement, reduction, or rotation on the captured image based on a marker included in the captured image so that the angles of view of the blank image and the measurement image are aligned. The marker may be an object placed by the user that has a complementary color to green.

[0073] <Additional Notes> It may be provided in the following manner. (Appendix 1) An information processing system, A first captured image is acquired; the first captured image is a captured image of a plant in a first region, A second captured image is acquired; the second captured image is a captured image of a plant in a second region, determining a degree of plant growth in the first area based on the first captured image and the second captured image; Information processing system. (Appendix 2) 10. The information processing system of claim 1, The first region and the second region are different regions. Information processing system. (Appendix 3) 10. The information processing system according to claim 2, the first area is an area to which a predetermined pesticide or fertilizer has been supplied, The second area is an area to which the specified pesticide or fertilizer is not supplied. Information processing system. (Appendix 4) An information processing system according to any one of Supplementary Note 1 to Supplementary Note 3, calculating an area of ​​a leaf of a plant in the first region included in the first captured image as a first area based on the first captured image; calculating an area of ​​a leaf of a plant in the second region included in the second captured image as a second area based on the second captured image; determining a degree of plant growth in the first region based on the first area and the second area; Information processing system. (Appendix 5) An information processing system according to any one of Supplementary Note 1 to Supplementary Note 4, calculating an area of ​​a leaf of a plant in the first region included in the first captured image as a first area based on pixel values ​​of the first captured image; calculating an area of ​​a leaf of a plant in the second region included in the second captured image as a second area based on pixel values ​​of the second captured image; determining a degree of plant growth in the first region based on the first area and the second area; Information processing system. (Appendix 6) An information processing system according to any one of Supplementary Note 1 to Supplementary Note 5, acquiring germination data of a plant included in the first captured image as first germination data; the first germination data includes the number of seeds sown and the number of seeds that have germinated in the first captured image; acquiring germination data of the plant included in the second captured image as second germination data; the second germination data includes the number of seeds sown and the number of germinated seeds of the plants included in the second captured image, determining a degree of growth of the plant in the first region based on the first captured image, the first germination data, the second captured image, and the second germination data; Information processing system. (Appendix 7) 7. The information processing system according to claim 6, Control to display the first screen, the first screen is configured to allow input of the number of seeds sown and the number of seeds that have germinated, acquiring the first germination data input via the first screen; acquiring the second germination data input via the first screen; Information processing system. (Appendix 8) An information processing system according to any one of Supplementary Note 1 to Supplementary Note 7, Control to display the second screen, the second screen is configured to allow selection of whether the plant to be imaged is a plant in the first area or a plant in the second area, acquiring an image captured after the plant in the first region is selected via the second screen as the first captured image; acquiring an image captured after the plant in the second region is selected via the second screen as the second captured image; Information processing system. (Appendix 9) An information processing system according to any one of Supplementary Note 1 to Supplementary Note 8, Control to display the third screen, the third screen is a screen displayed when capturing an image of a plant, a guide object is displayed on the third screen; the guide object is an image that guides adjustment of the angle of view when capturing an image of the plant. Information processing system. (Appendix 10) 10. The information processing system according to claim 9, The guide object is a transparent image. Information processing system. (Appendix 11) An information processing system according to any one of Supplementary Note 1 to Supplementary Note 10, Control to display the fourth screen, the fourth screen is configured to be able to accept a selection operation of a processing range of a captured image, determining a degree of vegetation in the first area based on a selected area selected as a processing range of the first captured image selected by the selection operation received via the fourth screen and a selected area selected as a processing range of the second captured image selected by the selection operation received via the fourth screen; Information processing system. (Appendix 12) 5. The information processing system according to claim 4, extracting a plant in the first region from the first captured image based on a size of an object included in the first captured image and a pixel value of a region of the object, and calculating an area of ​​a leaf of the extracted plant in the first region as a first area; extracting a plant in the second region from the second captured image based on a size of the object included in the second captured image and a pixel value of the region of the object, and calculating the area of ​​the leaves of the extracted plant in the second region as a second area. Information processing system. (Appendix 13) 5. The information processing system according to claim 4, inputting the first captured image into a trained model; The trained model is a trained model trained using a captured image of a plant as input data and the area of ​​each of a plurality of leaves of the plant, with overlapping leaves removed, as output data, acquiring, from the trained model, the area of ​​each of a plurality of leaves of the plant in the first region included in the first captured image as the first area; inputting the second captured image into the trained model; acquiring, from the trained model, the area of ​​each of a plurality of leaves of the plant in the second region included in the second captured image as the second area; Information processing system. (Appendix 14) 5. The information processing system according to claim 4, calculating an area of ​​the plant in the first region included in the first captured image and the first captured image as a first area based on the first captured image A and the first captured image B, which are the first captured images; the plant included in the first captured image is a plant to which vibration has been applied to the plant included in the first captured image, calculating an area of ​​the plant in the second region included in the secondA captured image and the secondB captured image as a second area based on the secondA captured image and the secondB captured image, which are the second captured images; the plant included in the second captured image is a plant to which vibration has been applied to the plant included in the second captured image, determining a degree of plant growth in the first region based on the first area and the second area; Information processing system. (Appendix 15) An information processing system according to any one of Supplementary Note 1 to Supplementary Note 13, outputting the determined degree of growth; Information processing system. (Appendix 16) An information processing method executed by an information processing system, A first captured image is acquired; the first captured image is a captured image of a plant in a first region, A second captured image is acquired; the second captured image is a captured image of a plant in a second region, determining a degree of plant growth in the first area based on the first captured image and the second captured image; Information processing methods. (Appendix 17) A program, Computer, A program for causing the information processing system described in any one of Supplementary Note 1 to Supplementary Note 15 to function as such.

[0074] Finally, while various embodiments of the present invention have been described, they are presented as examples and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. The embodiments and modifications thereof are intended to be included within the scope and spirit of the invention, and are also intended to be included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0075] 100: Portable terminal device 110: Control unit 120: Storage section 130: Imaging unit 140: Input / output section 150: Communications Department

Claims

1. An information processing system, A first captured image is acquired; the first captured image is a captured image of a plant in a first region, A second captured image is acquired; the second captured image is a captured image of a plant in a second region, determining a degree of plant growth in the first area based on the first captured image and the second captured image; Information processing system.

2. 2. The information processing system according to claim 1, The first region and the second region are different regions. Information processing system.

3. 3. The information processing system according to claim 2, the first area is an area to which a predetermined pesticide or fertilizer has been supplied, The second area is an area to which the specified pesticide or fertilizer is not supplied. Information processing system.

4. 2. The information processing system according to claim 1, calculating an area of ​​a leaf of a plant in the first region included in the first captured image as a first area based on the first captured image; calculating an area of ​​a leaf of a plant in the second region included in the second captured image as a second area based on the second captured image; determining a degree of plant growth in the first region based on the first area and the second area; Information processing system.

5. 2. The information processing system according to claim 1, calculating an area of ​​a leaf of a plant in the first region included in the first captured image as a first area based on pixel values ​​of the first captured image; calculating an area of ​​a leaf of a plant in the second region included in the second captured image as a second area based on pixel values ​​of the second captured image; determining a degree of plant growth in the first region based on the first area and the second area; Information processing system.

6. 2. The information processing system according to claim 1, acquiring germination data of a plant included in the first captured image as first germination data; the first germination data includes the number of seeds sown and the number of germinated seeds of the plants included in the first captured image; acquiring germination data of the plant included in the second captured image as second germination data; the second germination data includes the number of seeds sown and the number of germinated seeds of the plants included in the second captured image, determining a degree of growth of the plant in the first region based on the first captured image, the first germination data, the second captured image, and the second germination data; Information processing system.

7. 7. The information processing system according to claim 6, Controlling to display the first screen; the first screen is configured to allow input of the number of seeds sown and the number of seeds that have germinated, acquiring the first germination data input via the first screen; acquiring the second germination data input via the first screen; Information processing system.

8. 2. The information processing system according to claim 1, Controlling to display the second screen; the second screen is configured to allow selection of whether the plant to be imaged is a plant in the first area or a plant in the second area, acquiring an image captured after the plant in the first region is selected via the second screen as the first captured image; acquiring an image captured after the plant in the second region is selected via the second screen as the second captured image; Information processing system.

9. 2. The information processing system according to claim 1, Control to display the third screen, the third screen is a screen displayed when capturing an image of a plant, a guide object is displayed on the third screen, the guide object is an image that guides adjustment of the angle of view when capturing an image of the plant. Information processing system.

10. 10. The information processing system according to claim 9, The guide object is a transparent image. Information processing system.

11. 2. The information processing system according to claim 1, Control to display the fourth screen, the fourth screen is configured to be able to accept a selection operation of a processing range of a captured image, determining a degree of vegetation in the first area based on a selected area selected as a processing range of the first captured image selected by the selection operation received via the fourth screen and a selected area selected as a processing range of the second captured image selected by the selection operation received via the fourth screen; Information processing system.

12. 5. The information processing system according to claim 4, extracting a plant in the first region from the first captured image based on a size of an object included in the first captured image and a pixel value of a region of the object, and calculating an area of ​​a leaf of the extracted plant in the first region as a first area; extracting a plant in the second region from the second captured image based on a size of the object included in the second captured image and pixel values ​​of the region of the object, and calculating an area of ​​a leaf of the extracted plant in the second region as a second area; Information processing system.

13. 5. The information processing system according to claim 4, Inputting the first captured image into a trained model; The trained model is a trained model trained using a captured image of a plant as input data and the area of ​​each of a plurality of leaves of the plant, with overlapping leaves removed, as output data, acquiring, from the trained model, the area of ​​each of a plurality of leaves of a plant in the first region included in the first captured image as the first area; Inputting the second captured image into the trained model; acquiring, from the trained model, the area of ​​each of a plurality of leaves of the plant in the second region included in the second captured image as the second area; Information processing system.

14. 5. The information processing system according to claim 4, calculating an area of ​​the plant in the first region included in the firstA captured image and the firstB captured image as a first area based on the firstA captured image and the firstB captured image, which are the first captured images; the plant included in the first captured image is a plant to which vibration has been applied to the plant included in the first captured image, calculating an area of ​​the plant in the second region included in the secondA captured image and the secondB captured image as a second area based on the secondA captured image and the secondB captured image, which are the second captured images; the plant included in the second-B captured image is a plant to which vibration has been applied to the plant included in the second-A captured image, determining a degree of plant growth in the first region based on the first area and the second area; Information processing system.

15. 2. The information processing system according to claim 1, outputting the determined degree of growth; Information processing system.

16. An information processing method executed by an information processing system, A first captured image is acquired; the first captured image is a captured image of a plant in a first region, A second captured image is acquired; the second captured image is a captured image of a plant in a second region, determining a degree of plant growth in the first area based on the first captured image and the second captured image; Information processing methods.

17. A program, Computer, A program for causing the information processing system according to any one of claims 1 to 15 to function.

Images