Information processing apparatus, farm work support system, and information processing method

Abstract

To easily and highly accurately determine compound information of a cultivated plant.SOLUTION: An information processing device includes an image acquisition unit that acquires a captured image obtained by imaging a target object which is at least a part of a cultivated plant, a first determination unit that determines an optical characteristic at a predetermined position coordinate of the target object based on the captured image acquired by the image acquisition unit, and a second determination unit that determines compound information regarding a compound at the position coordinate based on the optical characteristic at the position coordinate determined by the first determination unit and predetermined determination information indicating a relationship between the optical characteristic and the compound.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an agricultural work support system, and an information processing method. [Background technology]

[0002] The portable measuring device disclosed in Patent Document 1 includes a halogen light source that irradiates a measurement object with measurement light including near-infrared light, and a measuring unit that receives transmitted light and / or reflected light from the measurement object to measure the quality of the measurement object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-101409 Summary of the Invention [Problem to be solved by the invention]

[0004] The internal quality of the measurement object (e.g., fruits and vegetables) of Patent Document 1 can be measured non-destructively, and furthermore, the measurement can be performed without bringing the crop to be measured back from the field, etc. However, in the portable measurement device of Patent Document 1, the measurement unit receives transmitted light and / or reflected light from one point on the measurement object and uses the received measurement light as spectral data for analysis, so there is room for improvement in the accuracy of quality measurement.

[0005] The present invention has been made to solve the problems of the conventional technology, and aims to provide an information processing device, an agricultural work support system, and an information processing method that can easily and accurately determine compound information of cultivated plants. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present invention includes an image acquisition unit that acquires an image of an object that is at least a part of a cultivated plant; a first determination unit that determines light characteristics of the object at a predetermined position coordinate based on the image acquired by the image acquisition unit; and a second determination unit that determines compound information related to the compound at the position coordinate based on the light characteristics at the position coordinate determined by the first determination unit and predetermined judgment information indicating the relationship between the light characteristics and the compound.

[0007] An agricultural work support system according to one embodiment of the present invention includes an imaging device that captures the image of the object, the information processing device, and a display device that displays information based on the compound information determined by the second determination unit.

[0008] An information processing method according to one aspect of the present invention includes a first step in which an imaging device captures an image of an object that is at least a part of a cultivated plant; a second step in which a first determination unit determines light characteristics at a predetermined position coordinate of the object based on the image captured in the first step; and a third step in which the second determination unit determines compound information related to the compound at the position coordinate based on the light characteristics at the position coordinate determined in the second step and judgment information indicating the relationship between the light characteristics and the compound. [Effects of the Invention]

[0009] According to the information processing device, agricultural work support system, and information processing method, compound information of cultivated plants can be determined easily and accurately. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram of a farm work support system. [Figure 2] FIG. 1 is a diagram showing how an image of an object is captured by an imaging device. [Figure 3] FIG. 2 is a diagram illustrating the flow of data in the agricultural work support system. [Figure 4]FIG. 1 is a diagram for explaining ray tracing. [Figure 5] FIG. 2 is a diagram schematically illustrating the relationship between each position coordinate and light characteristics on an object. [Figure 6] FIG. 10 is a diagram showing the relationship between optical characteristics and the amount of a predetermined compound. [Figure 7A] FIG. 1 is a first diagram schematically illustrating the relationship between predetermined position coordinates of an object and the amount of a predetermined compound. [Figure 7B] FIG. 2 is a second diagram schematically illustrating the relationship between predetermined position coordinates of an object and the amount of a predetermined compound. [Figure 8] FIG. 10 is a diagram showing an example of the configuration of a compound at each position coordinate determined by the second determination unit. [Figure 9] 1A and 1B are diagrams showing examples of the structure of a compound at a predetermined position coordinate of an object and examples of the structure of a compound having a predetermined structure. [Figure 10] 10A and 10B are diagrams illustrating examples of the structure of each position coordinate estimated by an estimation unit. [Figure 11] 10 is an example showing a support screen displayed on a display device. [Figure 12] FIG. 2 is a diagram illustrating a series of processing steps performed by the agricultural work support system. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of the present invention will be described below with reference to the drawings. Fig. 1 is a schematic diagram of an agricultural work support system S. As shown in Fig. 1, the agricultural work support system S includes an imaging device 21 that captures an image of an object 101, which is at least a part of a cultivated plant 100; an information processing device 41 that acquires an image 51 captured by the imaging device 21 and determines light characteristics (e.g., light intensity, luminance, absorbance, etc.) at a predetermined position coordinate Cn (n = 1, 2, 3, . . .) of the object 101 based on the image 51, thereby determining compound information related to the compound at the position coordinate Cn; and a display device 31 that displays information based on the compound information determined by the information processing device 41.

[0012] Here, the cultivated plant 100 is a plant cultivated by a farmer, such as a crop grown in a field such as a rice paddy, a field, or an orchard. Therefore, the imaging device 21 captures an image of an object 101 such as a fruit, a leaf, a stem, a seed, or a bud of the cultivated plant 100, and the information processing device 41 can determine compound information based on the captured image 51 captured by the imaging device 21.

[0013] The display device 31 can display information (quality information) on at least one of the moisture content, sugar content, and acidity of the object 101, allowing the farm worker to non-destructively grasp the quality information of the object 101. Note that the object 101 imaged by the imaging device 21 is not limited to at least a part of the cultivated plant 100, but may also be at least a part of a wild plant that grows naturally without being cultivated by a farm worker.

[0014] In the agricultural work support system S shown in Figure 1, an imaging device 21 and a display device 31 are provided in a portable terminal device 10 (support terminal) that can be carried by an agricultural worker or the like, and an information processing device 41 is provided in a fixed terminal device 40 (e.g., a server device) such as a fixed computer outside the support device.

[0015] 1, the imaging device 21, the display device 31, and the information processing device 41 may be provided in the same terminal device or in separate terminal devices. For example, the imaging device 21 may be provided in a portable terminal device (imaging terminal), and the information processing device 41 and the display device 31 may be provided in a fixed terminal device (support terminal). In such a case, the terminal device is, for example, a personal computer used by a farm worker or a manager.

[0016] Alternatively, the imaging device 21 may be provided in a portable terminal device (imaging terminal) or a work machine (such as a harvesting machine for harvesting crops or a cultivator for spraying pesticides), the display device 31 may be provided in a portable terminal device (support terminal), and the information processing device 41 may be provided in a fixed terminal device (for example, a server device). Furthermore, the imaging device 21, the information processing device 41, and the display device 31 may be provided in a single terminal device (support terminal). In these cases, the support terminal is a portable terminal device such as a personal computer, tablet terminal device, or smartphone with relatively high processing power. In the following explanation, the example shown in FIG. 1 will be mainly described as the agricultural work support system S, and detailed explanations of other variations will be omitted.

[0017] First, the imaging device 21 and the display device 31 will be described using a portable support terminal 10 as an example as shown in Fig. 1. In the example shown in Fig. 1, the support terminal 10 has the imaging device 21 and a main body (a portion including the display device 31 and the like other than the imaging device 21) connected by a cable, and the main body is a tablet-shaped terminal device.

[0018] The support terminal 10 has a first control device 11 (processing circuit) including one or more processors. The first control device 11 is a controller of the support terminal 10. The first control device 11 is communicably connected to a plurality of devices (such as the imaging device 21 and the display device 31) mounted on the support terminal 10, and can control these devices. For example, the first control device 11 controls (operates) the imaging device 21 to capture an image of the object 101, based on a signal (operation signal) input from an input interface. That is, in this embodiment, the first control device 11 controls both the imaging device 21 and the display device 31.

[0019] The first control device 11 includes one or more memories, various analog circuits, various digital circuits, etc. The one or more memories store (memorize) software programs and various data to be executed by one or more processors. The first control device 11 can read software programs from one or more memories using one or more processors and execute various processes based on the software programs. Note that the first control device 11 may also be able to execute various processes based on predetermined logic circuits using one or more processors.

[0020] The processor may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

[0021] The first control device 11 may execute various processes by having multiple physically separated processors cooperate with each other, and the configuration is not limited to the above-mentioned configuration. In such a case, the multiple processors are respectively installed in one or multiple computers that are physically separated from the support terminal 10, and these processors are connected to each other so as to be able to communicate with each other via a network such as a LAN, a WAN, or the Internet.

[0022] In addition, the software program may be stored on a recording medium communicatively connected to the first control device 11 or on an external server device 40 connected via the network, and installed from there into the memory.

[0023] 1, the support terminal 10 has a first storage unit 12. The first storage unit 12 is a storage medium such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive), and stores various information related to the support terminal 10. For example, the first storage unit 12 can store data (image data) of a captured image 51 captured by the imaging device 21 of an object 101.

[0024] As shown in FIG. 1, the support terminal 10 has a first communication device 13. The first communication device 13 is a device that transmits various data to an external device (e.g., a server device 40) of the support terminal 10 and receives data transmitted from the external device. The first communication device 13 performs wireless communication with the external device, for example, via Wi-Fi (Wireless Fidelity, registered trademark) of the IEEE802.11 series, which is a communication standard, a mobile phone communication network, or a data communication network. For example, the first communication device 13 transmits imaging data to the server device 40. In addition, the first communication device 13 receives information based on compound information of the target object 101 determined by the information processing device 41 from the server device 40.

[0025] As shown in Fig. 1, the support terminal 10 has an imaging device 21. The imaging device 21 can capture a captured image 51 of an object 101, which is at least a part of a cultivated plant 100. The imaging device 21 captures, for example, a still image of the object 101 as the captured image 51. Note that the imaging device 21 is only required to capture at least the captured image 51 of the object 101, and the imaging device 21 may also capture a moving image of the object 101 and generate moving image data.

[0026] The imaging device 21 can image the object 101 with reflected waves (reflected light) of infrared rays (near infrared, mid-infrared), visible light, ultraviolet rays (near ultraviolet, far ultraviolet), other electromagnetic waves, etc. The imaging device 21 can image the object 101 with reflected waves of one or more wavelengths L. The imaging device 21 can image the object 101 with reflected light of wavelengths L in a relatively narrow predetermined band (wavelength band). Each band of reflected light imaged by the imaging device 21 may be set to an appropriate band for determining compound information from the optical characteristics of the reflected light. Note that, in this embodiment, the case where the imaging device 21 images the object 101 with reflected waves of multiple wavelengths L will be mainly described, but the imaging device 21 can also image the object 101 with reflected waves of at least one wavelength L. It would be good if it came out.

[0027] The imaging device 21, for example, irradiates the object 101 with measurement light of a wavelength L in a relatively narrow predetermined band by the irradiation unit 23, and captures an image of the object 101 using light reflected from the measurement light. Below, the imaging device 21 will be described in detail, taking as an example a case where the imaging device 21 has the irradiation unit 23.

[0028] 1, the imaging device 21 has an imaging unit 22 and an irradiation unit 23. The imaging unit 22 converts an optical image formed by incident light into an image signal, and generates a captured image 51 from the image signal. The imaging unit 22 includes a lens 22a and an imaging element 22b. The lens 22a of the imaging unit 22 collects reflected light from the object 101, and the collected light beam forms an image on the light receiving surface of the imaging element 22b.

[0029] The imaging element 22b converts the optical image formed on the light receiving surface into an electrical signal (image signal). Examples of the imaging element 22b include a CMOS (Complementary Metal-Oxide Semiconductor) type and a CCD (Charge Coupled Device) type. The imaging element 22b has wavelength bands in which the optical image formed on the light receiving surface, i.e., the incident light, can be converted and wavelength bands in which it cannot be converted. Therefore, the imaging device 21 has one or more imaging elements 22b corresponding to appropriate wavelength bands for determining compound information from the optical characteristics of reflected light. For example, when the imaging device 21 captures an image of the object 101 using reflected light ranging from near ultraviolet to mid-infrared, the imaging device 21 has an imaging element 22b for near ultraviolet and an imaging element 22b for visible light to mid-infrared.

[0030] The irradiating unit 23 irradiates the object 101 with measurement light of a wavelength L in a relatively narrow predetermined band. Specifically, the irradiating unit 23 can irradiate the object 101 with measurement light of a plurality of different bands of wavelength L. The irradiating unit 23 irradiates the object 101 with measurement light of a wavelength L in a desired band based on a control signal from the first control device 11. The irradiating unit 23 irradiates the measurement light in a direction opposite to the incident direction of the reflected light on the lens 22a.

[0031] The irradiation unit 23 has one or more light sources 23a. The one or more light sources 23a are arranged around the lens 22a of the imaging unit 22. Examples of light sources 23a include an LED light source, a xenon lamp, a D2 lamp (deuterium lamp), and a mercury lamp. When the irradiation unit 23 has an LED light source as the light source 23a, the irradiation unit 23 has multiple LED light sources that are turned on with light of different wavelength bands L, and switches on and off the multiple LED light sources to irradiate measurement light of a desired wavelength band L. The irradiation unit 23 may have multiple LED light sources that are turned on with wavelengths L of the respective bands. In such a case, it is preferable that the LED light sources are arranged at equal intervals around the outer periphery of the lens 22a.

[0032] The configuration of the irradiating unit 23 is not limited to the above example, as long as it can irradiate measurement light of at least the desired band of wavelength L. For example, the irradiating unit 23 may irradiate measurement light of the desired band of wavelength L by switching between various filters (e.g., bandpass filters) that transmit only light of a predetermined band among the light irradiated from the light source 23 a.

[0033] With the above configuration, the imaging device 21 can capture images of the object 101 with reflected light of multiple wavelengths L by capturing images of the object 101 multiple times with reflected light of wavelengths L in different bands.

[0034] In the above example, the imaging device 21 captures an image of the object 101 using reflected light of measurement light irradiated by the irradiation unit 23. However, it is sufficient if the object 101 can be imaged using reflected light of a relatively narrow, predetermined band of wavelengths L. For this reason, light of a relatively wide band of wavelengths L (e.g., sunlight) may be irradiated onto the object 101, and the imaging device 21 may capture an image of the object 101 using reflected light that has been dispersed into a predetermined band of wavelengths L using a spectroscope. In such a case, the imaging device 21 may or may not include the irradiation unit 23 that irradiates measurement light of a relatively wide band of wavelengths L. The irradiation unit 23 may, for example, have a plurality of LED light sources that are lit at different bands of wavelengths L, and these plurality of LED light sources are lit simultaneously to irradiate measurement light of a relatively wide band of wavelengths L.

[0035] In the above example, the imaging device 21 captures an image of the object 101 multiple times with reflected light of wavelengths L in different bands, thereby capturing an image of the object 101 with reflected light of multiple wavelengths L. However, the imaging device 21 may simultaneously capture an image of the object 101 with reflected light of multiple wavelengths L by dispersing light into different bands of wavelengths L, as in a multispectral camera or hyperspectral camera. In such a case, the image obtained by the imaging device 21 capturing an image of the object 101 may be Since the data (data cube) includes wavelength information at each position of the image, it is possible to extract from the data cube a captured image 51 captured with reflected light of wavelength L in each band.

[0036] The imaging device 21 captures an image of the object 101 from at least one direction using reflected waves of different wavelengths L (more specifically, reflected light of the wavelength L in each band). It is preferable that the imaging device 21 captures an image of the object 101 in a 360° range without omission using reflected light of the wavelength L in each band. FIG. 2 is a diagram showing how the imaging device 21 captures an image of the object 101. In FIG. 2, the object 101 is shown as a single bunch of grapes (in other words, one or more fruits, hereinafter sometimes simply referred to as "fruit") on a grape vine grown in an orchard. In FIG. 2, the single bunch of grapes that is the object 101 is indicated by a solid line, and other parts (branches and leaves) are indicated by two-dot chain lines.

[0037] 2 also shows a case where the object 101 is surrounded by a light blocking body 25 that blocks light (external disturbances such as sunlight and lighting) directed toward the object 101 from outside the object 101. The light blocking body 25 preferably blocks infrared light, visible light, and ultraviolet light. In FIG. 2, the light blocking body 25 blocks external disturbances such as sunlight and lighting, but in an environment where such external disturbances do not exist (for example, at night when the sun has set or indoors with the lights turned off), the object 101 does not need to be surrounded by the light blocking body 25.

[0038] In FIG. 2, the imaging device 21 captures images of the 360° surroundings of the object 101 (a bunch of fruit), including arrows A1 to A6, using reflected light of wavelengths L in each band. Each of the arrows A1 to A6 shown in FIG. 2 indicates a direction from the surroundings of the object 101 (a bunch of fruit, each fruit) toward the bunch of fruit. Specifically, arrow A1 in FIG. 2 indicates one vertical direction (first direction, upward), and arrow A2 indicates the other vertical direction (second direction, downward). Arrow A3 in FIG. 2 indicates one horizontal direction (third direction) perpendicular to the vertical direction, and arrow A4 indicates the opposite direction to the third direction (fourth direction). Arrow A5 in FIG. 2 indicates one horizontal direction (fifth direction) perpendicular to the vertical direction and perpendicular to the third and fourth directions, and arrow A6 indicates the opposite direction to the fifth direction (sixth direction).

[0039] Furthermore, in order to capture images of all possible blind spots such as unevenness of the object 101, the imaging device 21 may capture images of the object 101 from different distances in each direction in addition to capturing images of the 360° circumference of the object 101 including the arrows A1 to A6. Therefore, in the example shown in Fig. 2, the imaging device 21 moves around the object 101 in a spiral around a central axis extending in a predetermined direction (the vertical direction in Fig. 2) while capturing images of the entire 360° circumference of the object 101 (R1 in Fig. 2), and then changes the distance from the object 101 to continue capturing images, and further moves in a spiral to capture images of the entire 360° circumference of the object 101 (R2 in Fig. 2).

[0040] In this case, the imaging device 21 may change the band of the reflected light to be captured each time the imaging device 21 moves around the object 101 and captures an image of the periphery of the object 101, or may repeatedly change the band of the reflected light to be captured while moving around the object 101. When the imaging device 21 changes the band of the reflected light to be captured each time the imaging device 21 moves around the object 101 and captures an image of the periphery of the object 101, the irradiating unit 23 changes the band of the wavelength L of the measurement light each time the imaging device 21 completes capturing an image of the periphery of the object 101. When the imaging device 21 repeatedly changes the band of the reflected light to be captured while moving around the object 101, the irradiating unit 23 changes the band of the wavelength L of the measurement light each time the imaging device 21 captures a predetermined number of images (for example, one image) of the object 101, thereby repeatedly changing the band of the reflected light to be captured while moving around the object 101. In such a case, the irradiating unit 23 may repeatedly change the band of the reflected light at predetermined elapsed times.

[0041] 2, for convenience of explanation, arrows A1 to A6 are used to indicate the directions in which the imaging device 21 captures an image of the object 101, but it goes without saying that it is preferable for the imaging device 21 to capture an image of the object 101 from other directions (for example, the direction between arrows A1 and A3, the direction between arrows A1 and A6, etc.). Also, it is preferable for the imaging device 21 to capture an image of the 360° periphery of the object 101 (fruit), but it is not necessary to capture an image of the entire 360° periphery. Also, in FIG. 2, a single imaging device 21 is moved to capture an image of the periphery of the object 101, but multiple imaging devices 21 may surround the periphery of the object 101 and capture an image of the periphery of the object 101.

[0042] As described above, the imaging device 21 receives the compound information of the object 101 from the information processing device 41 once. The captured images 51 required for the determination process (hereinafter sometimes referred to as the determination process) are captured in 360° around the object 101 (fruit), including the arrows A1 to A6, using reflected light of wavelength L in each band, so multiple images 51 (for example, at least 50 images) of the object 101 (fruit) are captured for each reflected light of wavelength L in each band.

[0043] The imaging of the object 101 by the imaging device 21 may be performed entirely manually by the farm worker, or the first control device 11 may appropriately assist the farm worker in taking images, or may perform the images automatically.

[0044] One or more captured images 51 captured by the imaging device 21 are transmitted to the server device 40 by the first communication device 13. Specifically, data (imaging data) of the captured image 51 captured by the imaging device 21 is associated with information (band information) relating to the band of wavelength L of reflected light when the captured image 51 is captured. In this embodiment, when the imaging device 21 captures the captured image 51, the first control device 11 associates the band information with the data (imaging data) of the captured image 51, and stores the data as processing data in the first storage unit 12. The first control device 11 also controls the first communication device 13 to transmit the processing data stored in the first storage unit 12 to the server device 40.

[0045] In addition to the band information, the data of the captured image 51 captured by the imaging device 21 (imaging data) may be associated with information that can be used to determine compound information of the object 101 in the information processing device 41. For example, the input interface may accept input of information about the object 101 (object information), and the first control device 11 may associate the object information with the imaging data. Examples of the object information include the type of cultivated plant 100 and a part of the object 101 (for example, fruit, leaf, stem, etc.).

[0046] The object information may also be set without being input by the farmer himself. For example, a tag on which an image code storing information for determining the type of cultivated plant 100 is printed may be attached to the cultivated plant 100, and the first control device 11 may acquire the object information (type of cultivated plant 100) by reading the image code with the imaging device 21. The object information may also include the date and time when the captured image 51 of the object 101 was captured, or, if the imaging device 21 (or the support terminal 10 on which the imaging device 21 is provided) is provided with a position detection device 14 that detects its own position using satellite signals or the like, the object information may also include position information (latitude and longitude) of the position where the captured image 51 was captured. The example shown in FIG. 1 illustrates a case where the position detection device 14 is provided in the support terminal 10.

[0047] Furthermore, if the imaging device 21 (or the support terminal 10 on which the imaging device 21 is provided) has an inertial measurement unit (IMU) 24, camera parameters (external parameters) may be associated with the imaging data. The inertial measurement unit 24 has an acceleration sensor that detects acceleration, a gyro sensor that detects angular velocity, and the like. In the example shown in FIG. 1, the imaging device 21 is provided with the inertial measurement unit 24.

[0048] As shown in FIG. 1 , the support terminal 10 has a display device 31. The display device 31 can display information (quality information) based on compound information determined by the information processing device 41. In addition to the quality information, the display device 31 may also display an image 51 captured by the imaging device 21 or display setting information of the support terminal 10. The information display device 31 is, for example, a liquid crystal display panel, an organic EL panel, or the like. The display device 31 is connected to the first control device 11 and can perform screen display based on a control signal from the first control device 11.

[0049] In this embodiment, the display device 31 has a display screen such as a touch display that can be operated for input, and outputs input information (such as an instruction to start capturing an image of the object 101 by the imaging device 21, an instruction to transmit the captured image data, etc.) to the first control device 11. In this way, the display device 31 may serve both as an output interface that outputs information and as an input interface that accepts input of information. Therefore, the farm worker can operate the display device 31 to capture images by the imaging device 21 and to operate the settings of the support terminal 10. Note that if the display device 31 does not also serve as an input interface, the support terminal 10 may have a physical switch or the like that accepts separate operations. That's fine.

[0050] Next, the information processing device 41 will be described using as an example a fixed terminal device 40 (server device) as shown in Fig. 1. The server device 40 is installed, for example, at an agricultural machinery manufacturer, an agricultural cooperative, a management company, or the like.

[0051] As shown in FIG. 1, the server device 40 has an information processing device 41 (second control device). The information processing device 41 includes one or more memories 42 (storage devices), various analog circuits, various digital circuits, etc. The one or more storage devices 42 store (store) software programs and various data to be executed by one or more processors. Like the first control device 11, the information processing device 41 can also read software programs from one or more storage devices 42 using one or more processors and execute various processes based on the software programs. Also, like the first control device 11, the information processing device 41 can also execute various processes based on predetermined logic circuits using one or more processors, and detailed description thereof will be omitted.

[0052] 1, the server device 40 has a second storage unit 48. The second storage unit 48 is a database configured with an SSD (solid state drive), an HDD (hard disk drive), etc., and stores various information related to the determination of compound information based on optical characteristics. For example, the database 48 can store data (imaging data) of an image 51 captured by the imaging device 21 of the object 101, and determined compound information.

[0053] 1, the server device 40 has a second communication device 49. The second communication device 49 is a device that transmits various data to an external device (e.g., the support terminal 10) of the server device 40 and receives data transmitted from the external device. The second communication device 49 is capable of wireless or wired communication with the first communication device 13. Specifically, the second communication device 49 performs wireless communication with the external device via, for example, Wi-Fi (Wireless Fidelity, registered trademark) of the IEEE802.11 series, which is a communication standard, a mobile phone communication network, a data communication network, or the like.

[0054] For example, the second communication device 49 receives imaging data (processing data) from the support terminal 10 via the first communication device 13. Furthermore, the second communication device 49 transmits information based on the compound information of the object 101 to the support terminal 10 via the first communication device 13. The processing data received by the second communication device 49 from the support terminal 10 via the first communication device 13 is stored in the database 48.

[0055] Hereinafter, the determination of compound information based on the captured image 51 by the information processing device 41 will be described in detail with reference to FIGS. 1, 3, etc. FIG. 3 is a diagram illustrating the flow of data in the agricultural work support system S. As shown in FIGS. 1 and 3, the information processing device 41 includes an image acquisition unit 43, a first determination unit 44, and a second determination unit 45. In addition to these, the information processing device 41 may also include an estimation unit 46. The image acquisition unit 43, the first determination unit 44, the second determination unit 45, and the estimation unit 46 are, for example, software programs stored in a memory or a storage device 42, and are executed by the information processing device 41.

[0056] The image acquiring unit 43 acquires a captured image 51 obtained by capturing an image of the object 101. Specifically, the image acquiring unit 43 acquires a plurality of captured images 51 captured with reflected waves of different wavelengths L. More specifically, the image acquiring unit 43 acquires a plurality of captured images 51 obtained by capturing an image of the object 101 from different directions for each band of reflected light of the wavelength L. As shown in FIG. 3 , the information processing device 41 refers to processing data stored in the database 48 and temporarily stores the processing data in the storage device 42 (memory). The image acquiring unit 43 acquires the plurality of captured images 51 by acquiring the processing data stored in the storage device 42. At this time, the image acquiring unit 43 may shape (convert) the processing data into data that is easier for the first determination unit 44 to process.

[0057] The first determination unit 44 determines the light characteristics at a predetermined position coordinate Cn of the object 101 based on the captured image 51 acquired by the image acquisition unit 43. Specifically, the first determination unit 44 determines the light characteristics at a plurality of position coordinates Cn based on the captured image 51. At this time, the plurality of position coordinates Cn are three-dimensional position coordinates Cn of the object 101. Furthermore, the position coordinates Cn are at least position coordinates Cn inside the object 101. Cn. The processing data includes a plurality of captured images 51 captured with reflected waves of different wavelengths L, so the first determination unit 44 can determine the spectral data at the position coordinate Cn based on the plurality of captured images 51. The first determination unit 44 will be described in detail below.

[0058] The first determination unit 44 determines the optical characteristics from the captured image 51 acquired by the image acquisition unit 43 using a ray tracing method. An example of the ray tracing method is ray tracing. The first determination unit 44 performs ray tracing on each pixel of each captured image 51 acquired by the image acquisition unit 43. FIG. 4 is a diagram for explaining ray tracing. The first determination unit 44 defines the trajectory of a ray from a predetermined reference P (observation point) toward each pixel (x, y). After defining the trajectory of the ray, the first determination unit 44 integrates (adds) the optical characteristics of each position (position coordinate Cn) through which the ray passes, to calculate the optical characteristics of each pixel. Specifically, the first determination unit 44 performs ray tracing on each pixel using the following equation (1).

[0059] TIFF2026005023000002.tif22144 where x is the x coordinate of the pixel in the captured image 51 y: y coordinate of the pixel in the captured image 51 θ: Angle of the light ray relative to the captured image 51 In: Light characteristics at each position (position coordinate Cn) σn: Transmittance at each position (position coordinate Cn)

[0060] The first determination unit 44 calculates the light characteristics and transmittance at each position coordinate Cn from all the light characteristics calculated using simultaneous equations, thereby enabling the first determination unit 44 to determine the relationship between each position coordinate Cn on the object 101 and the light characteristics (see FIG. 5).

[0061] More specifically, when calculating the optical characteristics and transmittance at each position coordinate Cn by ray tracing, the first determination unit 44 extracts multiple captured images 51 for each wavelength L of the reflected light at the time of capture from the processing data (multiple captured images 51) acquired by the image acquisition unit 43, and reconstructs (restores) a three-dimensional space in which the captured images 51 are captured for each wavelength L. That is, when the processing data acquired by the image acquisition unit 43 includes multiple captured images 51 captured with near-infrared light and multiple captured images 51 captured with mid-infrared light, the first determination unit 44 reconstructs a three-dimensional space of the multiple captured images 51 captured with near-infrared light and a three-dimensional space of the multiple captured images 51 captured with mid-infrared light, respectively. The first determination unit 44 reconstructs the three-dimensional space of the multiple captured images 51 using a three-dimensional reconstruction method such as Neural Radiance Fields (NeRF) or Gaussian Splatting from the multiple captured images 51 extracted for each wavelength L.

[0062] Note that the first determination unit 44 may reconstruct the three-dimensional space by any method other than NeRF or Gaussian Splatting, as long as it can reconstruct the three-dimensional space of the multiple captured images 51 extracted for each wavelength L. In this embodiment, the first determination unit 44 reconstructs the three-dimensional space of the multiple captured images 51 extracted for each wavelength L using NeRF. At this time, the first determination unit 44 reconstructs the three-dimensional space using NeRF based on camera parameters included in the processing data and associated with each captured image 51. Note that if the processing data does not include the camera parameters, the camera parameters may be estimated using SFM (Structure from Motion) such as COLMAP.

[0063] This allows the first determination unit 44 to determine the optical characteristics of the position coordinates Cn that cannot be observed in the captured image 51 acquired by the image acquisition unit 43, i.e., the position coordinates Cn inside the object 101. As described above, the first determination unit 44 performs three-dimensional reconstruction for each reflected light of wavelength L in each band, and can determine the optical characteristics of each reflected light of wavelength L in each band at each position coordinate Cn inside the object 101 from the reconstructed three-dimensional space. The first determination unit 44 can determine the distribution of optical characteristics for each reflected light of each band of wavelength L in the object 101. In other words, the first determination unit 44 can determine the spectral data for each position coordinate Cn inside the object 101.

[0064] The second determination unit 45 determines compound information about a compound at a position coordinate Cn based on the optical characteristics at the position coordinate Cn determined by the first determination unit 44 and predetermined determination information. The second determination unit 45 determines compound information at each position coordinate Cn based on the optical characteristics at each of the multiple position coordinates Cn determined by the first determination unit 44 and the determination information. In this embodiment, the first determination unit 44 determines spectral data for each position coordinate Cn, and therefore the second determination unit 45 determines compound information at the position coordinate Cn based on the spectral data and the determination information. The second determination unit 45 determines the amount of the compound at the position coordinate Cn as the compound information. Note that the compound information may be information about at least a compound, and the amount of the compound may be the amount of a single compound or the amount of a predetermined mixture containing multiple compounds.

[0065] The determination information indicates the relationship between the optical characteristics and the compound. In this embodiment, the determination information is stored in the storage device 42. The determination information is a calibration formula (calibration curve) that indicates the relationship between the optical characteristics of the wavelength L of each band and the amount of the compound (see FIG. 6). FIG. 6 is a diagram showing the relationship between the optical characteristics and the amount of a predetermined compound. FIG. 6 shows a calibration formula in which the amount of the compound increases (approximately proportional relationship) as the value of the optical characteristic increases. Note that the calibration formula is defined for each single compound or for each predetermined mixture consisting of multiple compounds, and for each optical characteristic of the wavelength L of one or multiple bands.

[0066] The second determination unit 45 determines the amount of a compound based on the optical characteristics (spectral data) at each position coordinate Cn determined by the first determination unit 44 and the calibration formula stored in the storage device 42. The second determination unit 45 determines the amount of each compound for each position coordinate Cn using the calibration formula corresponding to the wavelength L of each band. This allows the second determination unit 45 to determine the amount of a compound contained in each position coordinate Cn of the object 101, i.e., the composition of the compound (see FIGS. 7A and 7B). FIGS. 7A and 7B are diagrams schematically illustrating the relationship between predetermined position coordinates C1 and C2 of the object 101 and the amount of a predetermined compound. FIG. 7A shows the amount of a compound at the first position coordinate C1, and FIG. 7B shows the amount of a compound at the second position coordinate C2.

[0067] In this embodiment, each calibration formula is stored in the second storage unit 48 (database) in association with each compound. The second determination unit 45 may refer to (acquire) an appropriate calibration formula to determine compound information for the target object 101 based on the target object information included in the processing data, and temporarily store the calibration formula in the storage unit 42 (memory). Furthermore, each calibration formula (determination information) may be stored externally (e.g., an external server device or storage medium capable of wireless or wired communication with the information processing device 41) other than the database 48. In such a case, a device that receives information used by the information processing device 41 for information processing, such as the second communication device 49 of the server device 40, acquires (receives) each calibration formula (determination information) from the outside. For this reason, in this embodiment, the second communication device 49 also serves as an acquisition device for acquiring determination information from the outside, but this is not limited thereto. That is, the information processing device 41 (second determination unit 45) may acquire determination information from the outside via an acquisition device other than the second communication device 49.

[0068] For example, if the object 101 is a grape, the second determination unit 45 acquires calibration formulas corresponding to the moisture, sugar content, acidity, pH, polyphenols, etc. from the second storage unit 48 and stores them in the storage unit 42 in order to determine these, for example. Also, if the object 101 is a tomato, the second determination unit 45 acquires calibration formulas corresponding to the moisture, sugar content, acidity, pH, lycopene, etc. from the second storage unit 48 and stores them in the storage unit 42 in order to determine these, for example.

[0069] As described above, the second determination unit 45 can determine the composition of compounds at each position coordinate Cn by determining the amount of the compound using the calibration formula for each position coordinate Cn and for each optical characteristic of the wavelength L in each band (see FIG. 8). This allows the second determination unit 45 to determine the distribution of each compound in the target object 101.

[0070] The estimation unit 46 estimates the structure of the object 101 based on the compound information for each of the multiple position coordinates Cn determined by the second determination unit 45. When the object 101 includes a fruit of the cultivated plant 100, the estimation unit 46 estimates at least one of the peel, flesh, and seeds of the fruit as the fruit structure. FIG. 9 is a diagram showing an example of the structure of a compound at a predetermined position coordinate Cn of the object 101 and an example of the structure of a compound having a predetermined structure. The left diagram of FIG. 9 shows the compound information at the predetermined position coordinate Cn determined by the second determination unit 45, and the right diagram of FIG. 9 shows an example of the structure of a compound in the predetermined structure of the object 101.

[0071] The second storage unit 48 stores a first table relating each structure and the amount of compounds contained in the structure. The first table is defined for each object 101, for example. If the object 101 is a fruit, the first table defines the peel, pulp, seeds, etc. in association with the amounts of compounds that make them up. The estimation unit 46 may refer to (acquire) the first table corresponding to the object 101 from the database 48 based on the object information included in the processing data acquired by image acquisition, and temporarily store the first table in the storage unit 42 (memory).

[0072] The estimation unit 46 refers to the first table and estimates which structure each position coordinate Cn corresponds to based on the compounds contained in each structure in the first table and the amounts of those compounds. For example, the estimation unit 46 compares (matches) the compounds contained in each structure in the first table and the amounts of those compounds with the amounts of those compounds at each position coordinate Cn, and estimates the structure at the position coordinate Cn to be the one with a relatively consistent amount of compounds.

[0073] Specifically, the estimation unit 46 determines whether the amount of each compound at each position coordinate Cn is within a predetermined range, based on the amount of the compound contained in each structure defined in the first table, and estimates the structure of each position coordinate Cn. For example, the estimation unit 46 determines whether the difference (Δdm, m = 1, 2, 3, ...) between the amount of each compound at each position coordinate Cn and the reference amount of the compound is less than a predetermined threshold. In this way, the estimation unit 46 can estimate the structure of the object 101 by estimating the structure of each position coordinate Cn (see FIG. 10).

[0074] 10, the estimation unit 46 can estimate each structure with its position coordinates Cn, and can therefore estimate the position coordinates Cn of the feature points of each structure. Therefore, the estimation unit 46 can also estimate the size of each structure (volume or proportion to the total volume) based on the position coordinates Cn of the feature points of each structure and a known arithmetic expression, etc.

[0075] In addition, the information processing device 41 may have AI (Artificial Intelligence) as an estimation unit 46, and the estimation unit 46 may estimate the structure, size, etc. of the target object 101 based on the compound information of each of the multiple position coordinates Cn through machine learning of the AI.

[0076] In the above example, when the object 101 is a fruit, the estimation unit 46 estimates the peel, flesh, and seeds as the respective structures. However, the estimation unit 46 may estimate other structures in more detail. In this case, the estimation unit 46 may estimate, for example, insect damage or internal disease damage of each structure.

[0077] When the estimation unit 46 estimates the structure of the object 101, it can estimate the amount of compounds for each structure. Therefore, if the object 101 is grapes, the estimation unit 46 can determine the moisture, sugar content, acidity, pH, polyphenols, etc. in each of the skin and flesh of the fruit based on the estimated structure and compound information. Furthermore, the estimation unit 46 can estimate the distribution of moisture, sugar content, acidity, pH, polyphenols, etc. in each structure (skin and flesh) of the object 101.

[0078] Furthermore, the estimation unit 46 may be able to estimate the harvest time of the object 101 based on the estimated structure, the size of the structure, compound information, etc. The estimation unit 46 estimates the harvest time by periodically referring to the estimated structure, the size of the structure, compound information, etc., or by referring to past estimation results for the same object 101 in chronological order. For example, the second storage unit 48 stores a second table in which the size of each structure at the harvest time is associated with the amount of a compound contained in each structure. The second table is defined for each object 101, for example, and if the object 101 is a fruit, the second table defines the size of the peel and flesh at the harvest time in association with the amount of a compound contained in the peel and flesh. The estimation unit 46 may refer to (obtain) a second table corresponding to the object 101 from the database 48 based on the object information included in the processing data acquired by the image acquisition, and temporarily store the second table in the storage device 42 (memory).

[0079] The estimation unit 46 refers to the second table and estimates whether the object 101 is ready to be harvested based on the size of each structure and the amount of compound at the harvest time in the second table. For example, the estimation unit 46 determines whether the size of each structure and the amount of compound in the object 101 are within a predetermined range, based on the size of each structure and the amount of compound defined in the second table, and estimates whether the object 101 is ready to be harvested. In this way, the estimation unit 46 can estimate the harvest time of the object 101.

[0080] In addition, when the information processing device 41 has an AI as the estimation unit 46, the estimation unit 46 may estimate the harvest time of the target object 101 by machine learning of the AI.

[0081] The estimation unit 46 stores the estimation results (for example, the structure of the target object 101, compound information of each structure, harvest time, etc.) in the storage device 42. The information processing device 41 stores the estimation results in the database 48 as estimation data in association with the target object information.

[0082] In the above-described embodiment, the information processing device 41 has the estimation unit 46, and the results (estimated data) estimated by the estimation unit 46 are stored in the second storage unit 48. However, in cases where the information processing device 41 does not have the estimation unit 46, for example, instead of or in addition to the estimated data, the compound information determined by the second determination unit 45 may be associated with the object information and stored as determined data in the database 48. In the following description, the estimated data and the determined data will be collectively referred to as result data.

[0083] The information processing device 41 controls the second communication device 49 to transmit information (e.g., result data) based on the compound information of the object 101 to the support terminal 10 via the first communication device 13. When the first communication device 13 receives the result data, the first control device 11 causes the display device 31 to display information (quality information) based on the result data. In other words, the display device 31 displays the quality information based on the compound information determined by the second determination unit 45. The display device 31 displays the quality information on a predetermined support screen M1. The display device 31 displays at least one of information on the moisture content, sugar content, and acidity of the object 101 as the quality information.

[0084] 11 is an example of a support screen M1 displayed by the display device 31. In the example shown in Fig. 11, the support screen M1 has a first information display section 111, a second information display section 112, and a third information display section 113. The first information display section 111 is an area that displays information about the object 101 captured by the imaging device 21. The first information display section 111 displays information based on the object information included in the result data, such as the type of cultivated plant 100, the location of the object 101 (e.g., fruit, leaves, stem, etc.), the date and time when the captured image 51 of the object 101 was captured, location information (latitude and longitude) of the location where the captured image 51 was captured, the field where the cultivated plant 100 is grown, etc.

[0085] The second information display unit 112 is an area that displays information based on the compound information. The second information display unit 112 displays, for example, at least one of information on the moisture content, sugar content, and acidity of the object 101 as the compound information. In the example shown in FIG. 11, the second information display unit 112 displays compound information at each position (position coordinate Cn) along with one captured image 51 showing the object 101. The second information display unit 112 shown in FIG. 11 also displays, as compound information, the moisture content, sugar content, acidity, pH, and polyphenols of a specific fruit in a bunch of grapes. The second information display unit 112 can change the position (position coordinate Cn) at which the compound information is displayed when the display device 31 accepts an operation.

[0086] The third information display unit 113 is an area that displays the estimation results estimated by the estimation unit 46. The third information display unit 113 displays, for example, information such as compound information for each structure as an estimation result. The third information display unit 113 shows, for example, compound information for each peel, pulp, and seed. The third information display unit 113 in Fig. 11 shows the size of these structures and the estimation result of the harvest time of the object 101 instead of the compound information for each structure. In the example shown in Fig. 11, the third information display unit 113 displays the volume of each structure and its proportion to the whole as the size of each structure. The third information display unit 113 displays the estimation result of the harvest time divided into multiple stages (for example, before the harvest time, at the harvest time, past the harvest time, etc.), and also displays the difference between the harvest time and the present (days until the harvest time). It displays the number of items, etc.

[0087] A series of processing steps performed by the agricultural work support system S will be described. FIG. 12 is a diagram illustrating a series of processing steps performed by the agricultural work support system S. That is, at least a part of the series of processing steps shown in FIG. 12 can be considered to be an information processing method according to this embodiment. As shown in FIG. 12, first, the imaging device 21 captures an image 51 of an object 101, which is at least a part of a cultivated plant 100 (S1, first step). Also, in the first step, the imaging device 21 captures a plurality of images 51 of the object 101 using reflected waves of different wavelengths L. Note that, when performing the first step, the farm worker may capture the image of the object 101 by surrounding the object 101 with a light shield 25, or may capture the image of the object 101 at night when the sun has set, in order to capture the image in an environment that blocks light (external disturbances such as sunlight and lighting) that may interfere with the imaging of the object 101 by the imaging device 21.

[0088] To explain the first step in more detail, the farm worker operates the input interface (display device 31) to input information (object information) about the object 101 (S1a). After inputting the object information, the farm worker operates the input interface to start capturing an image of the object 101 with the imaging device 21 (S1b).

[0089] The farm worker moves the imaging device 21 (support terminal 10) around the object 101 and captures images of the periphery of the object 101 from various distances and angles (S1c). At this time, the first control device 11 appropriately changes the band of wavelength L of the measurement light irradiated by the irradiation unit 23, so that the imaging device 21 captures multiple captured images 51 of the object 101 with reflected waves of different wavelengths L.

[0090] Therefore, in the processing of step S1c, the imaging device 21 captures images of the surroundings of the target object 101 multiple times for each wavelength L in a different band. At this time, if the imaging device 21 (or the support terminal 10 on which the imaging device 21 is provided) has an inertial measurement unit (IMU) 24, the first control device 11 may control the display device 31 to display information such as the direction in which the imaging device 21 should be moved and whether the required number of captured images 51 for the determination process is sufficient. The first control device 11 determines that the capturing of the captured images 51 is complete when the farmworker operates the input interface or when it determines that the required number of captured images 51 for the determination process is sufficient (S1d: Yes). On the other hand, if the capturing of the captured images 51 is not complete (S1d: No), the process returns to step S1c.

[0091] When the imaging device 21 completes capturing the captured image 51 of the object 101 (S1d: Yes), the one or more captured images 51 captured by the imaging device 21 are transmitted to the server device 40 (second communication device 49) by the first communication device 13 (S2). Specifically, the first control device 11 associates other information (bandwidth information, object information, etc.) with the data (imaging data) of the captured image 51 captured by the imaging device 21, and stores the data in the first storage unit 12 as data to be processed (S2a). The first control device 11 also controls the first communication device 13 to transmit the data to be processed stored in the first storage unit 12 to the server device 40 (S2b).

[0092] When the server device 40 (second communication device 49) receives processing data (one or more captured images 51) from the first communication device 13 (S3: Yes), the image acquisition unit 43 acquires the captured images 51 (S4). Specifically, the image acquisition unit 43 acquires multiple captured images 51 captured with reflected waves of different wavelengths L based on the processing data received by the second communication device 49. More specifically, the image acquisition unit 43 acquires the processing data stored in the storage device 42, and acquires multiple captured images 51 captured from different directions of the object 101 for each band of reflected light of wavelength L.

[0093] The first determination unit 44 determines light characteristics at a predetermined position coordinate Cn of the object 101 based on the captured image 51 captured in the first step (S5, second step). Specifically, in the second step, the first determination unit 44 determines light characteristics at a plurality of position coordinates Cn of the object 101 based on the captured image 51. Furthermore, in the second step, the first determination unit 44 determines spectral data at the position coordinates Cn based on the plurality of captured images 51.

[0094] In detail, the first determination unit 44 extracts a plurality of captured images 51 for each reflected light of wavelength L in each band from the processing data acquired by the image acquisition unit 43 (S5a), and The first determination unit 44 reconstructs (restores) a three-dimensional space of a plurality of captured images 51 captured by light (S5b). In S5a, the first determination unit 44 reconstructs the three-dimensional space by NeRF, for example, based on camera parameters included in the processing data and associated with each captured image 51.

[0095] When the first determination unit 44 reconstructs the three-dimensional space (S5b), it interpolates the plurality of captured images 51 in the three-dimensional space and determines light characteristics from the plurality of captured images 51 using a ray tracing method (such as ray tracing) (S5c). The first determination unit 44 performs ray tracing on the image reconstructed from the three-dimensional space, i.e., on each pixel of each captured image 51 acquired by the image acquisition unit 43, and calculates the light characteristics and transmittance at each position coordinate Cn from all the light characteristics calculated using simultaneous equations.

[0096] The first determination unit 44 repeats the processes of S5a to S5c for each reflected light of wavelength L in each band (S5d: Yes), and determines the optical characteristics of each position coordinate Cn inside the object 101 for each reflected light of wavelength L in each band, in other words, determines the spectral data of each position coordinate Cn inside the object 101 (S5e).

[0097] The second determination unit 45 determines compound information at the position coordinate Cn based on the optical characteristics at the position coordinate Cn determined in the second step (S5) and determination information indicating the relationship between the optical characteristics and the compound (S6, third step). In the third step, the second determination unit 45 determines compound information at the position coordinate Cn based on the spectrum data and the determination information. In addition, in the third step, the second determination unit 45 determines the amount of compound at the position coordinate Cn as compound information for each of the multiple position coordinates Cn based on the optical characteristics and determination information for each position coordinate Cn.

[0098] Specifically, the second determination unit 45 refers to (obtains) an appropriate calibration formula for determining compound information of the object 101 based on the object information included in the processing data (S6a). The second determination unit 45 determines compound information of each position coordinate Cn (amount of compound at each position coordinate Cn) based on the calibration formula and the optical characteristics (spectral data) at each position coordinate Cn determined by the first determination unit 44 in the second step (S5) (S6b).

[0099] The second determination unit 45 repeats the processes of S6a to S6b for each position coordinate Cn (S6c: Yes), and determines compound information (the amount of compound contained in each position coordinate Cn, in other words, the composition of the compound) for each position coordinate Cn of the object 101 (S6d).

[0100] When the second estimation unit 46 determines the compound information at all the position coordinates Cn (S6, third step), the estimation unit 46 estimates the structure of the object 101 based on the compound information at each of the multiple position coordinates Cn determined in the third step (S6) (S7, fourth step). For example, the estimation unit 46 refers to (obtains) the first table and the second table corresponding to the object 101 from the database 48 based on the object information included in the processing data acquired by image acquisition (S7a).

[0101] The estimation unit 46 refers to the first table and estimates which structure each position coordinate Cn corresponds to and its size (volume) based on the compounds contained in each structure in the first table and the amount of the compounds (S7b).The estimation unit 46 also refers to the second table and estimates whether the target object 101 is at harvest time based on the size of each structure and the amount of compounds at harvest time in the second table (S7c).

[0102] In addition, if the information processing device 41 has an AI as the estimation unit 46, the estimation unit 46 may estimate the structure, size of the structure, harvest time, etc. of the target object 101 by machine learning of the AI, without relying on the processing of S7a to S7c.

[0103] After performing estimation through the processes of S7a to S7c, the estimation unit 46 stores the estimated results (e.g., the structure of the object 101, compound information for each structure, and harvest time) in the memory device 42, and the information processing device 41 associates the estimated results with the object information and stores them in the database 48 as estimated data (result data) (S7d).

[0104] When the estimation unit 46 stores the estimated data in the database 48 (S7d), the information processing device 41 controls the second communication device 49 to transmit information (e.g., result data) based on the compound information of the object 101 to the support terminal 10 via the first communication device 13 (S8). When the first communication device 13 receives the result data, the first control device 11 causes the display device 31 to display information (quality information) based on the result data (S9). As a result, as shown in Fig. 12, the display device 31 displays information based on the compound information determined by the second determination unit 45 (information on at least one of the moisture content, sugar content, and acidity of the target object 101).

[0105] 12 is merely an example, and is not intended to be limiting. For example, when the imaging device 21 captures images of the surroundings of the object 101 for each reflected light of different wavelengths L, the first control device 11 may transmit processing data for each reflected light of each wavelength L to the server device 40. In such a case, after processing step S1c, the agricultural work support system S performs processing step S2 and then proceeds to processing step S1d.

[0106] Furthermore, when the imaging device 21 captures an image of the surroundings of the object 101 for each reflected light of a different band of wavelength L, and processing data is transmitted to the server device 40 for each reflected light of each band of wavelength L, the image acquisition unit 43 may perform the process of S5 for each reflected light of each band of wavelength L for which the processing data is received in step S4. In such a case, the agricultural work support system S repeats the processes of steps S1 to S5 for each reflected light of each band of wavelength L, and then proceeds to the process of step S6.

[0107] Furthermore, in the above-described embodiment, a case has been described in which a farm worker images the object 101 by surrounding it with a light shield 25 in order to perform imaging in an environment that blocks light that may hinder the imaging device 21 from capturing the image of the object 101, or in a time period such as at night when the sun has set. However, if the database 48 stores determination information (calibration formula) that takes the disturbance into account, the imaging device 21 may also capture the image of the object 101 in an environment where the disturbance is present.

[0108] A preferred embodiment of the present invention provides an information processing device 41, an agricultural work support system S, and an information processing method described in the following items. (Item 1) An information processing device 41 includes an image acquisition unit 43 that acquires an image 51 of an object 101 that is at least a part of a cultivated plant 100; a first determination unit 44 that determines light characteristics at a predetermined position coordinate Cn of the object 101 based on the image 51 acquired by the image acquisition unit 43; and a second determination unit 45 that determines compound information related to the compound at the position coordinate Cn based on the light characteristics at the position coordinate Cn determined by the first determination unit 44 and predetermined judgment information that indicates the relationship between the light characteristics and the compound.

[0109] According to the information processing device 41 relating to this item 1, compound information at a predetermined position coordinate Cn of the object 101 can be easily and non-destructively determined simply by using a captured image 51 of the object 101. Therefore, the information processing device 41 can determine the compound information of the object 101 with high accuracy. In particular, when determining the compound information, it is sufficient for the farmer to prepare the captured image 51 of the object 101, so that it is not necessary for the farmer to harvest the cultivated plant 100 that is the object 101, which is highly convenient. (Item 2) Item 1. The information processing device 41 according to item 1, wherein the second determination unit 45 determines an amount of the compound at the position coordinate Cn as the compound information.

[0110] According to the information processing device 41 relating to this item 2, the amount of a compound at a predetermined position coordinate Cn of the object 101 can be easily and non-destructively determined simply by using a captured image 51 of the object 101. (Item 3) The information processing device 41 described in item 2, wherein the first determination unit 44 determines the light characteristics at the plurality of position coordinates Cn of the object 101 based on the captured image 51 acquired by the image acquisition unit 43, and the second determination unit 45 determines the compound information at each position coordinate Cn based on the light characteristics at each of the plurality of position coordinates Cn determined by the first determination unit 44 and the judgment information.

[0111] According to the information processing device 41 relating to this item 3, the captured image 51 of the object 101 is By simply using this, compound information can be determined for a plurality of position coordinates Cn (in other words, a plurality of different portions) of the object 101. This makes it possible to easily determine the distribution of compounds contained in the object 101. (Item 4) The information processing device 41 according to item 3, further comprising an estimation unit 46 that estimates the structure of the object 101 based on the compound information of each of the plurality of position coordinates Cn determined by the second determination unit 45.

[0112] According to the information processing device 41 relating to this item 4, the structure of the object 101 can be estimated from the distribution of compounds contained in the object 101, and the usefulness of the determined compound information can be improved. (Item 5) The information processing device 41 described in item 4, wherein when the target object 101 includes a fruit of the cultivated plant 100, the estimation unit 46 estimates at least one of the peel, flesh, and seeds of the fruit as the structure of the fruit.

[0113] According to the information processing device 41 of this item 5, at least one of the peel, pulp, and seeds of a fruit can be easily and non-destructively determined as the structure of the object 101 simply by using the captured image 51 of the object 101. This further improves the usefulness of the determined compound information. (Item 6) An information processing device 41 described in any one of items 2 to 5, wherein the first determination unit 44 determines the light characteristics at the three-dimensional position coordinates Cn of the object 101 as the position coordinates Cn based on the captured image 51 acquired by the image acquisition unit 43.

[0114] According to the information processing device 41 relating to this item 6, compound information at a predetermined three-dimensional position coordinate Cn of the object 101 can be easily determined non-destructively by simply using a captured image 51 of the object 101. (Item 7) An information processing device 41 described in any one of items 2 to 6, wherein the first determination unit 44 determines the light characteristics of the object 101 at least at a position coordinate Cn inside the object 101 based on the captured image 51 acquired by the image acquisition unit 43.

[0115] According to the information processing device 41 of this item 7, compound information at the position coordinate Cn inside the object 101 can be easily determined non-destructively by simply using the captured image 51 obtained by capturing an image of the object 101. (Item 8) The information processing device 41 according to any one of items 2 to 7, wherein the image acquisition unit 43 acquires a plurality of captured images 51 captured with reflected waves of different wavelengths L, the first determination unit 44 determines spectral data at the position coordinate Cn based on the plurality of captured images 51 acquired by the image acquisition unit 43, and the second determination unit 45 determines the compound information at the position coordinate Cn based on the spectral data at the position coordinate Cn determined by the first determination unit 44 and the judgment information.

[0116] According to the information processing device 41 relating to this item 8, spectral data at the position coordinate Cn inside the object 101 can be determined simply by using the captured image 51 of the object 101, and compound information of the object 101 can be determined with higher accuracy. (Item 9) The information processing device 41 according to any one of items 1 to 8, wherein the first determination unit 44 determines the light characteristics from the captured image 51 acquired by the image acquisition unit 43 using a ray tracing method.

[0117] According to the information processing device 41 of this item 9, the first determination unit 44 can determine relatively realistic (photorealistic) light characteristics and transmittance by the ray tracing method. Therefore, the second determination unit 45 can determine the compound information of the object 101 with higher accuracy, regardless of the directionality of the reflected light. (Item 10) 10. The information processing device 41 according to item 9, wherein the first determination unit 44 uses ray tracing as the ray tracing method.

[0118] According to the information processing device 41 of this item 10, the first determination unit 44 can determine more realistic (photorealistic) light characteristics and transmittance by ray tracing. Therefore, the second determination unit 45 can determine compound information of the object 101 with higher accuracy. (Item 11) An agricultural work support system S comprising an imaging device 21 that captures the captured image 51 of the object 101, an information processing device 41 described in any one of items 1 to 10, and a display device 31 that displays information based on the compound information determined by the second determination unit 45.

[0119] According to the agricultural work support system S relating to this item 11, the agricultural worker can easily recognize the compound information determined by the information processing device 41 that has the unique effects described above. (Item 12) Item 12. The agricultural work support system S according to item 11, wherein the display device 31 displays at least one of information on the moisture content, sugar content, and acidity of the object 101 as information based on the compound information.

[0120] According to the agricultural work support system S relating to item 12, the agricultural worker can clearly recognize at least one of the moisture content, sugar content, and acidity of the object 101 simply by preparing the captured image 51 of the object 101 captured by the imaging device 21. (Item 13) An information processing method comprising: a first step in which an imaging device 21 captures an image 51 of an object 101, which is at least a part of a cultivated plant 100; a second step in which a first determination unit 44 determines light characteristics at a predetermined position coordinate Cn of the object 101 based on the image 51 captured in the first step; and a third step in which a second determination unit 45 determines compound information related to the compound at the position coordinate Cn based on the light characteristics at the position coordinate Cn determined in the second step and judgment information indicating the relationship between the light characteristics and a compound.

[0121] According to the information processing method of this item 13, compound information at a predetermined position coordinate Cn of the object 101 can be easily and non-destructively determined simply by using a captured image 51 of the object 101. Therefore, the information processing device 41 can determine the compound information of the object 101 with high accuracy. In particular, when determining the compound information, it is sufficient for the farmer to prepare the captured image 51 of the object 101, so that the farmer does not necessarily have to harvest the cultivated plant 100 that is the object 101, which is highly convenient. (Item 14) Item 14. The information processing method according to Item 13, wherein in the first step, the imaging device 21 captures a plurality of the captured images 51 of the object 101 using reflected waves of different wavelengths L, in the second step, the first determination unit 44 determines spectral data at the position coordinate Cn based on the plurality of the captured images 51, and in the third step, the second determination unit 45 determines the compound information at the position coordinate Cn based on the spectral data and the determination information.

[0122] According to the information processing method of this item 14, spectral data at the position coordinate Cn inside the object 101 can be determined simply by using the captured image 51 of the object 101, and compound information of the object 101 can be determined with higher accuracy. (Item 15) In the second step, the first determination unit 44 determines the light characteristics at the plurality of position coordinates Cn of the object 101 based on the captured image 51, and in the third step, the second determination unit 45 determines the amount of the compound at the position coordinate Cn as the compound information at each of the plurality of position coordinates Cn based on the light characteristics of each position coordinate Cn and the determination information, and an estimation unit 46 estimates the amount of the compound at the object 101 based on the compound information at each of the plurality of position coordinates Cn determined in the third step. Item 14. The information processing method according to item 13, further comprising a fourth step of estimating a structure.

[0123] According to the information processing method of this item 15, the structure of the object 101 can be estimated from the distribution of compounds contained in the object 101, and the usefulness of the determined compound information can be improved.

[0124] Although the present invention has been described above, the embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0125] 21: Imaging device 31:Display device 41: Information processing device (second control device) 42: Storage device (memory) 43: Image acquisition unit 44: First decision section 45: Second decision section 46: Estimation part 51: Captured image 100: Cultivated plants 101: Object Cn:Position coordinate S: Support System

Claims

1. an image acquisition unit that acquires a captured image of an object that is at least a part of a cultivated plant; a first determination unit that determines light characteristics of the object at a predetermined position coordinate based on the captured image acquired by the image acquisition unit; a second determination unit that determines compound information related to the compound at the position coordinates determined by the first determination unit based on the light characteristics at the position coordinates and predetermined determination information indicating a relationship between the light characteristics and the compound; An information processing device comprising:

2. The information processing device according to claim 1 , wherein the second determination unit determines, as the compound information, an amount of the compound at the position coordinates.

3. The first determination unit determines the light characteristics of the object at a plurality of the position coordinates based on the captured image acquired by the image acquisition unit; The information processing device according to claim 2 , wherein the second determination unit determines the compound information at each of the plurality of position coordinates based on the light characteristics at each of the plurality of position coordinates determined by the first determination unit and the determination information.

4. The information processing apparatus according to claim 3 , further comprising an estimation unit that estimates a structure of the object based on the compound information for each of the plurality of position coordinates determined by the second determination unit.

5. The information processing device according to claim 4 , wherein, when the target object includes a fruit of the cultivated plant, the estimation unit estimates at least one of a peel, a flesh, and a seed of the fruit as the structure of the fruit.

6. The information processing apparatus according to claim 2 , wherein the first determination unit determines the light characteristics at three-dimensional position coordinates of the object as the position coordinates based on the captured image acquired by the image acquisition unit.

7. The information processing device according to claim 2 , wherein the first determination unit determines the light characteristics of at least an internal position coordinate of the object based on the captured image acquired by the image acquisition unit.

8. the image acquisition unit acquires a plurality of captured images captured using reflected waves of different wavelengths, the first determination unit determines spectral data at the position coordinates based on the plurality of captured images acquired by the image acquisition unit; The information processing device according to claim 2 , wherein the second determination unit determines the compound information at the position coordinates determined by the first determination unit based on the spectrum data at the position coordinates and the determination information.

9. The information processing apparatus according to claim 1 , wherein the first determination unit determines the light characteristics from the captured image acquired by the image acquisition unit using a ray tracing method.

10. The information processing device according to claim 9 , wherein the first determination unit uses ray tracing as the ray tracing method.

11. an imaging device that captures the image of the object; An information processing device according to any one of claims 1 to 10; a display device that displays information based on the compound information determined by the second determination unit; and A farming support system that includes:

12. The agricultural work support system according to claim 11 , wherein the display device displays at least one of information on the moisture content, sugar content, and acidity of the object as information based on the compound information.

13. a first step of capturing an image of an object, the object being at least a part of a cultivated plant, using an imaging device; a second step in which a first determination unit determines light characteristics of the object at a predetermined position coordinate based on the captured image captured in the first step; a third step in which a second determination unit determines compound information related to the compound at the position coordinates determined in the second step, based on the light characteristics at the position coordinates determined in the second step and determination information indicating a relationship between the light characteristics and the compound; An information processing method comprising:

14. In the first step, the imaging device captures a plurality of images of the object using reflected waves of different wavelengths, In the second step, the first determination unit determines spectral data at the position coordinates based on the plurality of captured images, The information processing method according to claim 13 , wherein in the third step, the second determination unit determines the compound information at the position coordinates based on the spectrum data and the determination information.

15. In the second step, the first determination unit determines the light characteristics of the object at a plurality of the position coordinates based on the captured image; In the third step, the second determination unit determines an amount of the compound at each of the plurality of position coordinates as the compound information at the position coordinates based on the light characteristics of the position coordinates and the determination information, and 14. The information processing method according to claim 13, further comprising a fourth step in which an estimation unit estimates a structure of the object based on the compound information for each of the plurality of position coordinates determined in the third step.

Images