Image Processing Method

Abstract

To simply measure a state of present brightness in a farm field.SOLUTION: An image processing method comprises: acquiring an imaged image in which a state of a farm field including a cultivation object plant is imaged; determining a type of a color element, to which a pixel belongs, as a binary value of a white color or a black color, on the basis of hue information and lightness information of a pixel constituting the imaged image; and calculating an index indicating brightness in the farm field from a ratio of pixels which are determined to have a white color, of pixels constituting the imaged image.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an image processing method for measuring the brightness of a field. [Background technology]

[0002] In the cultivation of plants such as agricultural crops and fruit trees, it is necessary to manage the environment so as to maintain an appropriate growth environment according to the growth state of the plants in order to improve productivity. For example, Patent Document 1 discloses an environmental control system that controls the environment inside agricultural facilities such as vinyl greenhouses and greenhouses (hereinafter referred to as agricultural greenhouses). Patent Document 1 discloses that the agricultural greenhouses are equipped with adjustment devices for adjusting the internal environment, such as various measuring devices that acquire internal environmental data such as temperature, humidity, sunlight, and carbon dioxide concentration, as well as air conditioners, blowers, temperature and humidity control devices, carbon dioxide generators, and sunshades. It is also disclosed that a control device of the environmental control system controls the adjustment devices based on various data acquired by the measuring devices installed in the agricultural greenhouse, thereby maintaining the internal environment of the agricultural greenhouse at a preset environment.

[0003] In recent years, there has been a growing trend to differentiate agricultural crops and fruit trees produced in a particular region from those of the same type produced in other regions by branding them as specialty products. One example of such a specialty product is the Ruby Roman grape cultivar. By improving the soil and managing moisture in the fields in cultivation facilities and appropriately controlling the amount of sunlight and other brightness during harvest, it has become possible to consistently cultivate large berries with bright red skin. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-87100 Summary of the Invention [Problem to be solved by the invention]

[0005] In order to ensure that the skin of the berries in the above cultivars turns a vivid red at harvest time, it is necessary to maintain the brightness of the field at an appropriate value during the veraison period. This appropriate value is defined, for example, as a certain level of brightness when looking up at the cultivation shelves from the bunches of fruit. However, measuring the brightness of the field during the veraison period requires photographing the appearance of the bunches of fruit and the cultivation shelves with a camera or the like, and then analyzing the camera images on a PC or the like. This has led to the problem that growers of these cultivars are unable to measure the current brightness of the field during the veraison period or at harvest time.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to provide a technology that can easily measure the current brightness state in a farm field. [Means for solving the problem]

[0007] An image processing method according to one aspect of the present disclosure includes acquiring a captured image of the state of a field containing plants to be cultivated, determining the type of color element to which each pixel belongs as a binary value of white or black based on hue information and brightness information of the pixels constituting the captured image, and calculating an index indicating the brightness of the field from the proportion of pixels constituting the captured image that are determined to be white. This makes it possible to easily measure the current brightness state of the field using the captured image of the state of the field containing plants to be cultivated.

[0008] The captured images may also be moving images captured from a viewpoint looking from below toward above the plants to be cultivated. This allows for capturing images appropriate for measuring the current brightness state in the field. The RGB values ​​of the pixels constituting the captured image may also be converted into a color space represented by the hue, saturation, and brightness information of the pixels. This enables binarization processing for measuring the current brightness state in the field based on the pixel information converted into the HSV color space represented by elements such as hue, saturation, and brightness.

[0009] Furthermore, when the brightness information of a pixel constituting the captured image is equal to or greater than a first threshold, the type of color element to which the pixel belongs may be determined to be white. When the brightness information of a pixel constituting the captured image is less than the first threshold, the type of color element to which the pixel belongs may be determined to be black, provided that the hue information of the pixel belongs to a first range. When the brightness information of a pixel constituting the captured image is less than the first threshold, the type of color element to which the pixel belongs may be determined to be white, provided that the hue information of the pixel belongs to a second range. When the brightness information of a pixel constituting the captured image is less than the first threshold and the hue information of the pixel does not belong to either the first range or the second range, the type of color element to which the pixel belongs may be determined to be white, provided that the brightness information of the pixel is equal to or greater than a second threshold. Furthermore, when the lightness information of a pixel constituting the captured image is less than a first threshold and the hue information of the pixel does not belong to either the first range or the second range, the type of color element to which the pixel belongs whose lightness information does not satisfy the condition of being equal to or greater than the second threshold may be determined to be black. This enables binarization processing using at least the hue information and saturation information of pixel information converted into the HSV color space represented by elements such as hue, saturation, and lightness.

[0010] Also, Captured imageare moving images captured at a predetermined frame rate, and the index indicating the brightness of the field may be calculated from a plurality of consecutive frame images. By calculating the index indicating the brightness of the field using a plurality of consecutive frame images, it is possible to calculate the index with a certain degree of accuracy even when, for example, camera shake or tilt occurs.

[0011] Furthermore, an index indicating the brightness of the farm field calculated from the captured image may be notified by voice. This allows the calculated index to be notified by voice to an operator whose line of sight is directed toward tree branches, leaves, cultivation shelves, etc. Furthermore, a voice may be notified to ensure that the imaging camera that captures the captured image maintains the correct posture. This allows an operator whose line of sight is directed toward tree branches, leaves, cultivation shelves, etc., when measuring the index indicating the brightness of the farm field to be aware of the posture of the imaging camera in order to obtain an appropriate captured image. [Effects of the Invention]

[0012] The present disclosure provides a technology that can easily measure the current brightness state in a farm field. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an image processing apparatus according to an embodiment. [Figure 2] 10A and 10B are diagrams illustrating a measurement form of an index value indicating the brightness of a farm field by an image processing device. [Figure 3] FIG. 2 is a diagram showing an example of an image captured by an imaging camera. [Figure 4] FIG. 1 is a diagram illustrating an example of a functional configuration of an image processing apparatus. [Figure 5] 10 is a flowchart showing an example of a process flow for calculating a cavity opening ratio according to the embodiment. [Figure 6] 10 is a flowchart illustrating an example of a flow of a process for notifying the tilt of the imaging camera according to the embodiment. [Figure 7]10A and 10B are diagrams illustrating the correlation between brightness measured by the image processing device according to the embodiment and a conventional image analysis result. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The configurations of the following embodiments are examples, and the present disclosure is not limited to the configurations of the embodiments. Furthermore, the following embodiments can be combined as much as possible.

[0015] First Embodiment FIG. 1 is a diagram illustrating an image processing device 10 according to this embodiment. The image processing device 10 according to this embodiment is a portable information processing device capable of measuring an index indicating the current brightness of a field in a growing environment for cultivated varieties such as agricultural crops and fruit trees. Examples of such information processing devices include a smartphone, a tablet PC, and a laptop PC. As shown in FIG. 1, the image processing device 10 includes an imaging camera 10a and is equipped with an application program (hereinafter simply referred to as an “app”) for measuring the brightness index of a field in which plants such as agricultural crops and fruit trees are grown based on images captured by the camera. A worker Z1 who measures the current brightness index of the field carries the image processing device 10 and performs measurements while pruning tree leaves 21, etc., so that the measured index value indicates a constant brightness. In the following description, the image processing device 10 and the imaging camera 10a are described as being integrally configured; however, the imaging camera 10a may be configured separately from the image processing device 10.

[0016] 1, the image processing device 10 is a computer including, as its components, a processor 101, a main memory unit 102, an auxiliary memory unit 103, an input / output unit 104, and a communication unit 105, all of which are interconnected by a connection bus. The main memory unit 102 and the auxiliary memory unit 103 constitute a memory, and are recording media readable by the image processing device 10. Each of the above components may be provided in multiple units, or some of the components may not be provided at all.

[0017] The processor 101 is a central processing unit that controls the entire image processing device 10. The processor 101 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor). The processor 101 is an example of a control unit. For example, the processor 101 deploys a program stored in the auxiliary storage unit 103 in an executable manner in a working area of ​​the main storage unit 102, and controls peripheral devices through the execution of the program, thereby providing a function that meets a predetermined purpose.

[0018] The main memory unit 102 stores programs executed by the processor 101, data processed by the processor, etc. The main memory unit 102 includes a flash memory, a RAM (Random Access Memory), and a ROM (Read Only Memory). The auxiliary memory unit 103 includes a portable recording medium, a non-volatile semiconductor memory (flash memory, EPROM (Erasable Programmable ROM)), etc. The portable recording medium includes, for example, a silicon disk, a solid state drive device, a hard disk drive (HDD), etc. The auxiliary storage unit 103 is a recording medium such as a card. The auxiliary storage unit 103 stores various programs and various data in a readable and writable manner. The auxiliary storage unit 103 is used as a storage area that supports the main storage unit 102, and stores programs executed by the processor 101, data processed by the processor 101, etc. The auxiliary storage unit 103 stores, for example, an operating system (OS), various programs, various tables, etc. The OS includes a communication interface program that transfers data between devices connected via the communication unit 105. Here, the image processing device 10 may be a single computer or a combination of multiple computers. Furthermore, information stored in the auxiliary storage unit 103 may be stored in the main storage unit 102, and information stored in the main storage unit 102 may be stored in the auxiliary storage unit 103. Note that the series of processes executed by the image processing device 10 As described above, the process is executed by the processor 101 according to a program. However, at least a part of the process can also be executed by hardware such as a digital circuit.

[0019] The input / output unit 104 is an interface for inputting and outputting data to and from devices connected to the image processing device 10. Input devices connected to the input / output unit 104 include, for example, an imaging camera 10a, a keyboard, a pointing device such as a touch panel or a mouse, a sensor 10c, and a microphone. Operation instructions and the like from an operator who operates the input devices are received via the input / output unit 104.

[0020] Furthermore, the input / output unit 104 is connected to output devices such as a display device 10b, such as an LCD (Liquid Crystal Display), an EL (Electroluminescence) panel, or an organic EL panel, and a speaker 10d. The display device 10b is combined with a touch panel to form an input device. Data and information processed by the processor 101 and data and information stored in the main memory unit 102 and the auxiliary memory unit 103 are output via the input / output unit 104. The communication unit 105 is a communication interface with a communication network. The communication unit 105 can have an appropriate configuration depending on the connection method with the communication network. Examples of the communication unit 105 include a LAN (Local Area Network) interface board and a wireless communication circuit for wireless communication.

[0021] The imaging camera 10a is, for example, a CCD (Charge Coupled Device) image sensor or a C Using an imaging element such as a MOS (Complementary Metal Oxide Semiconductor) image sensor The imaging camera 10a is provided, for example, on the side of the opening where the display device 10b of the image processing device 10 is open. The image obtained by capturing may be either a still image or a video at a predetermined frame rate (for example, 30 fps). The sensor 10c is, for example, an acceleration sensor that detects gravitational acceleration. The image processing device 10 detects the inclination of its own device with respect to the vertical direction based on the detection value detected via the sensor 10c.

[0022] FIG. 2 is a diagram illustrating a measurement mode of an index value indicating the brightness of a farm field using an image processing device 10. FIG. 2 illustrates a side view of an image taken using an imaging camera 10a when measuring an index value in a farm field where agricultural crops, fruit trees, etc. are grown. The measurement mode shown in FIG. 2 is an example of a measurement mode in a farm field where grapes 22 are grown. As shown in FIG. 2, in the farm field where grapes 22 are grown, cultivation shelves 20 are provided for tree branches and leaves 21 to grow on, and bunches of grapes 22, which are the fruit to be grown, are covered with hanging bags 22a to prevent damage from pests and diseases and sunburn of the skin and to promote coloring.

[0023] The image processing device 10 according to this embodiment captures an image of the cultivation shelf 20 on which the branches and leaves 21 are growing from above, from below the grapevines 22 on which the hanging bags 22a are hung, via the imaging camera 10a. For example, a worker Z1 measuring a brightness index value in a farm field holds the image processing device 10 in one hand while the imaging camera 10a is in the shooting state, and captures an image of the cultivation shelf 20 on which the branches and leaves 21 are growing from a viewpoint facing upward from below the grapevines 22 on which the hanging bags 22a are hung. Alternatively, while the imaging camera 10a is in the shooting state, the image processing device 10 may be placed on a support stand or the like, and the image of the cultivation shelf 20 on which the branches and leaves 21 are growing from a viewpoint facing upward from below the grapevines 22 on which the hanging bags 22a are hung may be captured. Furthermore, when the imaging camera 10a and the image processing device 10 are configured separately, the image processing device 10 may be placed on a support stand or the like. Then, a worker Z1 carrying an imaging camera 10a may photograph the cultivation rack 20 on which the branches and leaves 21 are spread, from a viewpoint looking upward from below the grapes 22 on which the hanging bags 22a are hung. The photographed moving images are transmitted to the image processing device 10, for example, via a communication network such as wireless.

[0024] For example, if the height h1 of the cultivation shelf 20 is approximately 1.7 m above the ground 23 of the field, the imaging camera 10a is installed at a height h2 of approximately 1.0 m so that the opening of the imaging unit of the imaging camera 10a, i.e., the surface of the display device 10b, is parallel to the ground 23. The image processing device 10 then acquires an image captured by the imaging camera 10a in the shooting state, showing the grapes 22 with hanging bags 22a, the branches and leaves 21, and the state of the cultivation shelf 20 within a viewing angle 24 (the area enclosed by dashed lines). The viewing angle of the imaging camera 10a according to this embodiment is approximately 75°.

[0025] The imaging height of the imaging camera 10a is appropriately set depending on the cultivation target in the field to be measured, the height of the cultivation shelves, the viewing angle 24 of the imaging camera 10a, etc. The image processing device 10 may also notify the operator Z1 that the horizontal angle of the opening of the imaging unit of the imaging camera 10a with respect to the ground 23 is within a predetermined range based on the tilt of the device itself relative to the vertical direction detected via the sensor 10c, etc. For example, if an operator Z1 is measuring the brightness of the field while holding the image processing device 10 in one hand, a voice message such as "The camera is tilted. Please adjust the camera to a horizontal position" can be notified via the speaker 10d. When measuring the brightness of the field, the operator Z1, whose line of sight is directed toward tree branches, leaves 21, or the cultivation shelves 20, can be encouraged to pay attention to the orientation of the imaging camera 10a to obtain appropriate images.

[0026] FIG. 3 is a diagram showing an example of an image captured by the imaging camera 10a. As shown in the captured image Z2 in FIG. 3, the state of the cultivation rack 20, on which branches and leaves 21 are spread, is captured from below the grapevines 22, on which hanging bags 22a are hung. The image processing device 10 according to this embodiment calculates the sky-open ratio as an index indicating the brightness of the farm field, based on the captured image Z2 captured by the imaging camera 10a. Here, the sky-open ratio is, for example, the proportion of an area into which sunlight can be incident without being blocked by leaves 21 when looking up at the sky in a certain area, such as a farm field. The brightness index may be any index other than the sky-open ratio, as long as it has the same meaning as the proportion of an area into which sunlight can be incident without being blocked by leaves when looking up at the sky in a certain area, such as a farm field. It may also be an index obtained by performing a predetermined calculation on the sky-open ratio. In this embodiment, the sky-open ratio represents the proportion of pixel areas corresponding to areas where the sky is reflected in the captured image Z2 captured by the imaging camera 10a. In this embodiment, the openness ratio is an example of an "index indicating brightness in a farm field."

[0027] Specifically, the image processing device 10 acquires video images captured at a predetermined frame rate via the imaging camera 10a. The image processing device 10 then converts the color elements of each pixel in each frame from a color space represented by (R, G, B) to an HSV (Hue, Saturation, Value) color space, which represents elements such as hue, saturation, and brightness. As a result, the RGB values ​​of each pixel constituting the captured image Z2 captured by the imaging camera 10a can be expressed as hue information represented by 360-degree angle information, with 0 degrees representing red and returning to yellow, green, blue, and red. Similarly, saturation, which indicates the vividness of a color, and brightness, which indicates the brightness of a color, can each be expressed as numerical information ranging from 0 to 100 percent. Hereinafter, pixels converted to the HSV color space are also referred to as bright spots.

[0028] The image processing device 10 according to this embodiment calculates the sky openness ratio in the captured image Z2, i.e., the proportion of the pixel area corresponding to the area where the sky is reflected, based on each piece of information (color space information) expressed in HSV. The image processing device 10 determines the color type to which the pixel belongs, for example, based on numerical information indicating brightness. For example, if the brightness of the bright spot to be determined is 90 percent or higher, the image processing device 10 determines that the color type to which the bright spot belongs is white, and if not, determines that the color type is other than white.

[0029] Next, the image processing device 10 calculates a value indicating a hue for the origin determined to be a color type other than white. Based on the angle information, the image processing device 10 classifies whether the bright point to be distinguished belongs to black or white. For example, the image processing device 10 classifies bright points whose hue is in the range of 0 degrees to 177.7 degrees (ranging from red to yellow and blue-green including green) as belonging to black. Also, the image processing device 10 classifies bright points whose hue is in the range of 183.4 degrees to 240.8 degrees (ranging from blue to ultramarine) as belonging to white.

[0030] Furthermore, for bright spots that were identified as not belonging to either black or white using the above-mentioned hue, the image processing device 10 again uses brightness to determine whether the bright spot belongs to white or black. For example, if the brightness of the bright spot to be identified is 2% or higher, the image processing device 10 identifies the bright spot as belonging to the color type of white, and otherwise identifies the bright spot as belonging to the color type of black.

[0031] Through the above processing, the image processing device 10 can obtain a binary image discriminated as white or black for each frame of the moving image captured via the imaging camera 10a. The image processing device 10 can then calculate the percentage of bright points discriminated as belonging to the color type white from the binarized frame images, thereby determining the sky openness ratio in the captured moving image frames, i.e., the percentage of pixel areas corresponding to areas where the sky is reflected. The image processing device 10 according to this embodiment can calculate the sky openness ratio as an index showing the brightness of the field, based on the captured image Z2 captured via the imaging camera 10a.

[0032] (Functional configuration) Next, the functional configuration of the image processing device 10 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram schematically illustrating an example of the functional configuration of the image processing device 10. The image processing device 10 includes, as functional components, a captured image acquisition unit 11, an image processing unit 12, a display processing unit 13, a notification processing unit 14, a threshold information DB 15, and an openness ratio DB 16. A processor 101 of the image processing device 10 executes the processes of the captured image acquisition unit 11, the image processing unit 12, the display processing unit 13, and the notification processing unit 14 using a computer program loaded in a main memory unit 102. However, any one of the functional components, or part of the processes, may be executed by hardware such as a digital circuit.

[0033] The threshold information DB 15 and the void ratio DB 16 are constructed by a database management system (DBMS) program executed by the processor 101 managing data stored in the auxiliary storage unit 103. The threshold information DB 15 and the void ratio DB 16 are, for example, relational databases. Note that any of the functional components of the image processing device 10, or part of the processing thereof, may be executed by another computer on a communication network connected via the communication unit 105.

[0034] The threshold information DB15 stores threshold information in the HSV color space corresponding to cultivated varieties of agricultural crops, fruit trees, etc. when calculating the air-open ratio. Examples of such cultivated varieties include grape varieties, apricots, peaches, pears, apples, figs, kiwis, hops, melons, etc. The air-open ratio DB16 stores the calculated air-open ratio in association with date and time information, the type of cultivated variety, etc. The calculated air-open ratio may be stored in the air-open ratio DB16 in association with, for example, the field number for which the air-open ratio was calculated, the plot number in the field, and location information such as latitude and longitude.

[0035] The captured image acquisition unit 11 acquires a captured image Z2 captured via the imaging camera 10a. For example, a moving image captured at a predetermined frame rate of 30 fps is acquired. The captured image acquisition unit 11 temporarily stores the acquired captured image Z2 in a predetermined storage area of ​​the main storage unit 102 and transfers the captured image to the image processing unit 12.

[0036] The image processing unit 12 converts the color elements of the captured image Z2 delivered from the captured image acquisition unit 11 from a color space represented by (R, G, B) to an HSV color space represented by elements such as hue, saturation, and brightness. The conversion to the HSV color space is performed for each frame, and each pixel of the converted captured image Z2 is temporarily stored in a predetermined storage area of ​​the main storage unit 102.

[0037] The image processing unit 12 performs binarization processing based on the pixel (bright spot) information for each frame converted into the HSV color space, and calculates the open-sky ratio for that frame, i.e., the proportion of the pixel area corresponding to the sky. For example, if the brightness of the bright spot being processed is equal to or greater than a predetermined threshold (e.g., 90 percent), the image processing unit 12 determines that the color type to which the bright spot belongs is white, and paints out any bright spots with a brightness equal to or greater than the threshold in white. On the other hand, for bright spots determined to have a brightness less than the threshold, the image processing unit 12 determines whether they belong to white or black based on the hue information.

[0038] For example, the image processing unit 12 classifies bright points whose hue belongs to a predetermined range (for example, a range from 0 degrees to 177.7 degrees) as a hue belonging to black, and paints bright points within that range black. Also, the image processing unit 12 classifies bright points whose hue belongs to a range (for example, a range from 183.4 degrees to 240.8 degrees) different from the predetermined range painted black as a hue belonging to white, and paints bright points within that range white.

[0039] For bright spots whose hue information is determined not to belong to either black or white, the image processing unit 12 again uses the brightness information to determine whether they belong to white or black. For example, if the brightness of the bright spot being processed is equal to or greater than a predetermined threshold (e.g., 2 percent), the image processing unit 12 determines that the bright spot belongs to the white color type and paints any bright spots with a brightness equal to or greater than the threshold in white. On the other hand, if the brightness of the bright spot being processed is less than a predetermined threshold, the image processing unit 12 determines that the bright spot belongs to the black color type and paints any bright spots with a brightness less than the threshold in black.

[0040] The image processing unit 12 then calculates the air gap ratio for the captured video frame by calculating the percentage of bright points determined to belong to the white color type from the frame images that have been binarized into black or white. The calculated air gap ratio is temporarily stored in a predetermined storage area of ​​the main storage unit 102 and transferred to the display processing unit 13. Note that the image processing unit 12 may use multiple consecutive frame images to calculate the air gap ratio. For example, from the air gap ratios calculated for each of five consecutive frames, the frames with the maximum and minimum air gap ratios may be excluded, and the air gap ratios calculated for the remaining three frames may be averaged. By calculating the air gap ratio using multiple consecutive frame images, it is possible to calculate the air gap ratio with a certain degree of accuracy even when, for example, camera shake or tilt occurs.

[0041] The display processing unit 13 displays the calculated air-open ratio on a display device 10b such as an LCD connected via the input / output unit 104. Similarly, the display processing unit 13 may notify the calculated air-open ratio as a voice message via a speaker 10d or the like connected via the input / output unit 104. The calculated air-open ratio can be notified to the worker Z1 whose line of sight is directed toward the tree branches, leaves 21, or cultivation shelves 20 via a voice message.

[0042] The notification processor 14 issues a notification to ensure that the horizontal angle of the opening surface of the imaging unit of the imaging camera 10a with respect to the ground 23 does not deviate from a predetermined range, based on the inclination of the device with respect to the vertical direction detected via the sensor 10c, etc. For example, if the inclination of the device with respect to the vertical direction detected via the sensor 10c, etc., deviates from the predetermined range, the notification processor 14 issues a voice message via the speaker 10d, etc. connected via the input / output unit 104. For example, a voice message such as "The camera is tilted. Please level the camera" is issued to the worker Z1.

[0043] (Processing flow) The processing of the image processing device 10 according to this embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing an example of the processing flow related to calculation of the aperture ratio. Fig. 6 is a flowchart showing an example of the processing flow for notifying the tilt of the imaging camera.

[0044] In FIG. 5, after the start of processing, the image processing device 10 acquires a captured image Z2 captured via the imaging camera 10a (step S1), and the processing proceeds to step S2. As shown in FIG. 3, the imaging camera 10a, positioned at a predetermined height, captures an image of a cultivation shelf 20, on which the fruits, branches, and leaves 21 of a cultivated variety are growing, at a predetermined frame rate, as viewed from above in a field where a brightness index is to be measured. In step S2, the image processing device 10 converts the color elements of each pixel constituting the acquired frame image from a color space represented by (R, G, B) to an HSV color space represented by elements such as hue, saturation, and brightness, and the processing proceeds to step S3. Through the processing of step S2, the color elements of the frame image captured by the imaging camera 10a are represented as hue information, which is 360-degree angular information, with red representing 0 degrees and returning to yellow, green, blue, and red. Furthermore, saturation, which indicates the vividness of a color, and brightness, which indicates the brightness of a color, are each represented as numerical information ranging from 0 to 100 percent.

[0045] In step S3, the color type for binarization is determined based on the brightness information of the pixel (bright spot) converted into the HSV color space. For example, if the brightness of the bright spot to be processed is equal to or greater than a first threshold (step S3, "Yes"), the image processing device 10 determines that the color type of the bright spot is white, and the process proceeds to step S7; otherwise (step S3, "No"), the process proceeds to step S4. Here, the first threshold can be, for example, a brightness value of 90 percent.

[0046] In step S4, the color type for binarization is determined based on the hue information of the bright spot converted into the HSV color space. If the hue of the bright spot to be processed belongs to the first range (step S4, "Yes"), the image processing device 10 determines that the color type of the bright spot is black, and the process proceeds to step S8; if not (step S4, "No"), the process proceeds to step S5. Here, the first range can be, for example, a range in hue angle value from 0 degrees to 177.7 degrees.

[0047] In step S5, for bright spots determined in step S4 to have hue angle values ​​that do not fall within the first range, the color type for binarization is again determined based on the hue information. If the hue angle value of the bright spot to be processed falls within the second range (step S5, "Yes"), the image processing device 10 determines that the color type of the bright spot is white, and the process proceeds to step S7; otherwise (step S5, "No"), the process proceeds to step S6. Here, the second range can be, for example, a range of hue angle values ​​from 183.4 degrees to 240.8 degrees.

[0048] In step S6, for bright spots determined to have hue angles that do not fall within either the first or second range, the color type for binarization is again determined based on the brightness value. For example, if the brightness value of the bright spot to be processed is equal to or greater than the second threshold (step S6, "Yes"), the image processing device 10 determines that the color type of the bright spot is white, and the process proceeds to step S7; otherwise (step S6, "No"), the process proceeds to step S8. Here, the second threshold can be, for example, a brightness value of 2%.

[0049] In the image processing device 10, by the processing of steps S3 to S6, it is determined whether the color element of each pixel that constitutes the frame image after conversion into the HSV color space is a bright point that belongs to either the white or black color type.

[0050] In step S7, bright points (pixels) to be processed that have been determined to belong to the white color category in the processes of steps S3 to S6 are filled in white. Similarly, in step S8, bright points (pixels) to be processed that have been determined to belong to the black color category in the processes of steps S3 to S6 are filled in black. After steps S7 and S8, the process proceeds to step S9.

[0051] In step S9, the open-air ratio in the frame image that has been binarized to black or white is calculated from the proportion of bright points that have been determined to belong to the color type of white. The image processing device 10 temporarily stores the calculated open-air ratio in a predetermined storage area of ​​the main storage unit 102, and proceeds to step S10. In step S10, the calculated open-air ratio is displayed on the display device 10b, such as an LCD, connected via the input / output unit 104. The calculated open-air ratio is also notified as a voice message via the speaker 10d, etc., connected via the input / output unit 104. After the processing of step S10, this routine is temporarily terminated.

[0052] In step S9, the image processing device 10 may calculate the air openness ratio using multiple consecutive frame images. For example, the air openness ratios calculated for each of five consecutive frames may be calculated by excluding the frames with the maximum and minimum air openness ratios and averaging the air openness ratios calculated for the remaining three frames. By using the air openness ratios calculated for multiple consecutive frame images in calculating the air openness ratio, it is possible to provide an air openness ratio with a certain degree of accuracy even when camera shake or tilt occurs, for example. Furthermore, by appropriately setting the number of frame images used to calculate the air openness ratio depending on, for example, the location of the field, the type of cultivated variety, weather, etc., it is possible to provide an air openness ratio that corresponds to these differences.

[0053] Next, FIG. 6 will be described. In FIG. 6, after starting the process, the image processing device 10 acquires a detection value for the vertically downward direction of the device from the sensor 10c connected via the input / output unit 104 (step S11), and the process proceeds to step S12. In step S12, based on the acquired detection value, it is determined whether the tilt of the opening surface of the imaging unit of the imaging camera 10a with respect to the horizontal plane is within a predetermined range. Here, the horizontal plane is a plane perpendicular to the vertical direction. An example of the predetermined range is a tilt range of ±1.5 degrees with respect to the horizontal plane. However, such a range may be set appropriately based on the viewing angle of the imaging camera 10a, the height position of the camera, etc.

[0054] In step S12, if the sensor detection value is within a predetermined range (step S12, "Yes"), the orientation of the imaging camera 10 is determined to be within the normal range, and the processing of this routine is temporarily terminated. On the other hand, if the sensor detection value is not within the predetermined range (step S12, "No"), the processing proceeds to step S13. In step S13, a predetermined audio message is announced through the speaker 10d connected via the input / output unit 104. For example, an audio message such as "The camera is tilted. Please adjust the camera to a horizontal position" is announced to an operator Z1 who is measuring the brightness of the field while holding the image processing device 10 in one hand. This can prompt the operator Z1, whose line of sight is directed toward tree branches, leaves 21, or cultivation shelves 20 when measuring the brightness of the field, to be aware of the need to take appropriate images when measuring the openness ratio. After the processing of step S13, the routine is temporarily terminated.

[0055] Through the above processing, the image processing device 10 according to this embodiment can acquire, via the imaging camera 10a, a moving image of the current state of a field where agricultural crops, fruit trees, etc. are grown, captured at a predetermined frame rate. The image processing device 10 can convert the color elements of each pixel in each captured frame from a color space represented by (R, G, B) to an HSV (Hue, Saturation, Value) color space represented by elements such as hue, saturation, and brightness. Then, the image processing device 10 performs binarization processing based on the color space information represented by HSV, and obtains the open-sky ratio in the captured image Z2, i.e., The proportion of the pixel area corresponding to the sky can be calculated. As a result, the image processing device 10 according to this embodiment can provide a technique for easily measuring the current brightness state in a farm field.

[0056] Furthermore, the image processing device 10 according to this embodiment can display the calculated cavity open ratio on a display device 10b such as an LCD connected via the input / output unit 104. Similarly, the calculated cavity open ratio can be notified as a voice message via a speaker 10d or the like. Usability is improved because the calculated cavity open ratio can be notified via a voice message to the worker Z1 whose line of sight is directed towards the branches, leaves 21, or cultivation shelves 20.

[0057] Furthermore, the image processing device 10 according to this embodiment can notify the operator Z1 that the inclination of the opening surface of the imaging unit of the imaging camera 10a relative to the horizontal plane does not deviate from a predetermined range, based on the inclination of the device relative to the vertical direction detected via the sensor 10c, etc. When measuring an index showing the brightness of the field, the operator Z1, whose line of sight is directed toward the tree branches, leaves 21, or cultivation shelves 20, can be made aware of the need to take appropriate images when measuring the openness ratio.

[0058] <Comparison with conventional methods> 7 is a diagram illustrating the correlation between the brightness measured by the image processing device 10 according to the present embodiment and the results of conventional image analysis. In Fig. 7, the vertical axis represents the degree of brightness calculated by conventional image analysis, and the horizontal axis represents the degree of brightness calculated by the image processing device 10. As shown in graph g1 of Fig. 7, a good correlation was obtained between the openness ratio (%) calculated by performing binarization processing based on color space information expressed in HSV and the brightness level (%) based on image analysis. Linear approximation was performed using the openness ratio calculated using the image processing device 10 of this embodiment as variable x and the brightness level based on conventional image analysis as variable y, and the coefficients a and b for the linear approximation equation y = ax + b were identified using the least squares method. As a result, the following approximate equation (1) was obtained, as an example. Approximate formula (1): y=0.9547x-0.0066 The mean square error (R 2 ) is obtained as R 2 =0.9796, which means that we were able to obtain measurement accuracy equivalent to the degree of brightness based on image analysis.

[0059] <Other embodiments> The above-described embodiment is merely an example, and the present disclosure may be modified as appropriate within the scope of the present disclosure. The processes and means described in the present disclosure may be freely combined and implemented as long as no technical contradiction occurs.

[0060] Furthermore, a process described as being performed by one device may be shared and executed by multiple devices. Alternatively, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) by which each function is realized can be flexibly changed.

[0061] The present disclosure can also be realized by supplying a computer program that implements the functions described in the above embodiments to a computer, and having one or more processors of the computer read and execute the program. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. Non-transitory computer-readable storage media include, for example, any type of disk, such as a magnetic disk (e.g., a floppy disk, a hard disk drive (HDD), etc.), an optical disk (e.g., a CD-ROM, a DVD disk, a Blu-ray disk), a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic This includes any type of medium suitable for storing electronic instructions, such as a card, flash memory, or optical card. [Explanation of symbols]

[0062] 10 Image processing device 10a Imaging camera 10b Display Device 10c Sensor 10d speaker 11. Image acquisition unit 12 Image processing section 13 Display processing section 14. Abandoned Treatment Area 15 Threshold Information Database 16 Openness Rate Database

Claims

1. Acquiring a photographed image of the state of a field including a plant to be cultivated; determining the type of color element to which a pixel belongs based on hue information and brightness information of the pixel constituting the photographed image, and binarizing the type of color element to which the pixel belongs as white or black; calculating an index indicating brightness of the farm field from a ratio of pixels determined to be white among pixels constituting the captured image; An image processing method comprising:

2. The image processing method according to claim 1 , wherein the captured images are moving images captured from a viewpoint looking from below to above the plant to be cultivated.

3. 3. The image processing method according to claim 1, further comprising converting RGB values ​​of pixels constituting the photographed image into a color space represented by information on hue, saturation, and brightness of the pixels.

4. 4. The image processing method according to claim 1, wherein when brightness information of a pixel constituting the photographed image is equal to or greater than a first threshold, the type of color element to which the pixel belongs is determined to be white.

5. 4. The image processing method according to claim 1, wherein when the brightness information of a pixel constituting the captured image is less than a first threshold, the type of color element to which the pixel belongs is determined to be black, provided that the hue information of the pixel belongs to a first range.

6. 4. The image processing method according to claim 1, wherein when the brightness information of a pixel constituting the captured image is less than a first threshold, the type of color element to which the pixel belongs is determined to be white, provided that the hue information of the pixel belongs to a second range.

7. 4. The image processing method according to claim 1, wherein when the brightness information of a pixel constituting the captured image is less than a first threshold value and the hue information of the pixel does not belong to the first range or the second range, the type of the color element to which the pixel belongs is determined to be white, provided that the brightness information of the pixel is equal to or greater than a second threshold value.

8. 4. The image processing method according to claim 1, wherein when the brightness information of a pixel constituting the captured image is less than a first threshold value and the hue information of the pixel does not belong to either the first range or the second range, the type of color element to which the pixel belongs, whose brightness information does not satisfy the condition of being equal to or greater than the second threshold value, is determined to be black.

9. An image processing method described in any one of claims 1 to 8, wherein the captured image is a moving image captured at a predetermined frame rate, and an index indicating brightness in the field is calculated from multiple consecutive frame images.

10. The image processing method according to claim 1 , further comprising: announcing by voice an index indicating brightness in the farm field calculated from the captured image.

11. An acquisition unit that acquires photographed images of the state of a field including plants to be cultivated; a discrimination unit that discriminates the type of color element to which a pixel belongs based on hue information and brightness information of the pixel that constitutes the captured image, and binarizes the type of color element to which the pixel belongs as white or black; a calculation unit that calculates an index indicating brightness of the farm field from the ratio of pixels that are determined to be white among pixels that constitute the captured image; An image processing device comprising:

Images