Image Processing Device

Abstract

To simply measure a state of present brightness in a farm field.SOLUTION: An image processing apparatus comprises: an acquisition unit that acquires an imaged image in which a state of a farm field including a cultivation object plant is imaged; a determination unit that determines a type of color element, to which a pixel belongs, as a binary value of a white color or a black color on the basis of hue information and lightness information of the pixel constituting the imaged image; and a calculation unit that calculates an index indicating brightness in the farm field from a ratio of pixels which are determined to have a white color, of pixels constituting the imaged image.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device for measuring brightness of a farm field. [Background technology]

[0002] In the cultivation of plants such as agricultural crops and fruit trees, it is necessary to manage the environment so as to maintain an appropriate growth environment according to the growth state of the plants in order to improve productivity. For example, Patent Document 1 discloses an environmental control system that controls the environment inside agricultural facilities such as vinyl greenhouses and greenhouses (hereinafter referred to as agricultural greenhouses). Patent Document 1 discloses that the agricultural greenhouses are equipped with adjustment devices for adjusting the internal environment, such as various measuring devices that acquire internal environmental data such as temperature, humidity, sunlight, and carbon dioxide concentration, as well as air conditioners, blowers, temperature and humidity control devices, carbon dioxide generators, and sunshades. It is also disclosed that a control device of the environmental control system controls the adjustment devices based on various data acquired by the measuring devices installed in the agricultural greenhouse, thereby maintaining the internal environment of the agricultural greenhouse at a preset environment.

[0003] In recent years, there has been a growing trend to differentiate agricultural crops and fruit trees produced in a particular region from those of the same type produced in other regions by branding them as specialty products. One example of such a specialty product is the Ruby Roman grape cultivar. By improving the soil and managing moisture in the fields in cultivation facilities and appropriately controlling the amount of sunlight and other brightness during harvest, it has become possible to consistently cultivate large berries with bright red skin. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-87100 Summary of the Invention [Problem to be solved by the invention]

[0005] In order to ensure that the skin of the berries in the above cultivars turns a vivid red at harvest time, it is necessary to maintain the brightness of the field at an appropriate value during the veraison period. This appropriate value is defined, for example, as a certain level of brightness when looking up at the cultivation shelves from the bunches of fruit. However, measuring the brightness of the field during the veraison period requires photographing the appearance of the bunches of fruit and the cultivation shelves with a camera or the like, and then analyzing the camera images on a PC or the like. This has led to the problem that growers of these cultivars are unable to measure the current brightness of the field during the veraison period or at harvest time.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to provide a technology that can easily measure the current brightness state in a farm field. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present disclosure includes an acquisition unit that acquires a captured image of the state of a field including plants to be cultivated, a discrimination unit that discriminates the type of color element to which a pixel belongs as a binary value of white or black based on hue information and brightness information of the pixels constituting the captured image, and a calculation unit that calculates an index indicating the brightness of the field from the proportion of pixels discriminated as white among the pixels constituting the captured image. This makes it possible to easily measure the current brightness state of the field using the captured image of the state of the field including plants to be cultivated.

[0008] The captured images may also be moving images captured from a viewpoint looking from below to above the plants to be cultivated, thereby enabling capture of images suitable for measuring the current brightness conditions in the field.

[0009] The discrimination unit may also convert the RGB values ​​of the pixels constituting the captured image into a color space represented by the hue, saturation, and brightness of the pixels, thereby enabling binarization processing to measure the current brightness state of the field based on the pixel information converted into the HSV color space represented by elements such as hue, saturation, and brightness.

[0010] The discrimination unit may also be configured to determine that the type of color element to which a pixel belongs is white when the lightness information of the pixel constituting the captured image is equal to or greater than a first threshold. The discrimination unit may also be configured to determine that the type of color element to which a pixel belongs is black when the lightness information of the pixel constituting the captured image is less than the first threshold, provided that the hue information of the pixel belongs to a first range. The discrimination unit may also be configured to determine that the type of color element to which a pixel belongs is white when the lightness information of the pixel constituting the captured image is less than the first threshold, provided that the hue information of the pixel belongs to a second range. The discrimination unit may also be configured to determine that the type of color element to which a pixel belongs is white when the lightness information of the pixel constituting the captured image is less than the first threshold and the hue information of the pixel does not belong to either the first range or the second range, provided that the lightness information of the pixel is equal to or greater than a second threshold. Furthermore, when the brightness information of a pixel constituting the captured image is less than a first threshold and the hue information of the pixel does not belong to either the first range or the second range, the discrimination unit may determine that the type of color element to which the pixel, whose brightness information does not satisfy the condition of being equal to or greater than the second threshold, belongs is black. This enables binarization processing using at least the hue information and saturation information of the pixel information converted into the HSV color space represented by elements such as hue, saturation, and brightness.

[0011] Also, Captured image are moving images captured at a predetermined frame rate, and the index indicating the brightness of the field may be calculated from a plurality of consecutive frame images. By calculating the index indicating the brightness of the field using a plurality of consecutive frame images, it is possible to calculate the index with a certain degree of accuracy even when, for example, camera shake or tilt occurs.

[0012] The farm system may also include a first notification unit that notifies by voice of an index indicating the brightness of the field calculated from the captured image. This allows the calculated index to be notified by voice to an operator whose line of sight is directed toward tree branches, leaves, cultivation shelves, etc. The farm system may also include a second notification unit that notifies by voice to maintain the correct posture of the imaging camera that captures the captured image. This allows an operator whose line of sight is directed toward tree branches, leaves, cultivation shelves, etc., to be aware of the posture of the imaging camera in order to obtain an appropriate captured image when measuring the index indicating the brightness of the field.

[0013] Furthermore, the image processing device may be configured to include an integrated imaging camera that captures the captured images. This allows the worker to easily measure the current brightness state in the field based on the captured images taken by the imaging camera configured to be integrated with the image processing device. Alternatively, the image processing device may be configured separately from the imaging camera that captures the captured images, and the acquisition unit may acquire the captured images being captured by the imaging camera via a communication interface. This allows the worker to measure the current brightness state in the field while carrying a separate imaging camera, which is expected to improve usability. [Effects of the Invention]

[0014] The present disclosure provides a technology that can easily measure the current brightness state in a farm field. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating an image processing apparatus according to an embodiment. [Figure 2] FIG. 10 is a diagram illustrating a measurement form of an index indicating the brightness of a farm field using an image processing device. [Figure 3] FIG. 2 is a diagram showing an example of an image captured by an imaging camera. [Figure 4] FIG. 1 is a diagram illustrating an example of a functional configuration of an image processing apparatus. [Figure 5]10 is a flowchart showing an example of a process flow for calculating a cavity opening ratio according to the embodiment. [Figure 6] 10 is a flowchart illustrating an example of a flow of a process for notifying the tilt of the imaging camera according to the embodiment. [Figure 7] 10A and 10B are diagrams illustrating the correlation between brightness measured by the image processing device according to the embodiment and a conventional image analysis result. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The configurations of the following embodiments are examples, and the present disclosure is not limited to the configurations of the embodiments. Furthermore, the following embodiments can be combined as much as possible.

[0017] First Embodiment FIG. 1 is a diagram illustrating an image processing device 10 according to this embodiment. The image processing device 10 according to this embodiment is a portable information processing device capable of measuring an index indicating the current brightness of a field in a growing environment for cultivated varieties such as agricultural crops and fruit trees. Examples of such information processing devices include a smartphone, a tablet PC, and a laptop PC. As shown in FIG. 1, the image processing device 10 includes an imaging camera 10a and is equipped with an application program (hereinafter simply referred to as an "app") for measuring the brightness index of a field in which plants such as agricultural crops and fruit trees are grown based on images captured by the camera. An operator Z1 who measures the current brightness index of the field carries the image processing device 10 and performs measurements while pruning tree leaves 21, etc., so that the measured brightness becomes a constant brightness index. In the following description, the image processing device 10 and the imaging camera 10a are described as being integrally configured; however, the imaging camera 10a may be configured separately from the image processing device 10. The image processing device 10 is connected to the imaging camera 10a, which is configured separately, via a communication unit 105, for example.

[0018] 1, the image processing device 10 is a computer including, as its components, a processor 101, a main memory unit 102, an auxiliary memory unit 103, an input / output unit 104, and a communication unit 105, all of which are interconnected by a connection bus. The main memory unit 102 and the auxiliary memory unit 103 constitute a memory, and are recording media readable by the image processing device 10. Each of the above components may be provided in multiple units, or some of the components may not be provided at all.

[0019] The processor 101 is a central processing unit that controls the entire image processing device 10. The processor 101 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor). The processor 101 is an example of a control unit. For example, the processor 101 deploys a program stored in the auxiliary storage unit 103 in an executable manner in a working area of ​​the main storage unit 102, and controls peripheral devices through the execution of the program, thereby providing a function that meets a predetermined purpose.

[0020] The main memory 102 stores programs executed by the processor 101, data processed by the processor, etc. The main memory 102 includes a flash memory, a RAM (Random Access Memory), and a ROM (Read Only Memory). The auxiliary memory 103 includes a portable recording medium, an Volatile semiconductor memory (flash memory, EPROM (Erasable Programmable ROM)) The portable recording medium includes, for example, a silicon disk, a solid state drive device, a hard disk drive (HDD), etc. The auxiliary storage unit 103 is a recording medium such as a card. The auxiliary storage unit 103 stores various programs and various data in a readable and writable manner. The auxiliary storage unit 103 is used as a storage area supporting the main storage unit 102 and stores programs executed by the processor 101, data processed by the processor 101, and the like. The auxiliary storage unit 103 stores, for example, an operating system (OS), various programs, various tables, and the like. The OS includes a communication interface program that transfers data between devices connected via the communication unit 105. Here, the image processing device 10 may be a single computer or a combination of multiple computers. Furthermore, information stored in the auxiliary storage unit 103 may be stored in the main storage unit 102, and information stored in the main storage unit 102 may be stored in the auxiliary storage unit 103. As described above, the processor 101 executes a program to execute a series of processes in the image processing device 10. However, at least a part of the series of processes may be executed by hardware such as a digital circuit.

[0021] The input / output unit 104 is an interface for inputting and outputting data to and from devices connected to the image processing device 10. Input devices connected to the input / output unit 104 include, for example, an imaging camera 10a, a keyboard, a pointing device such as a touch panel or a mouse, a sensor 10c, and a microphone. Operation instructions and the like from an operator who operates the input devices are received via the input / output unit 104.

[0022] Furthermore, the input / output unit 104 is connected to output devices such as a display device 10b, such as an LCD (Liquid Crystal Display), an EL (Electroluminescence) panel, or an organic EL panel, and a speaker 10d. The display device 10b is combined with a touch panel to form an input device. Data and information processed by the processor 101 and data and information stored in the main memory unit 102 and the auxiliary memory unit 103 are output via the input / output unit 104. The communication unit 105 is a communication interface with a communication network. The communication unit 105 can have an appropriate configuration depending on the connection method with the communication network. Examples of the communication unit 105 include a LAN (Local Area Network) interface board and a wireless communication circuit for wireless communication.

[0023] The imaging camera 10a is, for example, a CCD (Charge Coupled Device) image sensor or a C Using an imaging element such as a MOS (Complementary Metal Oxide Semiconductor) image sensor The imaging camera 10a is provided, for example, on the side of the opening where the display device 10b of the image processing device 10 is open. The image obtained by capturing may be either a still image or a video at a predetermined frame rate (for example, 30 fps). The sensor 10c is, for example, an acceleration sensor that detects gravitational acceleration. The image processing device 10 detects the inclination of its own device with respect to the vertical direction based on the detection value detected via the sensor 10c.

[0024] FIG. 2 is a diagram illustrating a measurement mode of an index showing the brightness of a farm field using an image processing device 10. FIG. 2 illustrates a side view of an image capture using an imaging camera 10a when measuring an index value in a farm field where agricultural crops, fruit trees, etc. are grown. The measurement mode shown in FIG. 2 is an example of a measurement mode in a farm field where grapes 22 are grown. As shown in FIG. 2, in the farm field where grapes 22 are grown, cultivation shelves 20 are provided for tree branches and leaves 21 to grow on, and bunches of grapes 22, which are the fruits to be grown, are covered with hanging bags 22a to prevent damage from pests and diseases and sunburn of the skin and to promote coloring.

[0025] The image processing device 10 according to this embodiment detects, via the imaging camera 10a, the location where the hanging bag 22a is hanging. The image capturing camera 10a captures an image of the cultivation shelf 20 on which the branches and leaves 21 are growing from below the grapevines 22 on which the hanging bags 22a are hung. For example, a worker Z1 measuring a brightness index value in a farm field holds the image processing device 10 in one hand while keeping the image capturing camera 10a in a shooting state, and captures an image of the cultivation shelf 20 on which the branches and leaves 21 are growing from a viewpoint facing upward from below the grapevines 22 on which the hanging bags 22a are hung. Alternatively, the image processing device 10 may be placed on a support stand or the like while keeping the image capturing camera 10a in a shooting state, and capture an image of the cultivation shelf 20 on which the branches and leaves 21 are growing from a viewpoint facing upward from below the grapevines 22 on which the hanging bags 22a are hung. Furthermore, when the image capturing camera 10a and the image processing device 10 are configured separately, the image processing device 10 may be placed on a support stand or the like. Then, a worker Z1 carrying an imaging camera 10a may photograph the cultivation rack 20 on which the branches and leaves 21 are spread, from a viewpoint looking upward from below the grapes 22 on which the hanging bags 22a are hung. The photographed moving images are transmitted to the image processing device 10 via a communication network such as wireless communication, for example.

[0026] For example, if the height h1 of the cultivation shelf 20 is approximately 1.7 m above the ground 23 of the field, the imaging camera 10a is installed at a height h2 of approximately 1.0 m so that the opening of the imaging unit of the imaging camera 10a, i.e., the surface of the display device 10b, is parallel to the ground 23. The image processing device 10 then acquires an image captured by the imaging camera 10a in the shooting state, showing the grapes 22 with hanging bags 22a, the branches and leaves 21, and the state of the cultivation shelf 20 within a viewing angle 24 (the area enclosed by dashed lines). The viewing angle of the imaging camera 10a according to this embodiment is approximately 75°.

[0027] The imaging height of the imaging camera 10a is appropriately set depending on the cultivation target in the field to be measured, the height of the cultivation shelves, the viewing angle 24 of the imaging camera 10a, etc. The image processing device 10 may also notify the operator Z1 that the horizontal angle of the opening of the imaging unit of the imaging camera 10a with respect to the ground 23 is within a predetermined range based on the tilt of the device itself relative to the vertical direction detected via the sensor 10c, etc. For example, if an operator Z1 is measuring the brightness of the field while holding the image processing device 10 in one hand, a voice message such as "The camera is tilted. Please adjust the camera to a horizontal position" can be notified via the speaker 10d. When measuring the brightness of the field, the operator Z1, whose line of sight is directed toward tree branches, leaves 21, or the cultivation shelves 20, can be encouraged to pay attention to the orientation of the imaging camera 10a to obtain appropriate images.

[0028] FIG. 3 is a diagram showing an example of an image captured by the imaging camera 10a. As shown in the captured image Z2 in FIG. 3, the state of the cultivation rack 20, on which branches and leaves 21 are spread, is captured from below the grapevines 22, on which hanging bags 22a are hung. The image processing device 10 according to this embodiment calculates the sky-open ratio as an index indicating the brightness of the farm field, based on the captured image Z2 captured by the imaging camera 10a. Here, the sky-open ratio is, for example, the proportion of an area into which sunlight can be incident without being blocked by leaves 21 when looking up at the sky in a certain area, such as a farm field. The brightness index may be any index other than the sky-open ratio, as long as it has the same meaning as the proportion of an area into which sunlight can be incident without being blocked by leaves when looking up at the sky in a certain area, such as a farm field. It may also be an index obtained by performing a predetermined calculation on the sky-open ratio. In this embodiment, the sky-open ratio represents the proportion of pixel areas corresponding to areas where the sky is reflected in the captured image Z2 captured by the imaging camera 10a. In this embodiment, the openness ratio is an example of an "index indicating brightness in a farm field."

[0029] Specifically, the image processing device 10 acquires moving images captured at a predetermined frame rate via the imaging camera 10a. Then, the image processing device 10 converts the color elements of each pixel for each frame from a color space expressed as (R, G, B) to an HSV (Hue, Saturation, Value) color space expressed by elements such as hue, saturation, and brightness. As a result, the RGB values ​​of each pixel constituting the captured image Z2 captured by the imaging camera 10a are expressed as yellow, green, blue, and The hue information is expressed as a 360-degree angle that returns to red. Similarly, the saturation, which indicates the vividness of the color, and the value, which indicates the brightness of the color, can each be expressed as numerical information in the range of 0 to 100 percent. Hereinafter, pixels converted into the HSV color space will also be referred to as bright points.

[0030] The image processing device 10 according to this embodiment calculates the sky openness ratio in the captured image Z2, i.e., the proportion of the pixel area corresponding to the area where the sky is reflected, based on each piece of information (color space information) expressed in HSV. The image processing device 10 determines the color type to which the pixel belongs, for example, based on numerical information indicating brightness. For example, if the brightness of the bright spot to be determined is 90 percent or higher, the image processing device 10 determines that the color type to which the bright spot belongs is white, and if not, determines that the color type is other than white.

[0031] Next, the image processing device 10 classifies the origin determined to be a color type other than white as black or white based on the angle information indicating the hue. For example, the image processing device 10 classifies bright points to be determined whose hue falls within a range of 0 degrees to 177.7 degrees (a range from red to yellow and blue-green including green) as belonging to a black hue. Furthermore, the image processing device 10 classifies bright points to be determined whose hue falls within a range of 183.4 degrees to 240.8 degrees (a range from blue to ultramarine) as belonging to a white hue.

[0032] Furthermore, for bright spots that were identified as not belonging to either black or white using the above-mentioned hue, the image processing device 10 again uses brightness to determine whether the bright spot belongs to white or black. For example, if the brightness of the bright spot to be identified is 2% or higher, the image processing device 10 identifies the bright spot as belonging to the color type of white, and otherwise identifies the bright spot as belonging to the color type of black.

[0033] Through the above processing, the image processing device 10 can obtain a binary image discriminated as white or black for each frame of the moving image captured via the imaging camera 10a. The image processing device 10 can then calculate the percentage of bright points discriminated as belonging to the color type white from the binarized frame images, thereby determining the sky openness ratio in the captured moving image frames, i.e., the percentage of pixel areas corresponding to areas where the sky is reflected. The image processing device 10 according to this embodiment can calculate the sky openness ratio as an index showing the brightness of the field, based on the captured image Z2 captured via the imaging camera 10a.

[0034] (Functional configuration) Next, the functional configuration of the image processing device 10 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram schematically illustrating an example of the functional configuration of the image processing device 10. The image processing device 10 includes, as functional components, a captured image acquisition unit 11, an image processing unit 12, a display processing unit 13, a notification processing unit 14, a threshold information DB 15, and an openness ratio DB 16. A processor 101 of the image processing device 10 executes the processes of the captured image acquisition unit 11, the image processing unit 12, the display processing unit 13, and the notification processing unit 14 using a computer program loaded in a main memory unit 102. However, any one of the functional components, or part of the processes, may be executed by hardware such as a digital circuit.

[0035] The threshold information DB 15 and the void ratio DB 16 are constructed by a database management system (DBMS) program executed by the processor 101 managing data stored in the auxiliary storage unit 103. The threshold information DB 15 and the void ratio DB 16 are, for example, relational databases. Note that any of the functional components of the image processing device 10, or part of the processing thereof, may be executed by another computer on a communication network connected via the communication unit 105.

[0036] The threshold information DB15 stores threshold information in the HSV color space corresponding to cultivated varieties of agricultural crops, fruit trees, etc. when calculating the air-open ratio. Examples of such cultivated varieties include grape varieties, apricots, peaches, pears, apples, figs, kiwis, hops, melons, etc. The air-open ratio DB16 stores the calculated air-open ratio in association with date and time information, the type of cultivated variety, etc. The calculated air-open ratio may be stored in the air-open ratio DB16 in association with, for example, the field number for which the air-open ratio was calculated, the plot number in the field, and location information such as latitude and longitude.

[0037] The captured image acquisition unit 11 acquires a captured image Z2 captured via the imaging camera 10a. For example, a moving image captured at a predetermined frame rate of 30 fps is acquired. The captured image acquisition unit 11 temporarily stores the acquired captured image Z2 in a predetermined storage area of ​​the main storage unit 102 and transfers the captured image to the image processing unit 12.

[0038] The image processing unit 12 converts the color elements of the captured image Z2 delivered from the captured image acquisition unit 11 from a color space represented by (R, G, B) to an HSV color space represented by elements such as hue, saturation, and brightness. The conversion to the HSV color space is performed for each frame, and each pixel of the converted captured image Z2 is temporarily stored in a predetermined storage area of ​​the main storage unit 102.

[0039] The image processing unit 12 performs binarization processing based on the pixel (bright spot) information for each frame converted into the HSV color space, and calculates the open-sky ratio for that frame, i.e., the proportion of the pixel area corresponding to the sky. For example, if the brightness of the bright spot being processed is equal to or greater than a predetermined threshold (e.g., 90 percent), the image processing unit 12 determines that the color type to which the bright spot belongs is white, and paints out any bright spots with a brightness equal to or greater than the threshold in white. On the other hand, for bright spots determined to have a brightness less than the threshold, the image processing unit 12 determines whether they belong to white or black based on the hue information.

[0040] For example, the image processing unit 12 classifies bright points whose hue belongs to a predetermined range (for example, a range from 0 degrees to 177.7 degrees) as a hue belonging to black, and paints bright points within that range black. Also, the image processing unit 12 classifies bright points whose hue belongs to a range (for example, a range from 183.4 degrees to 240.8 degrees) different from the predetermined range painted black as a hue belonging to white, and paints bright points within that range white.

[0041] For bright spots whose hue information is determined not to belong to either black or white, the image processing unit 12 again uses the brightness information to determine whether they belong to white or black. For example, if the brightness of the bright spot being processed is equal to or greater than a predetermined threshold (e.g., 2 percent), the image processing unit 12 determines that the bright spot belongs to the white color type and paints any bright spots with a brightness equal to or greater than the threshold in white. On the other hand, if the brightness of the bright spot being processed is less than a predetermined threshold, the image processing unit 12 determines that the bright spot belongs to the black color type and paints any bright spots with a brightness less than the threshold in black.

[0042] The image processing unit 12 then calculates the air gap ratio for the captured video frame by calculating the percentage of bright points determined to belong to the white color type from the frame images that have been binarized into black or white. The calculated air gap ratio is temporarily stored in a predetermined storage area of ​​the main storage unit 102 and transferred to the display processing unit 13. Note that the image processing unit 12 may use multiple consecutive frame images to calculate the air gap ratio. For example, from the air gap ratios calculated for each of five consecutive frames, the frames with the maximum and minimum air gap ratios may be excluded, and the air gap ratios calculated for the remaining three frames may be averaged. By calculating the air gap ratio using multiple consecutive frame images, it is possible to calculate the air gap ratio with a certain degree of accuracy even when, for example, camera shake or tilt occurs.

[0043] The display processing unit 13 displays the calculated aperture ratio on the display device 10b such as an LCD connected via the input / output unit 104. The calculated air-open ratio may be notified as a voice message via a speaker 10d or the like connected to the planter 10. The calculated air-open ratio can be notified to the worker Z1 whose line of sight is directed toward the branches, leaves 21, or cultivation shelves 20 via a voice message.

[0044] The notification processor 14 issues a notification to ensure that the horizontal angle of the opening surface of the imaging unit of the imaging camera 10a with respect to the ground 23 does not deviate from a predetermined range, based on the inclination of the device with respect to the vertical direction detected via the sensor 10c, etc. For example, if the inclination of the device with respect to the vertical direction detected via the sensor 10c, etc., deviates from the predetermined range, the notification processor 14 issues a voice message via the speaker 10d, etc. connected via the input / output unit 104. For example, a voice message such as "The camera is tilted. Please level the camera" is issued to the worker Z1.

[0045] (Processing flow) The processing of the image processing device 10 according to this embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing an example of the processing flow related to calculation of the aperture ratio. Fig. 6 is a flowchart showing an example of the processing flow for notifying the tilt of the imaging camera.

[0046] In FIG. 5, after the start of processing, the image processing device 10 acquires a captured image Z2 captured via the imaging camera 10a (step S1), and the processing proceeds to step S2. As shown in FIG. 3, the imaging camera 10a, positioned at a predetermined height, captures an image of a cultivation shelf 20, on which fruits, branches, and leaves 21 of a cultivated variety are growing, in a field where a brightness index is measured, at a predetermined frame rate, as viewed from above. In this embodiment, the control unit of the image processing device 10 that executes the processing of step S1 corresponds to an example of an "acquisition unit." In step S2, the image processing device 10 converts the color elements of each pixel constituting the acquired frame image from a color space represented by (R, G, B) to an HSV color space represented by elements such as hue, saturation, and brightness, and the processing proceeds to step S3. Through the processing of step S2, the color elements of the frame image captured by the imaging camera 10a are represented as hue information representing a 360-degree angle, with red representing 0 degrees and yellow, green, blue, and red returning. Furthermore, saturation, which indicates the vividness of a color, and lightness, which indicates the brightness of a color, are each expressed as numerical information in the range of 0 to 100 percent. In this embodiment, the control unit of the image processing device 10 that executes the processes of steps S2 to S8 corresponds to an example of a "discrimination unit."

[0047] In step S3, the color type for binarization is determined based on the brightness information of the pixel (bright spot) converted into the HSV color space. For example, if the brightness of the bright spot to be processed is equal to or greater than a first threshold (step S3, "Yes"), the image processing device 10 determines that the color type of the bright spot is white, and the process proceeds to step S7; otherwise (step S3, "No"), the process proceeds to step S4. Here, the first threshold can be, for example, a brightness value of 90 percent.

[0048] In step S4, the color type for binarization is determined based on the hue information of the bright spot converted into the HSV color space. If the hue of the bright spot to be processed belongs to the first range (step S4, "Yes"), the image processing device 10 determines that the color type of the bright spot is black, and the process proceeds to step S8; if not (step S4, "No"), the process proceeds to step S5. Here, the first range can be, for example, a range in hue angle value from 0 degrees to 177.7 degrees.

[0049] In step S5, for the bright spot determined in step S4 as having a hue angle value that does not belong to the first range, the color type for binarization processing is determined again based on the hue information. For example, if the hue angle value of the bright spot to be processed belongs to the second range (step S5, "Yes"), the image processing device 10 determines that the color type of the bright spot is white, and the processing continues. If not (step S5, "No"), the process proceeds to step S6. Here, the second range can be exemplified as a range in which the angle value of the hue is from 183.4 degrees to 240.8 degrees, for example.

[0050] In step S6, for bright spots determined to have hue angles that do not fall within either the first or second range, the color type for binarization is again determined based on the brightness value. For example, if the brightness value of the bright spot to be processed is equal to or greater than the second threshold (step S6, "Yes"), the image processing device 10 determines that the color type of the bright spot is white, and the process proceeds to step S7; otherwise (step S6, "No"), the process proceeds to step S8. Here, the second threshold can be, for example, a brightness value of 2%.

[0051] In the image processing device 10, by the processing of steps S3 to S6, it is determined whether the color element of each pixel that constitutes the frame image after conversion into the HSV color space is a bright point that belongs to either the white or black color type.

[0052] In step S7, bright points (pixels) to be processed that have been determined to belong to the white color category in the processes of steps S3 to S6 are filled in white. Similarly, in step S8, bright points (pixels) to be processed that have been determined to belong to the black color category in the processes of steps S3 to S6 are filled in black. After steps S7 and S8, the process proceeds to step S9.

[0053] In step S9, the air-space openness ratio in the frame image that has been binarized to black or white is calculated from the proportion of bright points determined to belong to the white color type. In this embodiment, the control unit of the image processing device 10 that executes the process of step S9 corresponds to an example of a "calculation unit." The image processing device 10 temporarily stores the calculated air-space openness ratio in a predetermined storage area of ​​the main storage unit 102, and proceeds to the process of step S10. In step S10, the calculated air-space openness ratio is displayed on the display device 10b, such as an LCD, connected via the input / output unit 104. The calculated air-space openness ratio is also notified as a voice message via the speaker 10d, etc., connected via the input / output unit 104. In this embodiment, the control unit of the image processing device 10 that executes the process of step S10 corresponds to an example of a "first notification unit." After the process of step S10, this routine is temporarily terminated.

[0054] In step S9, the image processing device 10 may calculate the air openness ratio using multiple consecutive frame images. For example, the air openness ratios calculated for each of five consecutive frames may be calculated by excluding the frames with the maximum and minimum air openness ratios and averaging the air openness ratios calculated for the remaining three frames. By using the air openness ratios calculated for multiple consecutive frame images in calculating the air openness ratio, it is possible to provide an air openness ratio with a certain degree of accuracy even when camera shake or tilt occurs, for example. Furthermore, by appropriately setting the number of frame images used to calculate the air openness ratio depending on, for example, the location of the field, the type of cultivated variety, weather, etc., it is possible to provide an air openness ratio that corresponds to these differences.

[0055] Next, FIG. 6 will be described. In FIG. 6, after starting the process, the image processing device 10 acquires a detection value for the vertically downward direction of the device from the sensor 10c connected via the input / output unit 104 (step S11), and the process proceeds to step S12. In step S12, based on the acquired detection value, it is determined whether the tilt of the opening surface of the imaging unit of the imaging camera 10a with respect to the horizontal plane is within a predetermined range. Here, the horizontal plane is a plane perpendicular to the vertical direction. An example of the predetermined range is a tilt range of ±1.5 degrees with respect to the horizontal plane. However, such a range may be set appropriately based on the viewing angle of the imaging camera 10a, the height position of the camera, etc.

[0056] In step S12, if the sensor detection value is within a predetermined range (step S12, “Yes”), the orientation of the imaging camera 10 is determined to be within the normal range, and the processing of this routine is temporarily terminated. On the other hand, if the sensor detection value is not within the predetermined range (step S12, “No”), the processing proceeds to step S13. In step S13, a predetermined voice message is output through the speaker 10d connected via the input / output unit 104. For example, to an operator Z1 who is measuring the brightness of a farm field while holding the image processing device 10 in one hand, a voice message such as “The camera is tilted. Please level the camera” is output. In this embodiment, the control unit of the image processing device 10 that executes the processing of steps S11 to S13 corresponds to an example of a “second unit.” When measuring the brightness of a farm field, the operator Z1, whose line of sight is directed toward tree branches, leaves 21, or cultivation shelves 20, can be reminded to take appropriate images when measuring the openness ratio. After the processing of step S13, the routine is temporarily terminated.

[0057] Through the above processing, the image processing device 10 according to this embodiment can capture, via the imaging camera 10a, video images of the current state of a field where crops, fruit trees, and the like are grown, captured at a predetermined frame rate. The image processing device 10 can convert the color elements of each pixel in each captured frame from a color space represented by (R, G, B) to a hue, saturation, value (HSV) color space, which represents elements such as hue, saturation, and brightness. The image processing device 10 then performs binarization processing based on the color space information represented by HSV, and can calculate the sky openness ratio in the captured image Z2, i.e., the proportion of pixel areas corresponding to the sky. As a result, the image processing device 10 according to this embodiment can provide a technology that can easily measure the current brightness state of a field.

[0058] Furthermore, the image processing device 10 according to this embodiment can display the calculated cavity open ratio on a display device 10b such as an LCD connected via the input / output unit 104. Similarly, the calculated cavity open ratio can be notified as a voice message via a speaker 10d or the like. Usability is improved because the calculated cavity open ratio can be notified via a voice message to the worker Z1 whose line of sight is directed towards the branches, leaves 21, or cultivation shelves 20.

[0059] Furthermore, the image processing device 10 according to this embodiment can notify the operator Z1 that the inclination of the opening surface of the imaging unit of the imaging camera 10a relative to the horizontal plane does not deviate from a predetermined range, based on the inclination of the device relative to the vertical direction detected via the sensor 10c, etc. When measuring an index showing the brightness of the field, the operator Z1, whose line of sight is directed toward the tree branches, leaves 21, or cultivation shelves 20, can be made aware of the need to take appropriate images when measuring the openness ratio.

[0060] <Comparison with conventional methods> 7 is a diagram illustrating the correlation between the brightness measured by the image processing device 10 according to the present embodiment and the results of conventional image analysis. In Fig. 7, the vertical axis represents the degree of brightness calculated by conventional image analysis, and the horizontal axis represents the degree of brightness calculated by the image processing device 10. As shown in graph g1 of Fig. 7, a good correlation was obtained between the openness ratio (%) calculated by performing binarization processing based on color space information expressed in HSV and the brightness level (%) based on image analysis. Linear approximation was performed using the openness ratio calculated using the image processing device 10 of this embodiment as variable x and the brightness level based on conventional image analysis as variable y, and the coefficients a and b for the linear approximation equation y = ax + b were identified using the least squares method. As a result, the following approximate equation (1) was obtained, as an example. Approximate formula (1): y=0.9547x-0.0066 The mean square error (R 2 ) is obtained as R 2 =0.9796, which means that we were able to obtain measurement accuracy equivalent to the degree of brightness based on image analysis.

[0061] <Other embodiments> The above-described embodiment is merely an example, and the present disclosure may be modified as appropriate within the scope of the present disclosure. The processes and means described in the present disclosure may be freely combined and implemented as long as no technical contradiction occurs.

[0062] Furthermore, a process described as being performed by one device may be shared and executed by multiple devices. Alternatively, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) by which each function is realized can be flexibly changed.

[0063] The present disclosure can also be realized by providing a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer via a non-transitory computer-readable storage medium connectable to the computer's system bus or via a network. Non-transitory computer-readable storage media include any type of medium suitable for storing electronic instructions, such as any type of disk, including magnetic disks (e.g., floppy disks, hard disk drives (HDDs), etc.), optical disks (e.g., CD-ROMs, DVDs, Blu-ray disks), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, or optical cards. [Explanation of symbols]

[0064] 10 Image processing device 10a Imaging camera 10b Display Device 10c Sensor 10d speaker 11. Image acquisition unit 12 Image processing section 13 Display processing section 14 Notification processing section 15 Threshold Information Database 16 Openness Rate Database

Claims

1. an acquisition unit that acquires a photographed image of the state of a field including a plant to be cultivated; a discrimination unit that discriminates the type of color element to which a pixel belongs based on hue information and brightness information of the pixel that constitutes the captured image, and binarizes the type of color element to which the pixel belongs as white or black; a calculation unit that calculates an index indicating brightness of the farm field from the ratio of pixels that are determined to be white among pixels that constitute the captured image; a notification unit that notifies by voice so that the posture of the imaging camera that captures the captured image is maintained in a normal posture; An image processing device comprising:

2. The image processing device according to claim 1 , wherein the captured images are moving images captured from a viewpoint looking from below toward above the plant to be cultivated.

3. 3. The image processing device according to claim 1, wherein the determining unit converts RGB values ​​of pixels that make up the captured image into a color space represented by information on hue, saturation, and brightness of the pixels.

4. The image processing device according to claim 1 , wherein the discrimination unit discriminates the type of a color element to which a pixel belongs as white when brightness information of the pixel constituting the captured image is equal to or greater than a first threshold value.

5. 4. The image processing device according to claim 1, wherein when the brightness information of a pixel constituting the captured image is less than a first threshold, the discrimination unit discriminates the type of color element to which the pixel belongs as black, provided that the hue information of the pixel belongs to a first range.

6. 4. The image processing device according to claim 1, wherein when the brightness information of a pixel constituting the captured image is less than a first threshold, the discrimination unit discriminates the type of color element to which the pixel belongs as white, provided that the hue information of the pixel belongs to a second range.

7. the determining unit determines whether brightness information of pixels constituting the captured image is less than a first threshold value, and 4. The image processing device according to claim 1, wherein, when the hue information of the pixel does not belong to the first range or the second range, the type of color element to which the pixel belongs is determined to be white, provided that the lightness information of the pixel is equal to or greater than a second threshold.

8. 4. The image processing device according to claim 1, wherein when the brightness information of a pixel constituting the captured image is less than a first threshold and the hue information of the pixel does not belong to either a first range or a second range, the discrimination unit discriminates the type of color element to which the pixel, whose brightness information does not satisfy the condition of being equal to or greater than a second threshold, belongs as black.

9. An image processing device described in any one of claims 1 to 8, wherein the captured image is a moving image captured at a predetermined frame rate, and an index indicating brightness in the field is calculated from multiple consecutive frame images.

10. The image processing device according to claim 1 , further comprising a first notification unit that notifies by voice of an index that indicates brightness in the farm field, the index being calculated from the captured image.

11. The image processing device according to claim 1 , further comprising an imaging camera that captures the captured image.

12. The image processing device according to claim 1 , wherein the image processing device is configured separately from an imaging camera that captures the captured image, and the acquisition unit acquires the captured image being captured by the imaging camera via a communication interface.

Images