Image determination method and image determination apparatus

Abstract

To improve accuracy of a growth diagnosis of plants by using an image.SOLUTION: An image determination method comprises processing of: acquiring image data to be obtained by photographing a plant installed with a slender member having a slender shape; detecting an image area corresponding to the slender member from the acquired image data; detecting an angle formed with respect to a vertical direction by the slender member by image processing performed on the detected image area; and determining whether or not the image data is photographed in a windless state on the basis of the detected angle to output a determination result.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to an image determination method and an image determination device.

Background Art

[0002] In recent years, the population of fruit tree producers has been decreasing, and an improvement in productivity is demanded. In order to improve productivity, for example, machine learning and the like are utilized to obtain information on fruit trees from images, and the development of effective production technologies has been advanced.

[0003] Specifically, for example, the use of a plurality of optical filters by switching and imaging a crop in a plurality of observation wavelength ranges, and estimating the normalized difference vegetation index, which is a growth index of a plant, from the obtained image have been studied. Also, calculating a plurality of index values from an image of a crop and diagnosing the growth state of the crop based on the plurality of index values have been studied.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the growth diagnosis of plants using images, there is a problem that the accuracy of analysis and estimation may decrease depending on the images used. That is, when diagnosing the growth state of a plant, for example, if an image of a state where the leaves are swaying due to wind is used, the swaying of the leaves becomes an obstacle and it is difficult to perform accurate analysis and estimation.

[0006] To obtain images in windless conditions, one might consider methods such as measuring wind speed with an anemometer while taking images and selecting those taken during windless conditions. However, this is not easily implemented due to the increased cost associated with using an anemometer and the difficulty in accurately synchronizing the wind speed measurement with image capture.

[0007] This disclosure is made in view of the above, and aims to provide an image determination method and an image determination device that can improve the accuracy of plant growth diagnosis using images. [Means for solving the problem]

[0008] According to one aspect of the present disclosure, the image determination method includes acquiring image data obtained by photographing a plant on which an elongated member having an elongated shape is installed, detecting an image region corresponding to the elongated member from the acquired image data, detecting the angle that the elongated member makes with respect to the vertical direction by image processing on the detected image region, determining whether or not the image data was taken in a windless state based on the detected angle, and outputting the determination result.

[0009] Furthermore, according to one aspect of the present disclosure, the image determination device includes a memory and a processor connected to the memory, the processor acquires image data obtained by photographing a plant on which an elongated member having an elongated shape is installed, detects an image region corresponding to the elongated member from the acquired image data, detects the angle that the elongated member makes with respect to the vertical direction by image processing on the detected image region, and performs a process to determine whether or not the image data was taken in a windless state based on the detected angle. [Effects of the Invention]

[0010] The image recognition method and image recognition apparatus disclosed herein can improve the accuracy of plant growth diagnosis using images. [Brief explanation of the drawing]

[0011] [Figure 1]Figure 1 is a block diagram showing the configuration of an image determination device according to one embodiment. [Figure 2] Figure 2 is a diagram illustrating the capture of image data. [Figure 3] Figure 3 shows a specific example of marker detection. [Figure 4] Figure 4 shows a specific example of contour extraction. [Figure 5] Figure 5 shows a specific example of principal component analysis. [Figure 6] Figure 6 is a flowchart showing an image determination method according to one embodiment. [Figure 7] Figure 7 shows a specific example of the angle detection result. [Figure 8] Figure 8 shows another specific example of the angle detection results. [Figure 9] Figure 9 shows a modified example of the marker. [Figure 10] Figure 10 is a block diagram showing an example of the hardware configuration of an image determination device. [Modes for carrying out the invention]

[0012] An embodiment of the present disclosure will be described below with reference to the attached drawings. The embodiment described below is illustrative and should not be interpreted as limiting.

[0013] Figure 1 is a block diagram showing the configuration of an image determination device 100 according to one embodiment. The image determination device 100 shown in Figure 1 includes an image data input unit 110, a marker detection unit 120, a contour extraction unit 130, a principal component analysis unit 140, a windless determination unit 150, and a determination result output unit 160.

[0014] The image data input unit 110 receives the input of image data obtained by photographing fruit trees. At this time, the image data input unit 110 receives the input of image data of a fruit tree provided with a marker for detecting the state of the wind. Specifically, the image data is obtained, for example, as shown in FIG. 2, by the camera C photographing the fruit tree T provided with the marker 201. The marker 201 is an elongated member having an elongated shape such as a string shape, a cylindrical shape or a flag shape formed of a lightweight material such as yarn, paper or cloth, and one end thereof is fixed to a branch of the fruit tree T or the like. Therefore, the marker 201 is deformed or changes its posture in response to the wind with the fixed end as a fulcrum. The marker 201 is preferably colored with a color that does not resemble the color in the orchard, such as red or orange, so that it can be easily detected from the image data.

[0015] The marker detection unit 120 detects an image area corresponding to the marker 201 from the input image data. Specifically, the marker detection unit 120 executes threshold determination regarding the R (red) component, the G (green) component, and the B (blue) component for each pixel constituting the image data, for example, and binarizes the image data to detect the image area corresponding to the marker 201. Thereby, the marker detection unit 120 detects the image area corresponding to the marker 201 in the image data shown in the upper figure of FIG. 3, for example, as the pixel group 202 shown in the lower figure of FIG. 3.

[0016] The contour extraction unit 130 extracts the contour of the pixel group 202 detected by the marker detection unit 120, and detects the angle of the marker 201 with respect to the vertical direction based on the shape of the extracted contour. Specifically, the contour extraction unit 130 detects an adjacent and continuous pixel group from the pixel group 202, and extracts a rectangular contour, for example, that includes each detected pixel group. That is, as shown in FIG. 4, for example, the contour extraction unit 130 detects adjacent and continuous pixel groups, and generates the smallest rectangles 211, 212, and 213 that include each pixel group.

[0017] Then, the contour extraction unit 130 selects the contour with the largest area among the extracted contours, and detects the angle formed by the longitudinal direction of the selected contour with respect to the vertical direction. Therefore, for example, in the example shown in FIG. 4, the contour extraction unit 130 selects the rectangle 211 with the largest area, and detects the angle formed by the longitudinal direction of this rectangle 211 with respect to the vertical direction. The angle detected by contour extraction in this way corresponds to the posture of the marker 201, and if there is no wind during the shooting of the image data, the angle is close to 0 degrees.

[0018] The principal component analysis unit 140 performs principal component analysis on the pixel group 202 detected by the marker detection unit 120, and detects the angle of the marker 201 with respect to the vertical direction based on the straight line indicating the feature amount of the pixel group 202. Specifically, the principal component analysis unit 140 derives a straight line that approximates the distribution of the pixel group 202 by principal component analysis. That is, as shown in FIG. 5 for example, the principal component analysis unit 140 derives a straight line 221 such that the sum of the distances to each pixel of the pixel group 202 is minimized.

[0019] Then, the principal component analysis unit 140 detects the angle formed by the derived straight line 221 with respect to the vertical direction. The angle detected by principal component analysis in this way corresponds to the posture of the marker 201, and if there is no wind during the shooting of the image data, the angle is close to 0 degrees.

[0020] The windless condition determination unit 150 determines whether the image data including the image region of the marker 201 was captured in windless conditions, based on the angles detected by the contour extraction unit 130 and the principal component analysis unit 140. Specifically, the windless condition determination unit 150 determines whether the difference between the angle detected by the contour extraction unit 130 and the angle detected by the principal component analysis unit 140 is within a predetermined tolerance range. If the difference in angles is within the predetermined tolerance range, the windless condition determination unit 150 then determines whether each of these angles is below a predetermined threshold. If both angles are below the predetermined threshold, the windless condition determination unit 150 determines that the image data was captured in windless conditions. In this way, the windless condition determination unit 150 determines that the image data was captured in windless conditions when the angles of the marker 201 detected by contour extraction and principal component analysis are approximately equal and close to 0 degrees.

[0021] The judgment result output unit 160 outputs the judgment result from the windless judgment unit 150. Specifically, the judgment result output unit 160 displays the image data along with the judgment result of whether or not the image data was taken in windless conditions, for example, on a display or prints it on a predetermined sheet of paper.

[0022] Next, the image determination method using the image determination device 100 configured as described above will be explained with reference to the flowchart shown in Figure 6.

[0023] When using images to diagnose the growth of fruit trees, the fruit trees on which the marker 201 is placed are photographed and image data is acquired. The acquired image data is then input to the image data input unit 110 (step S101). The marker detection unit 120 then detects the image region corresponding to the marker 201 from the input image data (step S102). That is, for example, a threshold determination is performed for each pixel constituting the image data, and binarization is performed to extract the pixels corresponding to the marker 201, thereby extracting the pixel group 202 corresponding to the marker 201.

[0024] The data of the pixel group 202 is output to the contour extraction unit 130 and the principal component analysis unit 140, and the angle that the marker 201 makes with respect to the vertical direction is detected by image processing. Specifically, The contour extraction unit 130 extracts contours from the pixel group 202 that encompass adjacent, continuous pixel groups (step S103). That is, as shown in Figure 4, for example, the smallest rectangles 211, 212, and 213 that enclose the continuous pixel groups are generated. The contour extraction unit 130 then selects the contour with the largest area from the extracted contours and detects the angle that the longitudinal direction of the selected contour makes with respect to the vertical direction in the image data. For example, in the example shown in Figure 4, the rectangle 211 with the largest area is selected, and the angle that the longitudinal direction of this rectangle 211 makes with respect to the vertical direction in the image data is detected. The detected angle corresponds to the angle that the marker 201 makes with respect to the vertical direction.

[0025] Furthermore, the principal component analysis unit 140 performs principal component analysis on the pixel group 202 (step S104). That is, as shown in Figure 5, for example, a straight line 221 that approximates the distribution of the pixel group 202 is derived. The principal component analysis unit 140 then detects the angle that the straight line 221 approximating the distribution of the pixel group 202 makes with respect to the vertical direction in the image data. The detected angle corresponds to the angle that the marker 201 makes with respect to the vertical direction.

[0026] Furthermore, the angle detection by the contour extraction unit 130 and the angle detection by the principal component analysis unit 140 may be performed simultaneously, or one may be performed first and the other later.

[0027] When the angle of the marker 201 is detected by the contour extraction unit 130 and the principal component analysis unit 140, the windless determination unit 150 determines whether the difference between the angle detected by the contour extraction unit 130 and the angle detected by the principal component analysis unit 140 is within a predetermined allowable range (step S105). That is, since the angle of the marker 201 is detected by two methods, contour extraction and principal component analysis, the windless determination unit 150 determines whether the angles detected by the two methods are equivalent to each other.

[0028] Here, specific examples of the angles of marker 201 detected by contour extraction and principal component analysis, respectively, will be explained with reference to Figures 7 and 8. Figure 7 shows a specific example of angle detection when the pixel group 202 includes noise pixels. Figure 7(a) shows a specific example of angle detection by contour extraction, and Figure 7(b) shows a specific example of angle detection by principal component analysis.

[0029] As shown in Figure 7(a), in contour extraction, the contours of noise pixel groups included in the pixel group 202 are also extracted, but the contour 301 with the largest area is selected, and the angle indicated by the selected contour 301 is detected. Since the contour 301 with the largest area is likely to be the image region corresponding to marker 201, contour extraction can accurately detect the angle of marker 201 even in the presence of noise.

[0030] On the other hand, as shown in Figure 7(b), in principal component analysis, a straight line 302 is derived that approximates the pixel group 202, including the noise pixels, and the angle indicated by the derived straight line 302 is detected. Therefore, according to principal component analysis, the angle of marker 201 may not be accurately detected in the presence of noise.

[0031] Next, Figure 8 shows a specific example of angle detection when the pixel group 202 corresponding to marker 201 is not sufficiently detected from the image data. Figure 8(a) shows a specific example of angle detection by contour extraction, and Figure 8(b) shows a specific example of angle detection by principal component analysis.

[0032] As shown in Figure 8(a), in contour detection, since the pixel group 202 does not contain adjacent, continuous groups of sufficiently large pixels, the contour 311 with the largest area is also relatively small, and the angle represented by this contour 311 is detected. Although the contour 311 with the largest area corresponds to an image region of a part of the marker 201, it is highly likely that it does not reflect the orientation of the entire marker 201. Therefore, according to contour detection, if the pixel group 202 is not sufficiently detected, the angle of the marker 201 may not be accurately detected.

[0033] On the other hand, as shown in Figure 8(b), in principal component analysis, a straight line 312 is derived that approximates the pixel group 202 which does not contain a sufficient number of pixels, and the angle indicated by the derived straight line 312 is detected. Although the pixel group 202 does not contain a sufficient number of pixels, the distribution of these pixels is likely to reflect the orientation of the marker 201. Therefore, according to principal component analysis, the angle of the marker 201 can be accurately detected even if the pixel group 202 is not sufficiently detected.

[0034] Thus, if the detection accuracy of the pixel group 202 corresponding to the marker 201 is poor, the angles detected by contour extraction and principal component analysis will differ significantly. Therefore, the windless determination unit 150 determines whether the angles detected by contour extraction and principal component analysis are equivalent. If the difference in the angles detected by contour extraction and principal component analysis is not within a predetermined tolerance range (step S105No), it is determined that the detection accuracy of the pixel group 202 is not sufficiently good, and the determination of whether the image data was captured in windless conditions is not performed.

[0035] In contrast, if the difference in angles detected by contour extraction and principal component analysis is within a predetermined tolerance range (step S105 Yes), the detection accuracy of the pixel group 202 is determined to be sufficiently good, and a determination is made as to whether or not there is no wind based on the angles detected by contour extraction and principal component analysis (step S106). That is, the windless determination unit 150 determines whether or not the angles detected by contour extraction and principal component analysis are each below a predetermined threshold.

[0036] In this windless condition determination, if the angles detected by contour extraction and principal component analysis are each below a predetermined threshold, the image data is determined to have been taken in windless conditions. Conversely, if the angles detected by contour extraction and principal component analysis are each above a predetermined threshold, the image data is determined not to have been taken in windless conditions. The determination result output unit 160 then outputs a determination result indicating whether or not the image data was taken in windless conditions (step S107).

[0037] This makes it easy to determine whether or not the image data was taken in windless conditions, allowing for analysis and estimation of indicator values ​​using image data taken in windless conditions. As a result, the accuracy of plant growth diagnosis using images can be improved.

[0038] As described above, according to this embodiment, an image region corresponding to the marker is detected from image data of a fruit tree on which a marker is placed, and the angle of the marker is detected from this image region by contour extraction and principal component analysis. Then, it is determined whether or not the image data was taken in windless conditions based on the angle of the marker. Therefore, it becomes possible to perform analysis and estimation using image data taken in windless conditions, and the accuracy of plant growth diagnosis using images can be improved.

[0039] Furthermore, the marker 201 installed on the fruit tree may be formed by twisting together multiple string-like members. That is, for example, as shown in Figure 9(a), the marker 201 may be formed by twisting together a first string-like member 201a and a second string-like member 201b. In this case, it is preferable that the first string-like member 201a and the second string-like member 201b are different colors. This makes it possible to reliably detect the pixel group 202 corresponding to the marker 201 from the image data by performing threshold judgment according to each color.

[0040] Alternatively, the marker 201 may be attached to the other end of a rod-shaped member 401, one end of which is attached to a branch of a fruit tree, as shown in Figure 9(b). This prevents the string-like member marker 201 from becoming entangled in the branch of the fruit tree.

[0041] In the above embodiment, the image determination device 100 can be configured using a processor and memory. Figure 10 is a block diagram showing an example of the hardware configuration of the image determination device 100 according to one embodiment. As shown in Figure 10, the image determination device 100 has an input / output unit 101, a processor 102, memory 103, and storage 104.

[0042] The input / output unit 101 is an interface for the user to input information and output information to the user. The input / output unit 101 may include, for example, a keyboard, display, touch panel, microphone, or speaker. The input / output unit 101 accepts image data as input and outputs the result of the windlessness determination.

[0043] The processor 102 includes, for example, a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), or a DSP (Digital Signal Processor), and provides overall control of the image determination device 100, as well as performing various calculations.

[0044] The memory 103 includes, for example, RAM (Random Access Memory) or ROM (Read Only Memory) and stores information used for arithmetic processing performed by the processor 102.

[0045] Storage 104 includes, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various types of data.

[0046] Furthermore, the processing performed by the image determination device 100 described in the above embodiment can also be written as a program that can be executed by a computer. In this case, this program can be stored in a computer-readable and non-transitory recording medium and installed on the computer. Examples of such recording media include portable recording media such as CD-ROMs, DVD discs, and USB memory, as well as semiconductor memory such as flash memory. [Explanation of Symbols]

[0047] 101 Input / output section 102 processors 103 memory 104 storage 110 Image data input section 120 Marker detection unit 130 Contour extraction section 140 Principal component analysis section 150 Silence judgment section 160 Judgment Result Output Unit

Claims

1. A processor provided by a computer, Image data obtained by photographing a plant on which an elongated member having a long, slender shape is installed is acquired. From the acquired image data, the image region corresponding to the elongated member is detected, Image processing of the detected image region detects the angle that the elongated member makes with respect to the vertical direction. Based on the detected angle, it is determined whether or not the image data was taken in windless conditions. Output the judgment result. Having a process, The process for detecting the angle is as follows: Extract the contours of the pixel groups that make up the detected image region. The first angle indicated by the longitudinal direction of the extracted contour is detected, We derive a straight line that approximates the distribution of the pixel groups that make up the detected image region. Detect the second angle indicated by the derived line. Including processing, The process for making the determination is as follows: If the difference between the first angle and the second angle is within a predetermined tolerance range, it is determined whether or not the image data was taken in windless conditions based on the first angle and the second angle. Image recognition method including processing.

2. The process to be obtained is: Image data is obtained by photographing plants to which elongated red or orange members have been attached. The image determination method according to claim 1, including processing.

3. The aforementioned acquisition process is, Image data is obtained by photographing plants on which a twisted string-like member, formed by twisting together multiple elongated members of different colors, is attached. The image determination method according to claim 1, including processing.

4. Memory and It has a processor connected to the aforementioned memory, The aforementioned processor, Image data obtained by photographing a plant on which an elongated member having a long, slender shape is installed is acquired. From the acquired image data, the image region corresponding to the elongated member is detected, Image processing of the detected image region detects the angle that the elongated member makes with respect to the vertical direction. Based on the detected angle, it is determined whether or not the image data was taken in windless conditions. Execute the process, The process for detecting the angle is as follows: Extract the contours of the pixel groups that make up the detected image region. The first angle indicated by the longitudinal direction of the extracted contour is detected, We derive a straight line that approximates the distribution of the pixel groups that make up the detected image region. Detect the second angle indicated by the derived line. Including processing, The process for making the determination is as follows: If the difference between the first angle and the second angle is within a predetermined tolerance range, it is determined whether or not the image data was taken in windless conditions based on the first angle and the second angle. Image determination device including processing.

Images