Agricultural Information Management System

Abstract

We provide an agricultural information management system that offers high user convenience. [Solution] The server 20 includes a grouping unit 23 that extracts and groups similar image groups consisting of multiple image data having similar features from a series of image data captured by the shooting unit 10 in a time-series manner, an individual identification unit 24 that individually identifies unique objects within the images of the similar image group, and a display screen creation unit 26 that can present information on the number or proportion of objects for each similar image group as a display screen on the display unit 31.

Description

[Technical Field]

[0001] The present invention relates to an agricultural information management system. [Background Art]

[0002] In recent years, agricultural information management systems that extract and present necessary information from images captured by cameras or the like have been proposed. For example, Patent Document 1 discloses a system in which a worker who harvests fruits such as tomatoes wears smart glasses, and counts the number of harvested fruits using images captured by the smart glasses. [Prior Art Literature] [Patent Literature]

[0003] [Patent Document 1] International Publication No. 2020 / 203764 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] The state of crops at a cultivation site changes daily depending on environmental factors and human factors. For this reason, in agricultural management, it is important to frequently observe and record the conditions of agricultural fields. It is difficult to say that conventional agricultural information management systems have high convenience for users. To improve the convenience of agricultural information management systems, there is a need for a system that efficiently organizes information related to production from images obtained at cultivation sites.

[0005] The present invention has been made in view of the above problem, and an object thereof is to provide an agricultural information management system with high user convenience. [Means for Solving the Problem]

[0006] To solve the above problems, the agricultural information management system of the present invention is characterized by comprising: a grouping unit that extracts and groups similar image groups consisting of multiple image data having similar characteristics from a group of captured image data; an individual identification unit that individually identifies a unique object within the image of the similar image group; and a display unit that displays information related to the object for each of the similar image groups.

[0007] According to the above configuration, by grouping similar image sets and summarizing information for each similar image set, growth information and other details can be efficiently and precisely extracted from image information taken by workers during inspections or from image information acquired by fixed-point cameras installed in the fields, making it easier to recall information when reviewing the captured images. In particular, by identifying unique objects (such as fruits) within each similar image set (individual identification), information related to objects within each similar image set can be recognized while avoiding double detection or detection omissions of objects across different images.

[0008] Furthermore, in the above-mentioned agricultural information management system, the display unit can be configured to display the number or percentage of the target object as information related to the target object.

[0009] Furthermore, in the above-mentioned agricultural information management system, the display unit can be configured to display the ripeness trend of the object as information related to the object.

[0010] Furthermore, in the above-mentioned agricultural information management system, the display unit can be configured to display the status of physiological disorders occurring in the target object as information related to the target object.

[0011] According to the above configuration, it is possible to understand the number, proportion, maturity trends, and occurrence of physiological disorders of each target object, and to adjust the cultivation environment conditions based on this information.

[0012] Furthermore, the above-mentioned agricultural information management system can be configured to allow the number of objects to be changed by operation instructions from the operation input unit.

[0013] With the above configuration, if there is an error in the number of individually identified objects, the operator can easily correct it based on visual confirmation, and the number of objects for each group of similar images can be determined more accurately.

[0014] Furthermore, the above-mentioned agricultural information management system can be configured to exclude unnecessary image data from the image data included in each of the similar image groups based on operation instructions from the operation input unit.

[0015] According to the above configuration, if a group of similar images contains incorrectly grouped image data, excluding that image data will remove the objects within the incorrectly grouped image data from the count, allowing for a correct recount of the number of objects in that group of similar images.

[0016] Furthermore, in the above-mentioned agricultural information management system, the display unit can be configured to display information about the target object in combination with the shooting location information.

[0017] The above configuration allows us to determine the number of objects in each location within the field, making it easier to estimate personnel allocation and harvesting time during harvesting operations.

[0018] Furthermore, in the above-mentioned agricultural information management system, the display unit can be configured to display information about the object in combination with the date and time of shooting information.

[0019] Furthermore, the above-mentioned agricultural information management system can be configured such that the display unit can display information about the work performed by the worker for each group of similar images.

[0020] According to the above configuration, by grouping work performed in a farm field and presenting the information as a group of similar images, it becomes possible to refer to work results and create work records in a labor-saving manner without reviewing all captured videos.

Effects of the Invention

[0021] The agricultural information management system of the present invention groups similar image groups and summarizes information for each similar image group, thereby producing the effect that necessary information can be efficiently and carefully extracted from a large amount of image information, and it becomes easier to obtain a trigger for recollection when reviewing captured images.

Brief Description of Drawings

[0022] [Figure 1] 1 shows an embodiment of the present invention, and is a block diagram showing a schematic configuration of an agricultural information management system. [Figure 2] It is a flowchart showing processing steps in the agricultural information management system. [Figure 3] It is an explanatory diagram for explaining image grouping. [Figure 4] It is a diagram showing an example of a similar image group. [Figure 5] As a first display screen example, it is a diagram showing an example of a display screen in the agricultural information management system. [Figure 6] It is a diagram showing an example of a display screen after transitioning to a scene editing mode. [Figure 7] It is a diagram showing an example of a display screen after selecting and excluding unnecessary images in a scene. [Figure 8] It is a diagram showing an example of a display screen after selecting and excluding unnecessary scenes. [Figure 9] It is a diagram showing an example of a display screen in a representative image display format. [Figure 10] It is a diagram showing an example of a display screen when maturity is selected as attribute information. [Figure 11] It is a diagram showing an example of a display screen in which scenes including an arbitrary target object are preferentially displayed. [Figure 12]As example of a display screen, this figure shows an example of a display screen in an agricultural information management system. [Figure 13] This figure shows a modified example of display screen example 2. [Figure 14] This figure shows another variation of the display screen example 2. [Figure 15] As example of display screen 3, this figure shows an example of a display screen in an agricultural information management system. [Figure 16] This figure shows a modified example of display screen example 3. [Figure 17] As example of a display screen, this figure shows an example of a display screen in an agricultural information management system. [Figure 18] This figure shows a modified example of display screen example 4. [Figure 19] This figure shows another variation of display screen example 4. [Modes for carrying out the invention]

[0023] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Figure 1 is a block diagram showing the schematic configuration of the agricultural information management system (hereinafter referred to as "this system") according to this embodiment. As shown in Figure 1, this system is broadly composed of an imaging unit 10, a server 20, and a terminal 30. In Figure 1, the imaging unit 10 and the terminal 30 are shown as components included in this system, but the imaging unit 10 and the terminal 30 may be external elements not included in this system. That is, this system may consist only of the server 20, and the imaging unit 10 and the terminal 30 may be devices externally connected to this system. Furthermore, the server 20 and the terminal 30 may be integrated into a single unit such as a personal computer.

[0024] The camera unit 10 is a means of taking photographs in the field that is managed by this system. The camera unit 10 may be a portable camera carried by the worker when inspecting the field, or it may be a fixed camera installed in the field. The image data taken by the camera unit 10 may be a still image or a video.

[0025] The server 20 includes a main control unit 21, a storage unit 22, a grouping unit 23, an individual identification unit 24, an attribute determination unit 25, and a display screen creation unit 26. The main control unit 21 controls the entire server. The storage unit 22 stores image data captured by the imaging unit 10 and information obtained by the grouping unit 23, the individual identification unit 24, and the attribute determination unit 25 (details will be described later). The display screen creation unit 26 creates display screen data to be displayed on the display unit 31 of the terminal 30.

[0026] Terminal 30 is a terminal equipped with a display unit 31, which is a display device such as a screen, and an input unit (operation input unit) 32, which is an input device such as a keyboard or mouse, and is intended for use with personal computers, tablets, smartphones, etc. In this system, management information can be displayed on the display screen of the display unit 31, and instructions to modify the displayed information can be given through operation input from the input unit 32. In addition, multiple terminals 30 may be connected to the server 20.

[0027] The key feature of this system is its ability to efficiently and meticulously extract and organize production-related information from a large amount of captured image data (image data set), and to display it in a way that provides appropriate suggestions (triggers for recollection, minimal viewing burden) when users review the images. The following explains these features in detail.

[0028] Figure 2 is a flowchart showing the processing steps in this system. The processing steps of this system can be broadly divided into three stages: <data acquisition>, <information extraction>, and <information presentation>.

[0029] In the <Data Acquisition> process, a shooting device such as a camera or smart glasses, which constitutes the shooting unit 10, is activated (S1), and an image is captured by the shooting device (S2). The captured image data is stored in the storage unit 22, linked to additional information such as location information (shooting location information) such as the shooting position and shooting area within the field, and shooting date and time information. If the shooting device is a fixed camera, the location information linked to the image data can be set in advance for each shooting device. If the shooting device is a portable camera, it is preferable to acquire location information (and the direction the worker is facing) from a sensor attached to the shooting device at the same time as shooting, and to automatically link the acquired location information to the captured image data. However, the present invention is not limited thereto, and location information may be manually assigned to the image data after shooting.

[0030] In the <Information Extraction> process, the image data acquired in the <Data Acquisition> process is classified (grouped) by scene (S3), and the target object (such as fruit) is uniquely identified (individual identification) for each classified image group (S4). That is, if the same object appears across multiple consecutive images, that object is recognized as a single identified object. Various image judgments can be performed on the individually identified objects through image processing (S5). In addition, counting can be performed on the identified objects by similar image groups or for the entire field (S6, S7).

[0031] In the <Information Presentation> process, information is presented in a summarized form (S8) based on the image judgment and counting results from the <Information Extraction> process, thereby encouraging the user to recall, understand, and record the information. The display screen is designed to be viewed on smartphones, tablets, and personal computers, and the display layout may be set according to the display device.

[0032] Next, we will explain in more detail the processing steps in the <Information Extraction> process. For this example, we will use the case where the field managed by this system is a greenhouse for fruit (tomato) cultivation, and the object managed by this system is the fruit.

[0033] [Image grouping] In this system, for example, there is a large amount of continuous image data arranged in a time series. Furthermore, among this large amount of image data, there are many similar images that depict almost the same shooting area (at least a part of the shooting area overlaps) and include the same fruit. For this reason, this system performs grouping to create a group of similar images with similar features from the continuous images. This grouping is performed by the grouping unit 23 of server 20. Here, "similar features" refers to the similarity (similarity between vectors) when the images are viewed as a multidimensional matrix.

[0034] Grouping is performed, for example, by feature point matching using image processing. Other methods include calculating similarity using deep learning. Then, as shown in Figure 3, the images are arranged in chronological order (frame order), and similarity is calculated based on the distance between feature points in the images, and the images are grouped. If the distance between feature points in multiple images being compared is below a certain value, those images are grouped as similar images. If the captured image data is a video, it is preferable to sample the video at predetermined time intervals and perform grouping on the sampled images.

[0035] In the example in Figure 3, the images in frame (n-1) and frame (n) have different shooting areas and no similarity in feature points. Therefore, the images in frame (n-1) and frame (n) are determined not to be similar images. On the other hand, the images in frame (n) and frame (n+1) have a large overlap in shooting areas and the same fruit is captured in both. Specifically, three fruits (shown in a box in the figure) are captured near the left of the image in frame (n) and near the center of the image in frame (n+1). If we use these fruits as feature points and calculate the distance between feature points in the two images, we can determine that the images in frame (n) and frame (n+1) are similar images.

[0036] Furthermore, the images compared when determining similar images are not necessarily consecutive frames, as shown in the example in Figure 3. For example, an image with (n) frames may be compared with images with (n+2) or (n+3) frames and determined to be similar. To compare non-consecutive images in this way, possible methods include comparing a given image with other images taken within a predetermined time frame, or performing a brute-force comparison with all images taken for each operation or each day. By making it possible to determine similar images even with non-consecutive images, images taken when the gaze is momentarily averted or when the same area is viewed from the same angle can also be included in the grouping.

[0037] Furthermore, the example in Figure 3 illustrates a case where the fruit captured in the image is used as a feature point, and similar images are determined by feature point matching. However, the fruit is not necessarily the only part of the image that can be used as a feature point; other parts of the image can also be used as feature points.

[0038] By grouping similar images in this way and summarizing information for each group, growth information and other details can be efficiently and precisely extracted from images taken by workers during inspections or from images acquired by fixed-point cameras installed in the fields, making it easier to recall information when reviewing the captured images. Furthermore, individual identification by image group can be applied to purposes other than fruit identification. For example, it is possible to identify individual flowers at the flowering stage before fruit develops.

[0039] [Individual identification by image group] In a group of similar images, the same fruit may appear in multiple different images. These fruits can be recognized as the same fruit (unique fruit) within the group of similar images. That is, by identifying (individual identification) the unique fruit within each group of similar images and counting the identified unique fruits, the number of fruits within each group of similar images can be recognized while avoiding double detection or missed detections between different images. This individual identification is performed by the individual identification unit 24 of the server 20.

[0040] [Assigning attribute information through image recognition] For fruits uniquely detected within each group of similar images through grouping, attribute information for each fruit can be determined and assigned through image recognition (attribute information assignment). Examples of attribute information that can be determined include the ripeness of the fruit (ripeness determination) and the presence or absence of physiological disorders (physiological disorder determination). The determined attribute information is stored in correspondence with the similar image group that received the determination. The attribute information determination is performed by the attribute determination unit 25 of the server 20.

[0041] The ripeness of fruit can be determined by an image recognition system based on its color and size. Several types of physiological disorders can be identified, including hollow fruits, blossom-end rot, misshapen fruits, small fruits, cracked fruits, netted fruits, and poorly colored fruits. These types of physiological disorders can also be identified by the image recognition system.

[0042] In this way, by identifying and saving the attribute information of each fruit, it is possible to understand the ripening trends and the occurrence of physiological disorders in the fruits within the field, and to adjust the cultivation environment conditions based on this information. It is possible. Furthermore, attribute information determination can be applied to things other than fruit. For example, it is possible to determine whether a flower is normal or abnormal (flower attribute information) at the flower stage before fruit develops, and then determine which flowers should be removed based on the determination results.

[0043] [Counting of objects] This system enables accurate fruit counting (avoiding duplicate detection and missed detections) by identifying unique fruits through grouping. For example, in the group of four similar images shown in Figure 4, three or two unique fruits (shown in a box in the figure) are detected in each image. In this system's counting, these fruits are not counted as a total of 10 across the four images, but rather as a total of three unique fruits. Specifically, in the four images in Figure 4, three fruits are detected in the top left and bottom left images, and two fruits are detected in the top right and bottom right images. This indicates that fruits were correctly detected in the top left and bottom left images, but missed in the top right and bottom right images. In this case, even if some images within the group of similar images are missed, if they are correctly detected in other images, those fruits will be detected as unique fruits within the group of similar images, thus avoiding missed detections within the group of similar images.

[0044] In this way, by identifying unique fruits within a group of similar images, it is possible to perform highly accurate counting by avoiding duplicate counting or omissions of the same fruit in different images. Furthermore, this counting can be performed not only for each group of similar images, but also by adding up the counts from each group to perform a total count for the entire field. With this system, by counting fruits for each group of similar images, the number of fruits in each area of ​​the field can be determined, making it easier to estimate personnel allocation, harvesting time, and harvest yield (number of marketable fruits) during harvesting.

[0045] [Example of display screen in this system 1: Example of display of shooting results] Figure 5 shows an example of a display screen in this system. This display screen allows information to be presented for each group of similar images. In other words, scenes 1-6 in Figure 5 correspond to different groups of similar images. For each group of similar images, the number of uniquely identified fruits is displayed. For example, the number of fruits in the similar image group of scene 1 is 12, and the number of fruits in the similar image group of scene 2 is 3.

[0046] Furthermore, summing the number of fruits in all similar image groups will give the total number of fruits in the entire field. In the example in Figure 5, there are 20 similar image groups (Scenes 1-20) in the entire field, and the total number of fruits in the entire field is 100. Note that in the display screen of Figure 5, only Scenes 1-6 of Scenes 1-20 are displayed, but Scenes 7 and beyond can be displayed by scrolling the screen.

[0047] Furthermore, the fruits detected by this system are assigned attribute information determined by image recognition. One possible method for displaying this attribute information is to display icons indicating what type of attribute information the fruits containing are in each group of similar images. In the example in Figure 5, physiological disorders are selected as the attribute information in the attribute information switching unit 51. In this case, each scene displays an icon indicating the type of physiological disorder of the fruits contained in the scene. In the example in Figure 5, this icon display indicates that hollow fruits are included in the fruits in scene 1, and that misshapen fruits and blossom-end rot fruits are included in the fruits in scene 5.

[0048] Furthermore, if there are unnecessary images (images that have been incorrectly grouped) in each scene, the system operator can manually edit the images and select and exclude the unnecessary images. To exclude unnecessary images, click within the frame of the scene you want to edit. You can switch to edit mode by clicking icon button 52, for example. Figure 6 shows an example of the display screen when switching to edit mode for Scene 1. In Figure 6, the edit screen of Scene 1 is shown enlarged on the right side of the screen (screen enclosed in a thick border), but displaying the enlarged edit screen is not mandatory.

[0049] In the editing screen shown in Figure 6, all images classified as Scene 1 are displayed, and the operator can select which images to exclude. For example, a checkbox is provided in the upper right corner of each image, and checking this checkbox selects the image to be included in the scene (if the checkbox is not checked, the image is selected to be excluded from the scene). In the example in Figure 6, five images are classified as Scene 1, but the bottom image in the left column is an unnecessary image that was mistakenly classified as Scene 1. Therefore, by deleting the checkbox for the bottom image in the left column and leaving the checkboxes for the other four images, only the unnecessary images can be excluded from Scene 1.

[0050] When you recalculate after selecting and excluding unnecessary images using the checkmark (by clicking the recalculate button 53 in the editing screen), the fruit count in the scene will be recalculated using only the four images selected as images included in the scene. Figure 7 shows the display screen after recalculation from the display screen of Figure 6. In the display screen of Figure 7, the number of fruits included in Scene 1 has decreased from 12 before recalculation to 10 after excluding unnecessary images from Scene 1 and performing a recalculation. In addition, the total number of fruits in the field has decreased from 100 before recalculation to 98.

[0051] Furthermore, in the editing screen shown in Figure 6, the number of fruits included in Scene 1 (12) is displayed in the upper left corner of the editing screen. The operator can also visually count the number of fruits from the displayed image of Scene 1 and directly input the counted number of fruits (by changing the number of fruits displayed in the upper left corner of the editing screen). Additionally, clicking the reset button 54 on the editing screen will reset the recalculated (or changed) number of fruits to its initial value.

[0052] Thus, by providing an editing mode, this system allows for the exclusion of images that are not originally intended for aggregation from the aggregation if they are included in a group of similar images. This makes it easier for operators to make corrections based on visual confirmation, and allows for a more accurate determination of the number of fruits for each group of similar images. Furthermore, in the display screen after editing, images that have been selected and excluded from the scene may be displayed differently from other images (for example, displayed lighter than other images), as shown in Figure 7.

[0053] Furthermore, in this system, the operator may be able to exclude not only images within a scene, but also the entire scene. For example, in the display screen of Figure 5, a checkbox is provided in the upper right corner of each scene's frame, and scenes with a checkmark in this checkbox are selected. In the example in Figure 5, scenes 1, 2, and 5 are selected, while scenes 3, 4, and 6 are excluded. This is because the total number of fruits in the field is calculated by summing the number of fruits in the selected scenes, and scenes 3, 4, and 6 are excluded because they have 0 fruits and therefore do not affect the total number of fruits in the field.

[0054] Then, by unchecking the checkbox, the entire scene can be excluded, and the total number of fruits in the field can be recalculated. Figure 8 shows the display screen after excluding Scene 1 from the display screen of Figure 5. In Figure 8, because the entire Scene 1 has been excluded, the number of fruits in Scene 1 is not included, and the total number of fruits in the field has decreased from 100 before recalculation to 88. Such scene exclusion can be performed when the grouping results are unreliable (for example, when the same fruits are clearly shown in Scene 1 and Scene 2).

[0055] Furthermore, the system's display screen can also present attribute information in an easy-to-understand manner. For example, when searching for scenes where physiological disorders are occurring, the operator can set the type of physiological disorder, and the scenes containing that disorder can be highlighted. For example, in the example in Figure 9, the operator selects "hollow fruit" in the attribute type selection unit 55 to search for scenes containing hollow fruit. As a result of this selection, scenes containing hollow fruit, in this example, Scene 1, are highlighted (e.g., surrounded by a thick border). In this way, by highlighting similar image groups containing any given object in an identifiable manner, it becomes easier to search for and track the growth status of fruit in the field.

[0056] Furthermore, in the display screen shown in Figure 5, all images included in each scene are displayed (full image display format). In contrast, to make the display easier to see, a display format that shows only representative images in each scene (representative image display format) may be used, as shown in Figure 9. In the representative image display format, representative images are selected according to predetermined criteria, and it is preferable that these criteria be set arbitrarily by the operator. Possible criteria for selecting representative images include, for example, prioritizing the selection of images that contain many fruits or images that contain fruits with physiological disorders. In addition, the display format on the display screen (full image display format or representative image display format) may be arbitrarily switched by the operator. For example, a display format switching button 56 (see Figure 9) could be provided on the display screen, and the display format could be switched each time this display format switching button 56 is clicked.

[0057] If there are any noteworthy fruits (such as fruits with physiological disorders) in the image displayed on the screen based on the image analysis results, those fruits may be highlighted by surrounding them with a frame (see Figure 9). In this case, changing the color of the frame according to the type of analysis result (type of physiological disorder) will make it easier to identify the type of analysis result.

[0058] In the display screen of Figure 5, physiological disorders are selected as the attribute information, but it is possible to switch to a display with other attribute information selected by switching the selection in the attribute information switching unit 51. Figure 10 is a diagram showing an example of the display screen when ripeness is selected as the attribute information in the attribute information switching unit 51. In this case, the ripeness of the fruits contained in each scene is displayed as an icon. In the example of Figure 10, this icon display indicates that Scene 1 contains fruits with ripeness (JD) 5 and 4, Scene 2 contains fruits with ripeness 5, and Scene 5 contains fruits with ripeness 4. In this case as well, an attribute type can be selected in the attribute type selection unit 55, and the scenes containing the selected attribute type can be highlighted. In the example of Figure 10, by selecting fruits with ripeness 5, Scenes 1 and 2 containing them are highlighted.

[0059] Furthermore, the system's display screen can prioritize the display of scenes containing a desired object based on predetermined rules. For example, Figure 11 shows a display screen where scenes 1, 2, and 5, which contain fruit, are grouped at the top (prioritized display), and scenes 3, 4, and 6, which do not contain fruit, are displayed at the bottom, according to a rule that prioritizes the display of scenes containing the target object. This priority display allows scenes containing any desired object to be initially displayed in a more easily viewable position. The predetermined rules for priority display are not particularly limited; for example, scenes containing physiologically damaged fruit can also be prioritized. In addition, multiple rules for priority display can be prepared, and the operator can selectively set any rule to perform priority display.

[0060] [Example of display screen in this system 2: Example of display using area information] Display screen example 2 shows an example of a display screen when displaying information using area information within the field. As already explained in the <Data Acquisition> process, the captured image data is stored in the storage unit 22, linked to location information such as the shooting location and shooting area within the field. As a result, as shown in Figures 12 and 13, it is possible to select and display scenes belonging to a specific area using the location information within the field at the time of shooting. In the examples of Figures 12 and 13, an area selection unit 61 is provided on the left side of the display screen, and the display area can be selected in this area selection unit 61 (area A is selected in Figure 12, and area D is selected in Figure 13).

[0061] The scene selection unit 62 is a means for selecting which scene to display when the selected display area contains multiple scenes. In Figure 12, the scene selection unit 62 has selected to display scene 2, and in Figure 13, the scene selection unit 62 has selected to display scene 20. The display format selection unit 63 is a means for selecting the display format of the images included in the selected scene. In Figure 12, the display format selection unit 63 has selected a format that displays all scene images in scene 2 (i.e., the all-image display format), and the four captured images included in scene 2 are displayed. In Figure 13, a format that displays a representative image in scene 2 (i.e., the representative image display format) has been selected.

[0062] The display screens in Figures 12 and 13 can also display the number of fruits in the selected scene (3 in Figure 12, 10 in Figure 13) and attribute information. The attribute information selection units 64 and 65 are means for selecting which attribute information to display for the fruits in the selected scene. In Figures 12 and 13, ripeness is selected in the attribute information selection unit 64, and physiological disorders are selected in the attribute information selection unit 65. Below the attribute information selection unit 64, the number and percentage of fruits classified according to ripeness are displayed in a table (see Figure 12) or graph (see Figure 13). Below the attribute information selection unit 65, the number and percentage of fruits classified according to physiological disorders are displayed in a table or graph. Note that the content displayed in the table or graph is not particularly limited, and only the number or only the percentage may be displayed.

[0063] Figure 14 shows an example of a display screen, different from those shown in Figures 12 and 13, as Display Screen Example 2. In the display screen of Figure 14, two areas, Area A and Area D, are selected, and information for both areas is displayed. By enabling the simultaneous display of information for multiple areas on a single display screen in this way, it becomes easier to compare information for multiple areas.

[0064] [Example of display screen in this system 3: Example of display using shooting date and time information] Display screen example 3 shows an example of a display screen when displaying information using the date and time of shooting. As already explained in the <Data Acquisition> process, the captured image data is stored in the storage unit 22 linked to the date and time of shooting information. This makes it possible to aggregate and display information for a specific period using the date and time of shooting information, as shown in Figure 15. In the example in Figure 15, a period selection unit 71 is provided in the upper left of the display screen, and the period for which information will be displayed can be selected in this period selection unit 71 (in Figure 15, the period of July 2021 is selected). To the right of the period selection unit 71, the total number of fruits in the field during the selected period (display target period) is displayed graph (or table) for each day (in chronological order).

[0065] The attribute information selection units 72 and 73 are means for selecting which attribute information to display during the selected period. In Figure 15, ripeness is selected in the attribute information selection unit 72, and physiological disorders are selected in the attribute information selection unit 73. To the right of the attribute information selection units 72 and 73, a graph (or table) is displayed showing the percentage of fruits classified according to ripeness and the percentage of fruits classified according to physiological state during the selected period.

[0066] Figure 16 shows an example of a display screen different from Figure 15, as Display Screen Example 3. In the display screen of Figure 16, a specific date within the display period is selected in the date selection unit 74 (the 17th is selected in Figure 16). In this case, the selected date becomes the display target day, and the number of fruits in each scene (number of fruits per scene) on the display target day can be displayed in a graph (or table). It is also possible to display the total number of fruits for the display target day, which is the sum of the number of fruits per scene. Although not illustrated here, depending on the settings of this system, it is also possible to display information based on attribute information for the display target day. For example, it is possible to display the percentage of fruits classified according to ripeness on the display target day.

[0067] [Example of display screen in this system 4: Example of display of items other than fruits that are subject to management] Examples 1-3 of the display screens above show examples of displays targeting fruits in a field. However, the objects managed by this system are not limited to objects such as fruits; the tasks performed in the field can also be managed. Figure 17 is an example of a display screen in this system, shown as display screen example 4.

[0068] In the example shown in Figure 17, a period selection section 81 is provided in the upper left corner of the display screen, allowing the user to select the period for which information will be displayed (in Figure 17, the period for July 2021 is selected). To the right of the period selection section 81, the work performed in the field during the selected period (display period) is displayed. The method of display is not particularly limited, but for example, the work may be displayed in different colors on a calendar corresponding to the display period, or the number of times each work was performed during the display period may be displayed in a table. Of course, multiple types of displays may be used in combination. Such displays make it easy to recognize which work (leaf removal, pest control, harvesting, etc.) was performed on which day.

[0069] In this system, work details are recorded by grouping a large number of images (a series of images obtained in time) from fixed-point cameras and other sources based on similarity, generating similar image groups. For example, a series of images showing a worker can be grouped together. Conversely, a series of images not showing a worker can also be grouped together. Each similar image group can also record the time period in which it was taken, based on the date and time information of the images it contains. In this way, even for work performed in the field, grouping is performed and the information is presented as similar image groups, allowing for efficient reference and recording of work results without having to review all recorded videos.

[0070] For similar images showing workers (where work is recorded), the system can estimate the work content through image recognition processing and automatically record the estimated work content. For example, this system can be equipped with a machine learning model or image analysis program trained under desired conditions, and the work content can be estimated using these trained models or analysis programs. In this case, a work content estimation unit (not shown in Figure 1) can be provided within the server 20 as one of the functional units that operates using the trained model or analysis program. However, the present invention is not limited to this, and the recording of work content may also be done manually by the operator.

[0071] Furthermore, the display screen may show representative images of the work performed. In other words, it may display representative images of all the work performed in the field during the specified period. In this case, the representative images can also be selected from a group of similar images. By selecting representative images in this way, the amount of information displayed on the screen can be compressed (creation of highlight images), thereby reducing the viewing burden on the operator.

[0072] Figure 18 is shown as example display screen 3, illustrating an example of a display screen different from Figure 17. In the display screen of Figure 18, a specific date within the selection period is selected in the date selection unit 82. (In Figure 18, the 17th is selected), and an area is selected in the area selection unit 83 (Area A is selected in Figure 18). In this case, information for the selected date and area can be selected and displayed. In the example in Figure 18, multiple representative images are extracted from the images taken on the target date and highlighted along with the time, from which it can be recognized that pest control work was carried out at 14:00.

[0073] Figure 19 shows an example of a display screen different from Figures 17 and 18, as Display Screen Example 3. In the display screen of Figure 19, the user can select the work content to be displayed in the work content selection units 84 and 85. In the work content selection unit 84, leaf removal is selected, and in the work content selection unit 85, pest control is selected. In accordance with these selections, a representative image and date and time for leaf removal and a representative image and date and time for pest control are displayed, making it easy to recognize when and how a particular work was performed.

[0074] This system only needs to be able to display at least one of the display screen examples 1 to 3 described above, but it is preferable that it be able to display any two or more of the display screen examples 1 to 3 and that these be able to be switched between as desired.

[0075] The embodiments disclosed herein are illustrative in all respects and are not intended to be restrictive. Therefore, the technical scope of the present invention is not construed solely by the embodiments described above, but is defined by the claims. This includes all modifications within the meaning and scope of the equivalents of the claims. [Explanation of Symbols]

[0076] 10. Photography Department 20 servers 21 Main Control Unit 22 Memory section 23 Grouping section 24 Individual identification unit 25 Attribute determination section 26 Display screen creation section 30 devices 31 Display section 32 Input section (operation input section) 51 Attribute Information Switching Unit 52 Icon Buttons 53 Recalculate button 54 Reset button in the editing screen 55 Attribute Type Selection Section 56 Display format switching button 61,83 Area Selection Section 62 Scene Selection Section 63 Display format selection section 64, 65, 72, 73 Attribute Information Selection Section 71,81 Period Selection Section 74,82 Date selection section 84,85 Work content selection section

Claims

1. A grouping unit extracts and groups similar image data consisting of multiple image data with similar features from a set of captured image data, The image of the aforementioned group of similar images is provided with an individual identification unit that identifies a unique object within the group of similar images, An agricultural information management system characterized by comprising a display unit that displays information relating to the object for each of the aforementioned similar image groups.

2. The agricultural information management system according to claim 1, The agricultural information management system is characterized in that the display unit is capable of displaying the number or proportion of the object as information relating to the object.

3. The agricultural information management system according to claim 1, The agricultural information management system is characterized in that the display unit is capable of displaying the ripeness trend of the object as information related to the object.

4. The agricultural information management system according to claim 1, The agricultural information management system is characterized in that the display unit is capable of displaying the status of physiological disorders occurring in the object as information related to the object.

5. The agricultural information management system according to claim 2, An agricultural information management system characterized by the ability to change the number of objects based on operation instructions from an operation input unit.

6. The agricultural information management system according to claim 1, An agricultural information management system characterized by the ability to exclude unnecessary image data from the image data contained in each of the aforementioned similar image groups based on operation instructions from the operation input unit.

7. The agricultural information management system according to claim 1, The aforementioned display unit is characterized in that it can display information about the target object in combination with the shooting location information.

8. The agricultural information management system according to claim 1, The aforementioned display unit is characterized in that it can display information about the object in combination with the date and time of shooting information.

9. The agricultural information management system according to claim 1, The agricultural information management system is characterized in that the display unit can display information about the work performed by the worker for each group of similar images.

Images