Information provision system and information provision method

Abstract

To simplify sorting operations.SOLUTION: An information providing system comprises: an imaging device that captures an image of one or more objects to be sorted; and a projection device that is arranged alongside the imaging device and projects sorting information, which is information for facilitating the sorting work of the one or more objects, onto the positions of the one or more objects identified from image data obtained by the imaging device.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information providing system and an information providing method.

Background Art

[0002] Conventionally, the work of manually sorting harvested agricultural products has been carried out. In this case, the size of the agricultural products is visually confirmed by a person and sorted into grades such as weight. When a person visually confirms and sorts into grades such as weight, it is conceivable that the sorting sizes will differ depending on the person. Conventionally, an agricultural product sorting device that uses a photographed image to select agricultural products has been proposed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the agricultural product sorting device described in Patent Document 1, the size grade of agricultural products is calculated based on the contour of the agricultural products and the area of the internal region of the contour, and the agricultural products are sorted. However, such an agricultural product sorting device needs to have a mechanism for sorting agricultural products, so it is necessary to add a new configuration after the conventional configuration of the transport mechanism for transporting agricultural products. The mechanism for sorting agricultural products has a problem that the device becomes large as the number of sorting types increases. Therefore, there is a need for a technology that facilitates the sorting work with a simple configuration. Note that such a problem is common to all work that requires sorting, not limited to the sorting work of agricultural products.

[0005] In view of the above circumstances, an object of the present invention is to provide a technology that can facilitate the sorting work with a simple configuration.

Means for Solving the Problems

[0006] One aspect of the present invention is an information provision system comprising: an imaging device for imaging one or more objects to be sorted; and a projection device arranged alongside the imaging device for projecting sorting information, which is information for facilitating the sorting of the one or more objects, onto the positions of the one or more objects identified by the image data obtained by the imaging device.

[0007] Aspect 2 of the present invention is an information provision system of aspect 1, wherein the projection device projects the sorting information onto the position of the one or more objects in a manner corresponding to the information identified based on the one or more objects.

[0008] A third aspect of the present invention is an information provision system of the second aspect, wherein the information identified based on one or more objects is size information indicating the size of the one or more objects and information indicating a group corresponding to the size, and the projection device projects the size information and the group indicating information in different ways onto the positions of the one or more objects as sorting information.

[0009] A fourth aspect of the present invention is the information provision system of the third aspect, wherein the projection device projects the size information as text information and the group information as color, symbol, or graphic onto one or more objects.

[0010] Aspect 5 of the present invention is the information provision system of aspect 2, wherein the information identified based on one or more objects is the type of object, and the projection device projects the sorting information in a color corresponding to the type of object.

[0011] Aspect 6 of the present invention is an information provision method comprising: an imaging device imaging one or more objects to be sorted; and a projection device arranged alongside the imaging device projecting sorting information, which is information to facilitate the sorting of the one or more objects, onto the positions of the one or more objects identified by the image data obtained by the imaging device. [Effects of the Invention]

[0012] This invention makes it possible to simplify sorting operations with a simple configuration. [Brief explanation of the drawing]

[0013] [Figure 1] This is a diagram showing an example configuration of the information provision system in the embodiment. [Figure 2] This figure shows the specific configuration of the information generation device in the embodiment. [Figure 3] This figure shows the image after processing by the image processing unit 222. [Figure 4] This is a flowchart showing the processing flow performed by the information provision system in the embodiment. [Figure 5] This is a diagram illustrating the processing of the image processing unit in the embodiment. [Figure 6] This diagram illustrates the post-projection processing by the projection device in the embodiment. [Modes for carrying out the invention]

[0014] One embodiment of the present invention will be described below with reference to the drawings.

[0015] Figure 1 shows an example of the configuration of the information provision system 100 in an embodiment. The information provision system 100 is a system that provides information (hereinafter referred to as "sorting information") to facilitate the sorting work of objects to be sorted. Objects to be sorted are objects that need to be sorted according to at least their size, such as agricultural products. The sorting information is information that allows a person performing the sorting work (hereinafter referred to as "worker") to visually confirm the size of the object, such as information on the size of the object or information indicating the grade of the object. Information on the size of the object is information that indicates the specific size of the object (how much area it has, etc.), and information indicating the grade of the object is information that indicates whether the object is large, medium, small, etc.

[0016] In the information providing system 100, at least one of the information on the size of the object or the information indicating the grade of the object is provided to the operator. For example, the information providing system 100 provides at least one of the information on the size of the object or the information indicating the grade of the object to the operator by mapping it to the object. Thereby, the operator can grasp which size classification the object should be classified into just by looking at the object. Hereinafter, a specific configuration for realizing the above processing will be described. In the following description, a situation of sorting turnips as agricultural products will be described as an example.

[0017] The information providing system 100 includes an imaging device 10, a provided information generating device 20, and a projection device 30. When the object to be sorted is an agricultural product, the information providing system 100 includes a conveying device 50, a water storage unit 60, etc.

[0018] The conveying device 50 is a device for conveying agricultural products. The conveying device 50 is composed of at least a conveying unit 51 and a cleaning device 52. The conveying unit 51 conveys the turnip V, which is an agricultural product, to the water storage unit 60 via the cleaning device 52. The conveying unit 51 is, for example, a belt conveyor. The cleaning device 52 cleans the dirt (for example, mud, etc.) attached to the turnip V. The turnip V cleaned by the cleaning device 52 is conveyed to the water storage unit 60 by the conveying unit 51.

[0019] The water storage unit 60 is filled with water for floating the turnip V. The operator classifies the turnips floating in the water storage unit 60 according to their size.

[0020] The imaging device 10 images the interior of the water storage section 60. For example, the imaging device 10 images the turnip V floating in the water storage section 60. The imaging device 10 is, for example, a camera. The imaging device 10 is installed at a position where it can photograph the interior of the water storage section 60. For example, the imaging device 10 is installed directly above the water storage section 60. The image data captured by the imaging device 10 is output to the provision information generation device 20. The imaging device 10 captures the turnip V floating in the water storage section 60 in real time and outputs the generated image data to the provision information generation device 20 every time it captures an image.

[0021] The provision information generation device 20 generates sorting information by performing image processing on the image data output from the imaging device 10. The provision information generation device 20 outputs the generated sorting information as control information to the projection device 30.

[0022] The projection device 30 projects the sorting information into the water storage section 60 according to the control information output from the provision information generation device 20. The projection device 30 is, for example, a projector. The information projected by the projection device 30 is based on the image data captured at the position of the imaging device 10. Therefore, when the positions of the imaging device 10 and the projection device 30 are separated or when they are facing different directions, there is a high possibility that the sorting information will not be projected to the correct position. Therefore, it is desirable that the projection device 30 be arranged side by side with the imaging device 10. By projecting the sorting information to the correct position, the sorting information can be projected onto the turnip V floating in the water storage section 60.

[0023] FIG. 2 is a diagram showing a specific configuration of the provision information generation device 20 in the embodiment. The provision information generation device 20 includes a communication unit 21, a control unit 22, and a storage unit 23.

[0024] The communication unit 21 communicates with other devices. For example, the communication unit 21 communicates with the imaging device 10 and the projection device 30. The communication unit 21 receives image data from the imaging device 10. The communication unit 21 transmits control information to the projection device 30.

[0025] The control unit 22 controls the entire information generation device 20. The control unit 22 is composed of a processor such as a CPU (Central Processing Unit) and memory. The control unit 22 implements the functions of the acquisition unit 221, the image processing unit 222, and the generation unit 223 by executing a program.

[0026] Some or all of the acquisition unit 221, image processing unit 222, and generation unit 223 may be implemented by hardware (including circuitry) such as an ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), or FPGA, or by the cooperation of software and hardware. The program may be recorded on a computer-readable recording medium. Computer-readable recording media include non-temporary storage media such as portable media like flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may be transmitted via a telecommunications line.

[0027] Some of the functions of the acquisition unit 221, the image processing unit 222, and the generation unit 223 do not need to be pre-installed in the information generation device 20; they may be realized by installing additional application programs in the information generation device 20.

[0028] The acquisition unit 221 acquires image data received by the communication unit 21. The acquisition unit 221 stores the acquired image data in the storage unit 23.

[0029] The image processing unit 222 performs image processing on the image data stored in the storage unit 23. Specifically, the image processing unit 222 detects the turnip V captured in the image data using object detection technology. Furthermore, the image processing unit 222 extracts the region and position information of the detected turnip V using segmentation technology and performs size estimation. For example, the image processing unit 222 may implement the above processing using Yolact-Edge technology. Note that the object detection and segmentation techniques performed by the image processing unit 222 are not limited to those described above, and other existing methods may be used.

[0030] Furthermore, the image processing unit 222 classifies the turnips V according to their estimated size. For example, the image processing unit 222 classifies the turnips V into three groups (large, medium, and small) according to their size. Note that the size classification is not limited to this, and may be further subdivided. The image processing unit 222 colors each group with a different color so that each classified group can be distinguished. For example, the image processing unit 222 colors the information representing the turnips V with a color corresponding to the group, based on the region of the turnips V extracted by segmentation technology and the position information of the turnips V. The information representing the turnips V is, for example, a circle. Note that the information representing the turnips V is not limited to a circle, and may be any shape that can be classified into a group by the operator. Furthermore, the information representing the turnips V may be any size as long as it is large enough for the operator to see.

[0031] Furthermore, considering that the image will be projected by the projection device 30, the image processing unit 222 places information indicating the turnips V, which are colored in different colors for each group, onto a white background image based on the positional information of the turnips V. This allows the image processing unit 222 to generate an image in which the information indicating the turnips V captured in the image data is colored. In this example, the background color of the image on which the information indicating the turnips V is placed is set to white, but this may be changed as appropriate depending on the projection target. The image processing unit 222 may also include information on the estimated size.

[0032] Through the above processing, the image processing unit 222 generates the image shown in Figure 3. Figure 3 shows the image after processing by the image processing unit 222. Figure 3 shows information inf1 indicating multiple turnips V and size information indicating the size of the turnips V. Furthermore, it can be seen that each group is shown in a different color (represented by a different texture in Figure 3). As shown in Figure 3, since the groups can be distinguished by the coloring, the size of the information inf1 indicating the turnips V may be the same regardless of the group, or the size may be different for each group.

[0033] The generation unit 223 generates control information using the image generated by the image processing unit 222. For example, the generation unit 223 generates position information indicating the location of information inf1 and size information in the image generated by the image processing unit 222, and sorting information including information inf1 and size information. The generation unit 223 transmits the generated sorting information as control information to the projection device 30 via the communication unit 21.

[0034] The memory unit 23 stores image data 231. The memory unit 23 is constructed using a memory device such as a magnetic memory device or a semiconductor memory device.

[0035] Figure 4 is a flowchart showing the processing flow of the information provision system 100 in the embodiment. Figure 5 will be used as needed in the explanation of Figure 4. Figure 5 is a diagram illustrating the processing of the image processing unit 222 in the embodiment. The imaging device 10 images the inside of the water reservoir 60 (step S101). The imaging device 10 outputs the image data obtained by imaging to the information provision generation device 20 (step S102). The communication unit 21 of the information provision generation device 20 receives the image data output from the imaging device 10 (step S103). The acquisition unit 221 acquires the image data received by the communication unit 21 and stores the acquired image data in the storage unit 23.

[0036] When image data is stored in the storage unit 23, the image processing unit 222 reads the image data stored in the storage unit 23. The image data read by the image processing unit 222 is shown in Figure 5(A). As an example, Figure 5(A) shows three turnips V. The image processing unit 222 performs image processing on the read image data. First, the image processing unit 222 obtains information inf indicating the region of the turnip V captured in the image data and the position information of the turnip V using segmentation technology. Next, the image processing unit 222 obtains size information of the turnip V based on the acquired information inf indicating the region of the turnip V. The information inf indicating the region of the turnip V and the size information of the turnip V obtained by the image processing unit 222 are shown in Figure 5(B). As shown in Figure 5(B), it can be seen that the image processing by the image processing unit 222 has obtained information inf indicating the region of the turnip V and size information.

[0037] Next, the image processing unit 222 classifies each turnip V into multiple groups based on the acquired turnip V size information. At this time, the image processing unit 222 colors the information inf1, which indicates the turnip V used to identify each group, in a different color for each classified group. The classification results of the turnip V are shown in Figure 5(C). As shown in Figure 5(C), it can be seen that the three turnip V are classified into two groups. Subsequently, as shown in Figure 5(D), the image processing unit 222 places the information inf1, which indicates the turnip V and is colored in a different color for each group, onto an image with a specific background color (for example, white), based on the position information of the turnip V. The image processing unit 222 may also place size information, as shown in Figure 5(D). The image processing unit 222 outputs the processed image to the generation unit 223.

[0038] The generation unit 223 generates sorting information including information inf1 indicating the turnip V in the image after image processing output from the image processing unit 222, position information indicating the location of the size information, and information inf1 indicating the turnip V and size information (step S104). The generation unit 223 transmits the generated sorting information as control information to the projection device 30 via the communication unit 21 (step S105). The projection device 30 receives the control information transmitted from the information provision generation device 20. The projection device 30 projects the sorting information indicated by the received control information into the water storage unit 60 (step S106).

[0039] Figure 6 is a diagram illustrating the post-projection processing by the projection device 30 in the embodiment. As shown in Figure 6, information inf1 indicating the cub V and size information are projected onto each cub V floating in the water reservoir 60. By projecting information inf1 indicating the cub V and size information onto the water reservoir 60, the operator can easily determine which cub V belong to the same group. Note that in Figure 6, for the sake of simplicity, information inf1 indicating the cub V is projected onto all cub V, but it is not necessary for some of the information inf1 indicating the cub V to be not projected onto the cub V. This is because the cub V floating in the water reservoir 60 may move due to the water flow. In the present invention, even if information inf1 indicating the cub V is not projected onto all cub V, it is not necessary for information inf1 indicating the cub V to be projected onto all cub V, as it is possible to easily group the cub V by projecting information inf1 indicating the cub V onto some of the cub V, or by the position of the projected information inf1 indicating the cub V.

[0040] The information provision system 100 configured as described above includes an imaging device 10 that images objects to be sorted, an information provision generation device 20 that generates sorting information, which is information to facilitate the sorting work of objects, based on image data obtained by the imaging device 10, and a projection device 30 that projects the sorting information generated by the information provision generation device 20 onto the objects.

[0041] In this way, the information provision system 100 projects sorting information onto agricultural products that have been transported using the conventional configuration. Therefore, there is no need to add any new configurations after the conventional transport device 50. The additional configurations are the imaging device 10 and the projection device 30, and since no configurations need to be added to the conventional transport device 50, there is no need to change the configuration. Furthermore, since the sorting information is projected onto the agricultural products, even if the number of types to be classified increases, only the sorting information projected onto the agricultural products increases, and the size of the device remains unchanged. Therefore, workers can easily understand the groups to be classified by looking at the agricultural products. As a result, it becomes possible to simplify the sorting work with a simple configuration.

[0042] Furthermore, in the information provision system 100, the information provision generator 20 classifies one or more objects captured in the image data into multiple groups and generates sorting information that includes information showing different characteristics for each group. In this way, the information projected onto the agricultural products, which are the objects, shows different characteristics (different colors for each group), so workers can easily identify which group to classify by looking at the agricultural products. Therefore, it becomes possible to simplify the sorting process with a simple configuration.

[0043] Furthermore, in the information provision system 100, the information generation device 20 classifies one or more objects captured in the image data into multiple groups, at least according to their size. This allows agricultural products of the same grade to be classified into the same group. Therefore, it becomes possible to simplify the sorting process with a simple configuration.

[0044] Furthermore, in the information provision system 100, the information provision generator 20 generates sorting information by coloring the regions representing one or more objects belonging to each group on the image data so that each group has a different color. In this way, the regions representing agricultural products belonging to each group are colored by image processing. Therefore, the projection device 30 only needs to project the sorting information according to the instructions from the information provision generator 20. Thus, an existing projection device 30 can be used.

[0045] Furthermore, in the information provision system 100, one or more objects are located in the water reservoir 60, and the projection device 30 projects sorting information onto one or more objects located in the water reservoir 60. This ensures that the sorting information is projected while the washed agricultural products are located in the water reservoir 60. As a result, workers can immediately sort the agricultural products that have been transported by the transport device 50.

[0046] Furthermore, in the information provision system 100, the projection device 30 projects size information indicating the size of one or more objects and information indicating a group into which one or more objects have been classified (for example, coloring of the area representing the object) onto one or more objects as sorting information. This allows sorting to be performed not only based on color information but also on size information. Therefore, even when it is difficult to judge the color, agricultural products can be easily sorted based on size information.

[0047] <Example 1> The above-described embodiment shows a configuration in which sorting information is projected onto crops floating in the water reservoir 60. However, some crops do not need to be transported to the water reservoir 60. For example, potatoes do not need to be transported to the water reservoir 60. In this case, the projection device 30 may be configured to project sorting information onto crops located on the transport section 51 of the transport device 50, rather than onto crops located in the water reservoir 60. In this configuration, the imaging device 10 and the projection device 30 are positioned to image the transport section 51 of the transport device 50 and project the information onto crops being transported on the transport section 51 of the transport device 50. This makes it possible to apply the system to crops that are not transported to the water reservoir 60.

[0048] <Modification 2> In the embodiment described above, the information generation device 20 is configured to group agricultural products based on their size. The information generation device 20 may also group agricultural products using information other than size. For example, weight information may be used as information other than size. The information generation device 20 may also change the information used for grouping depending on the agricultural product.

[0049] <Variation 3> In the embodiment described above, the information generation device 20 was configured to color each group with a different color. The information generation device 20 may also indicate each group with information other than color, as long as the projection device 30 can project the information in a way that is different for each group. For example, the information generation device 20 may generate sorting information so as to project a different mark (e.g., star, heart, etc.) onto the crops for each group, or it may generate sorting information so as to project a different shape (e.g., circle, triangle, square, etc.) onto the crops for each group.

[0050] <Modification 4> Depending on the crop, some crops may be difficult to project colors onto. For example, if green vegetables are projected in green, workers may not be able to distinguish them. Therefore, the information generation device 20 may be configured to project different colors depending on the target crop.

[0051] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. [Explanation of Symbols]

[0052] 10…Imaging device, 20…Information generation device, 21…Communication unit, 22…Control unit, 23…Storage unit, 30…Projection device, 100…Information provision system, 221…Acquisition unit, 222…Image processing unit, 223…Generation unit

Claims

1. An imaging device that images one or more objects to be sorted, An image processing unit that identifies the size and position of one or more objects captured in the image data based on the image data obtained by the imaging device, A projection device is positioned alongside the imaging device and projects sorting information, which is information based on the size of the one or more objects and is used to facilitate the sorting of the one or more objects, onto the positions of the one or more objects identified by the image processing unit. An information provision system equipped with the following features.

2. The projection device, The sorting information is projected onto the position of the one or more objects in a manner corresponding to the information identified based on the one or more objects. The information provision system according to claim 1.

3. The information identified based on the one or more objects is size information indicating the size of the one or more objects, and information indicating a group corresponding to the size. The projection device, As sorting information, the size information and the group information are projected in different ways onto the position of one or more objects. The information provision system according to claim 2.

4. The projection device, The size information is projected onto one or more objects as textual information, and the information indicating the group is projected onto one or more objects as color, symbols, or shapes. The information provision system according to claim 3.

5. The information identified based on one or more of the aforementioned objects is the type of object, The projection device, The sorting information is projected in a color corresponding to the type of object. The information provision system according to claim 2.

6. The imaging device captures images of one or more objects to be sorted. Based on the image data obtained by the imaging device, the size and position of one or more objects captured in the image data are identified. In a projection device arranged alongside the imaging device, sorting information, which is information based on the size of the one or more objects and is used to facilitate the sorting of the one or more objects, is projected onto the location of the identified one or more objects. Information provision method.

Images