Information providing system and information providing method

Abstract

To simplify sorting operations.SOLUTION: An information providing system comprises: an imaging device that captures an image of one or more objects to be sorted; and a projection device that is arranged alongside the imaging device and projects sorting information, which is information for facilitating the sorting work of the one or more objects, onto the positions of the one or more objects identified from image data obtained by the imaging device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information providing system and an information providing method. [Background technology]

[0002] Conventionally, harvested agricultural products have been sorted manually. In this case, the size of the agricultural products is visually checked by the person and sorted into grades based on weight, etc. When sorting into grades based on weight, etc. by visual check by the person, it is possible that the classification size may differ depending on the person. Conventionally, a crop sorting device has been proposed that sorts agricultural products using captured images (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-102614 Summary of the Invention [Problem to be solved by the invention]

[0004] The crop sorting device described in Patent Document 1 sorts crops by calculating the size and grade of the crop based on the contour of the crop and the area of ​​the region inside the contour. However, such a crop sorting device needs to be equipped with a mechanism for sorting the crops, which requires a new component to be added after the conventional conveying mechanism for transporting the crops. The problem with crop sorting mechanisms is that the device becomes larger as the number of types of crops to be sorted increases. Therefore, there is a need for technology that facilitates sorting work with a simple configuration. Note that this problem is not limited to crop sorting work, but is common to all work that requires sorting.

[0005] In view of the above circumstances, an object of the present invention is to provide a technique that can facilitate sorting work with a simple configuration. [Means for solving the problem]

[0006] Aspect 1 of the present invention is an information provision system comprising an imaging device that captures images of one or more objects to be sorted, and a projection device that is arranged alongside the imaging device and projects sorting information, which is information for facilitating the sorting work of the one or more objects, onto the positions of the one or more objects identified by image data obtained by imaging the one or more objects.

[0007] A second aspect of the present invention is an information provision system according to the first aspect, wherein the projection device projects the sorting information onto the position of the one or more objects in a manner corresponding to information identified based on the one or more objects.

[0008] Aspect 3 of the present invention is an information provision system according to aspect 2, in which the information identified based on the one or more objects is size information indicating the size of the one or more objects and information indicating a group according to the size, and the projection device projects the size information and the information indicating the group in different modes onto the position of the one or more objects as the sorting information.

[0009] A fourth aspect of the present invention is an information providing system according to the third aspect, wherein the projection device projects the size information as text information and the information indicating the group as a color, symbol, or graphic onto the one or more objects.

[0010] Aspect 5 of the present invention is an information provision system according to aspect 2, in which the information identified based on the one or more objects is the type of object, and the projection device projects the sorting information in a color corresponding to the type of object.

[0011] A sixth aspect of the present invention is an information provision method in which an imaging device captures images of one or more objects to be sorted, and a projection device arranged alongside the imaging device projects sorting information, which is information for facilitating the sorting of the one or more objects, onto the positions of the one or more objects identified by image data obtained by the imaging device. [Effects of the Invention]

[0012] According to the present invention, it is possible to facilitate the sorting work with a simple configuration. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of the configuration of an information providing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a specific configuration of a provision information generating device according to an embodiment. [Figure 3] 10 is a diagram showing an image after processing by the image processing unit 222. FIG. [Figure 4] 1 is a flowchart showing a flow of processing performed by an information providing system according to an embodiment. [Figure 5] 5A and 5B are diagrams for explaining processing by an image processing unit in the embodiment. [Figure 6] 10A and 10B are diagrams for explaining post-projection processing by the projection device in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] FIG. 1 is a diagram showing an example of the configuration of an information provision system 100 according to an embodiment. The information provision system 100 is a system that provides information (hereinafter referred to as "sorting information") to facilitate the sorting of objects to be sorted. Objects to be sorted are items that require sorting at least according to size, such as agricultural products. The sorting information is information that enables a person performing sorting work (hereinafter referred to as "worker") to visually recognize the size of the objects, such as information on the size of the objects and information indicating the class of the objects. The information on the size of the objects is information that indicates the specific size of the objects (such as the area), and the information indicating the class of the objects is information that indicates, for example, whether the objects are large, medium, small, etc.

[0016] The information provision system 100 provides the worker with at least one of information on the size of an object or information indicating the class of the object. For example, the information provision system 100 provides the worker with at least one of information on the size of an object or information indicating the class of the object by mapping it to the object. This allows the worker to know what size the object should be classified into just by looking at the object. A specific configuration for realizing the above processing will be described below. In the following description, a situation in which turnips are sorted as agricultural produce will be described as an example.

[0017] The information provision system 100 includes an imaging device 10, a provision information generation device 20, and a projection device 30. When the objects to be sorted are agricultural products, the information provision system 100 includes a conveying device 50, a water storage unit 60, and the like.

[0018] The conveying device 50 is a device for conveying agricultural products. The conveying device 50 is composed of at least a conveying section 51 and a cleaning device 52. The conveying section 51 conveys turnips V, which are agricultural products, to a water storage section 60 via the cleaning device 52. The conveying section 51 is, for example, a belt conveyor. The cleaning device 52 cleans dirt (for example, mud) from the turnips V. The turnips V cleaned by the cleaning device 52 are conveyed to the water storage section 60 by the conveying section 51.

[0019] The water reservoir 60 is filled with water to float the turnips V. The worker sorts the turnips V floating in the water reservoir 60 according to size.

[0020] The imaging device 10 captures an image of the inside of the water storage section 60. For example, the imaging device 10 captures an image of a turnip V floating in the water storage section 60. The imaging device 10 is, for example, a camera. The imaging device 10 is installed in a position where it can capture an image of the inside of the water storage section 60. For example, the imaging device 10 is installed directly above the water storage section 60. Image data captured by the imaging device 10 is output to the provision information generation device 20. The imaging device 10 captures an image of the turnip V floating in the water storage section 60 in real time, and outputs the generated image data each time an image is captured to the provision information generation device 20.

[0021] The provision information generation device 20 generates assortment information by performing image processing on the image data output from the imaging device 10. The provision information generation device 20 outputs the generated assortment information to the projection device 30 as control information.

[0022] The projection device 30 projects the sorting information into the water storage unit 60 in accordance with the control information output from the provision information generation device 20. The projection device 30 is, for example, a projector. The information projected by the projection device 30 is based on image data captured at the position of the imaging device 10. Therefore, if the imaging device 10 and the projection device 30 are positioned apart or facing in different directions, there is a high possibility that the sorting information will not be projected in the correct position. Therefore, it is desirable to place the projection device 30 alongside the imaging device 10. By having the projection device 30 project the sorting information in the correct position, the sorting information can be projected onto the turnip V floating in the water storage unit 60.

[0023] 2 is a diagram showing a specific configuration of the provision information generating device 20 according to the embodiment. The provision information generating device 20 includes a communication unit 21, a control unit 22, and a storage unit .

[0024] The communication unit 21 communicates with other devices. For example, the communication unit 21 communicates with the image capture device 10 and the projection device 30. The communication unit 21 receives image data from the image capture device 10. The communication unit 21 transmits control information to the projection device 30.

[0025] The control unit 22 controls the entire provision information generation device 20. The control unit 22 is configured using a processor such as a CPU (Central Processing Unit) and a memory. The control unit 22 executes a program to realize the functions of an acquisition unit 221, an image processing unit 222, and a generation unit 223.

[0026] Some or all of the acquisition unit 221, image processing unit 222, and generation unit 223 may be realized by hardware (including circuitry) such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA, or may be realized by a combination of software and hardware. The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and non-transitory storage media such as storage devices built into a computer system, such as a hard disk. The program may be transmitted via a telecommunications line.

[0027] Some of the functions of the acquisition unit 221, the image processing unit 222, and the generation unit 223 do not need to be pre-installed in the provision information generation device 20, and may be realized by installing additional application programs in the provision information generation device 20.

[0028] The acquisition unit 221 acquires the image data received by the communication unit 21. The acquisition unit 221 stores the acquired image data in the storage unit .

[0029] The image processing unit 222 performs image processing on the image data stored in the storage unit 23. Specifically, the image processing unit 222 detects the Cub V captured in the image data using object detection technology. Furthermore, the image processing unit 222 extracts the area of ​​the detected Cub V and position information of the Cub V using segmentation technology to estimate its size. For example, the image processing unit 222 may achieve the above processing using Yolact-Edge technology. Note that the object detection and segmentation technology performed by the image processing unit 222 is not limited to the above, and other existing methods may be used.

[0030] Furthermore, the image processing unit 222 classifies the turnips V according to the estimated size. For example, the image processing unit 222 classifies the turnips V into three groups (large, medium, and small) according to size. Note that the size classification is not limited to this, and may be further classified into smaller types. The image processing unit 222 colors each classified group with a different color so that each group can be distinguished. For example, the image processing unit 222 colors the information indicating the turnips V with a color corresponding to the group based on the area of ​​the turnip V extracted using segmentation technology and the position information of the turnip V. The information indicating the turnips V is, for example, a circle. Note that the information indicating the turnips V is not limited to a circle, and may be any shape as long as it can be visually classified into a group by a worker. Furthermore, the information indicating the turnips V may be any size as long as it is a size that can be seen by a worker.

[0031] In consideration of the projection by the projection device 30, the image processing unit 222 arranges information indicating the turnip V, colored in a different color for each group, on an image with a white background based on the position information of the turnip V. This allows the image processing unit 222 to generate an image in which the information indicating the turnip V captured in the image data is colored. Here, as an example, the background color of the image in which the information indicating the turnip V is arranged is white, but this may be changed as appropriate depending on the projection target. The image processing unit 222 may also include information on the estimated size.

[0032] Through the above processing, the image processing unit 222 generates the image shown in Fig. 3. Fig. 3 is a diagram showing the image after processing by the image processing unit 222. Fig. 3 shows information inf1 indicating a plurality of turnips V and size information indicating the size of the turnips V. Furthermore, it can be seen that each group is shown in a different color (represented by a different texture in Fig. 3). As shown in Fig. 3, since the groups can be distinguished by coloring, the size of the information inf1 indicating the turnips V may be the same regardless of the group, or may be different for each group.

[0033] The generation unit 223 generates control information using the image generated by the image processing unit 222. For example, the generation unit 223 generates assortment information including the information inf1 and the size information, and position information indicating the positions of the information inf1 and the size information in the image generated by the image processing unit 222. The generation unit 223 transmits the generated assortment information to the projection device 30 via the communication unit 21 as control information.

[0034] The storage unit 23 stores image data 231. The storage unit 23 is configured using a storage device such as a magnetic storage device or a semiconductor storage device.

[0035] Fig. 4 is a flowchart showing the flow of processing performed by the information providing system 100 in the embodiment. In the explanation of Fig. 4, Fig. 5 will be used as necessary. Fig. 5 is a diagram for explaining the processing of the image processing unit 222 in the embodiment. The imaging device 10 captures an image of the inside of the water storage section 60 (step S101). The imaging device 10 outputs image data obtained by capturing the image to the provision information generation device 20 (step S102). The communication unit 21 of the provision information generation device 20 receives the image data output from the imaging device 10 (step S103). The acquisition unit 221 acquires the image data received by the communication unit 21 and stores the acquired image data in the storage unit 23.

[0036] When image data is saved in the storage unit 23, the image processing unit 222 reads out the image data saved in the storage unit 23. The image data read out by the image processing unit 222 is shown in FIG. 5(A). As an example, three turnips V are captured in FIG. 5(A). The image processing unit 222 performs image processing on the read out image data. As a result, the image processing unit 222 first uses segmentation technology to acquire information (inf) indicating the area of ​​the turnip V captured in the image data and position information of the turnip V. Next, the image processing unit 222 acquires size information of the turnip V based on the acquired information (inf) indicating the area of ​​the turnip V. The information (inf) indicating the area of ​​the turnip V acquired by the image processing unit 222 and the size information of the turnip V are shown in FIG. 5(B). As shown in FIG. 5(B), it can be seen that the information (inf) indicating the area of ​​the turnip V and size information have been acquired through image processing by the image processing unit 222.

[0037] Next, the image processing unit 222 classifies each turnip V into a plurality of groups based on the acquired size information of the turnip V. At this time, the image processing unit 222 colors the information inf1 indicating the turnip V, which is used to identify each group, in a different color for each classified group. The classification results of the turnip V are shown in FIG. 5(C). As shown in FIG. 5(C), it can be seen that three turnips V have been classified into two groups. Thereafter, as shown in FIG. 5(D), the image processing unit 222 arranges the information inf1 indicating the turnip V, which has been colored in a different color for each group, on an image of a specific background color (for example, white), based on the position information of the turnip V. Note that the image processing unit 222 may also arrange size information, as shown in FIG. 5(D). The image processing unit 222 outputs the image after image processing to the generation unit 223.

[0038] The generation unit 223 generates sorting information including position information indicating the position of the information inf1 indicating the turnip V and the size information in the processed image output from the image processing unit 222, and the information inf1 indicating the turnip V and the size information (step S104). The generation unit 223 transmits the generated sorting information as control information to the projection device 30 via the communication unit 21 (step S105). The projection device 30 receives the control information transmitted from the provision information generation device 20. The projection device 30 projects the sorting information indicated by the received control information into the water storage unit 60 (step S106).

[0039] FIG. 6 is a diagram illustrating processing after projection by the projection device 30 in an embodiment. As shown in FIG. 6, information inf1 indicating the turnip V and size information are projected onto each turnip V floating in the water storage section 60. By projecting the information inf1 indicating the turnip V and size information onto the water storage section 60, the worker can easily determine which turnips V belong to the same group. Note that in FIG. 6, for simplicity of explanation, information inf1 indicating the turnips V is projected onto all turnips V; however, information inf1 indicating some turnips V does not have to be projected onto the turnips V. This is because turnips V floating in the water storage section 60 may move due to the flow of water. In the present invention, even if information inf1 indicating the turnips V is not projected onto all turnips V, grouping can be easily performed based on the projected information inf1 indicating some turnips V or the position of the projected information inf1 indicating the turnips V. Therefore, it is not necessary to project information inf1 indicating the turnips V onto all turnips V.

[0040] The information provision system 100 configured as described above includes an imaging device 10 that captures images of objects to be sorted, a provision information generation device 20 that generates sorting information, which is information for facilitating the sorting work of the objects, based on image data obtained by imaging the imaging device 10, and a projection device 30 that projects the sorting information generated by the provision information generation device 20 onto the objects.

[0041] In this way, in the information provision system 100, sorting information is projected onto agricultural products, which are objects transported using a conventional configuration. Therefore, there is no need to add a new component after the transport device 50, which has a conventional configuration. The only components added are the imaging device 10 and the projection device 30, and no additional components are required for the transport device 50, which has a conventional configuration, so there is no need to change the configuration. Furthermore, because the sorting information is projected onto the agricultural products, even if the number of types to be classified increases, only the amount of sorting information projected onto the agricultural products increases, and the scale of the device remains unchanged. Therefore, workers can easily identify the group into which the agricultural products should be classified by looking at them. This makes it possible to facilitate sorting work with a simple configuration.

[0042] Furthermore, in the information provision system 100, the provision information generation device 20 classifies one or more objects captured in the image data into multiple groups and generates sorting information including information indicating different aspects for each group. In this way, the information projected onto the target crops is in a different form for each group (different colors for each group), so workers can easily identify the group to which the crops should be classified by looking at them. This makes it possible to facilitate sorting work with a simple configuration.

[0043] Furthermore, in the information provision system 100, the provision information generation device 20 classifies one or more objects captured in the image data into multiple groups at least according to size. This allows agricultural products of the same grade to be classified into the same group. This makes it possible to facilitate sorting work with a simple configuration.

[0044] Furthermore, in the information provision system 100, the provision information generation device 20 generates sorting information by coloring areas of image data that indicate one or more objects belonging to each group in a different color for each group. In this way, the areas that indicate the agricultural products belonging to each group are colored by image processing. Therefore, the projection device 30 simply projects the sorting information in response to instructions from the provision information generation device 20. Therefore, an existing projection device 30 can be used.

[0045] Furthermore, in information provision system 100, one or more objects are placed in water storage unit 60, which stores water, and projection device 30 projects sorting information onto one or more objects placed in water storage unit 60. As a result, the sorting information is projected onto the washed agricultural products while they are placed in water storage unit 60. This allows workers to immediately sort the agricultural products that have been transported by transport device 50 on the spot.

[0046] Furthermore, in the information providing system 100, the projection device 30 projects, as sorting information, size information indicating the size of one or more objects and information indicating the group into which the one or more objects have been classified (for example, coloring of the area indicating the object) in association with each other onto one or more objects. This allows sorting based on not only color information but also size information. Therefore, even when it is difficult to determine the color, agricultural products can be easily sorted based on size information.

[0047] <Variation 1> In the above-described embodiment, a configuration has been shown in which sorting information is projected onto crops floating in the water storage unit 60. However, some crops do not need to be transported to the water storage unit 60. An example of a crop that does not need to be transported to the water storage unit 60 is potatoes. In this case, the projection device 30 may be configured to project the sorting information onto crops positioned on the transport unit 51 of the transport device 50, rather than onto crops positioned in the water storage unit 60. In this configuration, the imaging device 10 and the projection device 30 are positioned so that they can capture an image of the top of the transport unit 51 of the transport device 50 and project the image onto crops being transported on the transport unit 51 of the transport device 50. This makes it possible to apply the projection information to crops that are not transported to the water storage unit 60.

[0048] <Variation 2> In the above-described embodiment, the provision information generation device 20 is configured to group agricultural products, which are targets, based on size. The provision information generation device 20 may also group agricultural products using information other than size. For example, weight information may be used as the information other than size. Note that the provision information generation device 20 may change the information used for grouping depending on the agricultural product.

[0049] <Variation 3> In the above-described embodiment, the provision information generation device 20 is configured to color each group with a different color. However, the provision information generation device 20 may indicate each group with information other than color, as long as the projection device 30 can project the information so that each group appears in a different manner. For example, the provision information generation device 20 may generate sorting information such that a different mark (e.g., star, heart, etc.) is projected onto the crops for each group, or such that a different shape (e.g., circle, triangle, square, etc.) is projected onto the crops for each group.

[0050] <Variation 4> It is conceivable that some agricultural crops are difficult to project colors onto. For example, if green vegetables are projected in green, it is conceivable that the worker will not be able to distinguish them. Therefore, the provision information generation device 20 may be configured to project different colors depending on the target agricultural crop.

[0051] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0052] 10...imaging device, 20...provided information generating device, 21...communication unit, 22...control unit, 23...storage unit, 30...projection device, 100...information providing system, 221...acquisition unit, 222...image processing unit, 223...generation unit

Claims

1. an imaging device that captures images of one or more objects to be sorted; a projection device that is arranged alongside the imaging device and projects sorting information, which is information for facilitating sorting work of the one or more objects, onto positions of the one or more objects identified by image data obtained by imaging the one or more objects; An information provision system comprising:

2. The projection device is projecting the sorting information onto the positions of the one or more objects in a manner according to information identified based on the one or more objects; The information providing system according to claim 1 .

3. the information identified based on the one or more objects is size information indicating a size of the one or more objects and information indicating a group according to the size, The projection device is projecting, as the sorting information, the size information and the information indicating the group in different modes onto the positions of the one or more objects; The information providing system according to claim 2 .

4. The projection device is the size information is projected as character information, and the information indicating the group is projected as a color, a symbol, or a graphic onto the one or more objects; The information providing system according to claim 3 .

5. the information identified based on the one or more objects is a type of the object; The projection device is projecting the sorting information in a color corresponding to the type of the object; The information providing system according to claim 2 .

6. an imaging device captures an image of one or more objects to be sorted; a projection device disposed next to the imaging device projects sorting information, which is information for facilitating sorting of the one or more objects, onto positions of the one or more objects identified by image data obtained by imaging the one or more objects; Information provision method.

Images