Information provision system, information provision generating device, and information provision method
The information providing system simplifies crop sorting by projecting sorting information onto objects using an imaging and projection system, addressing the complexity of existing devices and enabling efficient classification without scaling up the equipment.
Patent Information
- Application Number
- JP2024054370
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2024-03-28
- Publication Date
- 2024-11-28
- Estimated Expiration
- 2044-03-28
AI Technical Summary
Existing crop sorting devices require complex mechanisms for sorting, which scale up with the number of types to be sorted, necessitating a simpler configuration for efficient sorting.
An information providing system that includes an imaging device to capture objects, a provided information generating device to process image data, and a projection device to project sorting information onto the objects, facilitating sorting by size or grade without altering the existing conveyance system.
Enables efficient sorting with a simple configuration by projecting sorting information onto objects, allowing workers to classify items easily without increasing the scale of the equipment, even as the number of types increases.
Smart Images

Figure 0007591225000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information providing system, a provision information generating device, and an information providing method. [Background technology]
[0002] Conventionally, harvested agricultural products have been sorted manually. In this case, the size of the agricultural products is visually confirmed by the person and sorted into grades based on weight, etc. When the agricultural products are visually confirmed by the person and sorted into grades based on weight, etc., it is considered that the classification size may differ depending on the person. Conventionally, a crop sorting device that sorts agricultural products using captured images has been proposed (for example, see Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2022-102614 A Summary of the Invention [Problem to be solved by the invention]
[0004] In the crop sorting device described in Patent Document 1, the size and grade of the crop are calculated based on the contour of the crop and the area of the region inside the contour to sort the crop. However, since such a crop sorting device needs to be equipped with a mechanism for sorting the crop, a new configuration needs to be added after the conventional conveying mechanism for conveying the crop. The mechanism for sorting the crop has a problem that the device becomes larger as the number of types of crops to be sorted increases. Therefore, a technology that makes sorting work easy with a simple configuration is required. Note that such a problem is not limited to crop sorting work, but is a problem common to all work that requires sorting.
[0005] In view of the above circumstances, an object of the present invention is to provide a technique that can facilitate sorting work with a simple configuration. [Means for solving the problem]
[0006] A first aspect of the present invention is an information provision system comprising an imaging device that images one or more objects to be sorted, a provision information generating device that generates sorting information, which is information for facilitating the sorting task of the one or more objects, based on image data obtained by imaging by the imaging device, and a projection device that projects the sorting information generated by the provision information generating device onto the one or more objects.
[0007] A second aspect of the present invention is an information provision system according to the first aspect, in which the provision information generating device classifies the one or more objects captured in the image data into a plurality of groups, and generates the sorting information including information indicating different aspects for each group.
[0008] A third aspect of the present invention is the information provision system of the second aspect, wherein the provided information generating device classifies the one or more objects captured in the image data into a plurality of groups at least according to size.
[0009] A fourth aspect of the present invention is an information provision system according to the second or third aspect, in which the provision information generating device generates the sorting information by coloring information indicating the one or more objects belonging to each group on the image data in a different color for each group.
[0010] A fifth aspect of the present invention is an information provision system according to any one of the first to fourth aspects, wherein the one or more objects are located in a water storage area in which water is stored or on a transport device for transporting the objects, and the projection device projects the sorting information onto the one or more objects located in the water storage area or on the transport device.
[0011] A sixth aspect of the present invention is an information provision system according to any one of the first to fifth aspects, wherein the projection device projects, as the sorting information, size information indicating the size of the one or more objects and information indicating the group into which the one or more objects are classified, in association with each other, onto the one or more objects.
[0012] A seventh aspect of the present invention is an information provision system according to any one of the first to sixth aspects, wherein the object is an agricultural product.
[0013] Aspect 8 of the present invention is a provision information generation device comprising an acquisition unit that acquires image data of one or more objects to be sorted, a generation unit that generates sorting information, which is information for facilitating the sorting task of the one or more objects, based on the image data acquired by the acquisition unit, and a communication unit that transmits the sorting information generated by the generation unit to a projection device that projects the sorting information onto the one or more objects.
[0014] A ninth aspect of the present invention is an information provision method that involves imaging one or more objects to be sorted, generating sorting information based on image data obtained by imaging, the sorting information being information for facilitating the sorting task of the one or more objects, and projecting the generated sorting information onto the one or more objects. Effect of the Invention
[0015] According to the present invention, it is possible to facilitate the sorting work with a simple configuration. [Brief description of the drawings]
[0016] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information providing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing a specific configuration of a provision information generating device according to an embodiment. [Diagram 3] 13 is a diagram showing an image after processing by the image processing unit 222. FIG. [Figure 4] 4 is a flowchart showing a flow of processing performed by the information providing system in the embodiment. [Diagram 5] 5A to 5C are diagrams for explaining processing of an image processing unit in the embodiment. [Figure 6] 11A and 11B are diagrams for explaining post-projection processing by the projection device in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0018] FIG. 1 is a diagram showing a configuration example of an information providing system 100 in an embodiment. The information providing system 100 is a system that provides information (hereinafter referred to as "sorting information") for facilitating the sorting work of objects to be sorted. The objects to be sorted are objects that require sorting at least according to size, such as agricultural products. The sorting information is information that allows a person performing the sorting work (hereinafter referred to as "worker") to visually recognize the size of the objects, such as information on the size of the objects and information indicating the class of the objects. The information on the size of the objects is information indicating the specific size of the objects (such as the area), and the information indicating the class of the objects is information indicating whether the objects are large, medium, small, etc.
[0019] The information provision system 100 provides the worker with at least one of information on the size of the object or information indicating the class of the object. For example, the information provision system 100 provides the worker with at least one of information on the size of the object or information indicating the class of the object by mapping the object. This allows the worker to know what size the object should be classified into just by looking at the object. A specific configuration for realizing the above process will be described below. In the following description, a situation in which turnips are sorted as agricultural products will be described as an example.
[0020] The information provision system 100 includes an imaging device 10, a provision information generating device 20, and a projection device 30. When the objects to be sorted are agricultural products, the information provision system 100 includes a conveying device 50, a water storage unit 60, and the like.
[0021] The conveying device 50 is a device that conveys agricultural products. The conveying device 50 is composed of at least a conveying section 51 and a cleaning device 52. The conveying section 51 conveys turnips V, which are agricultural products, to the water storage section 60 via the cleaning device 52. The conveying section 51 is, for example, a belt conveyor. The cleaning device 52 cleans dirt (for example, mud, etc.) on the turnips V. The turnips V cleaned by the cleaning device 52 are conveyed to the water storage section 60 by the conveying section 51.
[0022] The water storage section 60 is filled with water to float the turnips V. The worker classifies the turnips V floating in the water storage section 60 according to size.
[0023] The imaging device 10 captures an image of the inside of the water storage section 60. For example, the imaging device 10 captures an image of a turnip V floating in the water storage section 60. The imaging device 10 is, for example, a camera. The imaging device 10 is installed in a position where it can capture an image of the inside of the water storage section 60. For example, the imaging device 10 is installed directly above the water storage section 60. Image data captured by the imaging device 10 is output to the provision information generation device 20. The imaging device 10 captures an image of the turnip V floating in the water storage section 60 in real time, and outputs the image data generated each time an image is captured to the provision information generation device 20.
[0024] The provision information generating device 20 generates assortment information by performing image processing on the image data output from the imaging device 10. The provision information generating device 20 outputs the generated assortment information to the projection device 30 as control information.
[0025] The projection device 30 projects the sorting information into the water storage section 60 in response to the control information output from the provision information generating device 20. The projection device 30 is, for example, a projector. The information projected by the projection device 30 is based on image data captured at the position of the imaging device 10. Therefore, if the imaging device 10 and the projection device 30 are located apart from each other or facing in different directions, there is a high possibility that the sorting information will not be projected at the correct position. Therefore, it is desirable to place the projection device 30 next to the imaging device 10. By projecting the sorting information at the correct position by the projection device 30, the sorting information can be projected onto the turnip V floating in the water storage section 60.
[0026] 2 is a diagram showing a specific configuration of the provision information generating device 20 according to the embodiment. The provision information generating device 20 includes a communication unit 21, a control unit 22, and a storage unit .
[0027] The communication unit 21 communicates with other devices. For example, the communication unit 21 communicates with the imaging device 10 and the projection device 30. The communication unit 21 receives image data from the imaging device 10. The communication unit 21 transmits control information to the projection device 30.
[0028] The control unit 22 controls the entire provision information generating device 20. The control unit 22 is configured using a processor such as a CPU (Central Processing Unit) and a memory. The control unit 22 realizes the functions of an acquisition unit 221, an image processing unit 222, and a generation unit 223 by executing a program.
[0029] Some or all of the acquisition unit 221, image processing unit 222, and generation unit 223 may be realized by hardware (including circuitry) such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA, or may be realized by a combination of software and hardware. The program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a non-transitory storage medium such as a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built into a computer system. The program may be transmitted via an electric communication line.
[0030] Some of the functions of the acquisition unit 221, the image processing unit 222, and the generation unit 223 do not need to be pre-installed in the provision information generation device 20, and may be realized by installing an additional application program in the provision information generation device 20.
[0031] The acquisition unit 221 acquires image data received by the communication unit 21. The acquisition unit 221 stores the acquired image data in the storage unit .
[0032] The image processing unit 222 performs image processing on the image data stored in the storage unit 23. Specifically, the image processing unit 222 detects the turnip V captured in the image data by object detection technology. Furthermore, the image processing unit 222 extracts the area of the detected turnip V and position information of the turnip V by segmentation technology, and estimates the size. For example, the image processing unit 222 may achieve the above processing by using Yolact-Edge technology. Note that the object detection and segmentation technology performed by the image processing unit 222 is not limited to the above, and other existing methods may be used.
[0033] Furthermore, the image processing unit 222 classifies the turnip V according to the estimated size. For example, the image processing unit 222 classifies the turnip V into three groups (large, medium, and small) according to the size. The size classification is not limited to this, and may be classified into more detailed types. The image processing unit 222 colors each group with a different color so that each classified group can be distinguished. For example, the image processing unit 222 colors the information indicating the turnip V with a color corresponding to the group based on the area of the turnip V extracted by the segmentation technology and the position information of the turnip V. The information indicating the turnip V is, for example, a circle. The information indicating the turnip V is not limited to a circle, and may be any shape as long as it can be classified into a group by the worker by looking at it. Furthermore, the information indicating the turnip V may be any size as long as it is a size that can be seen by the worker.
[0034] In consideration of the projection by the projection device 30, the image processing unit 222 arranges the information indicating the turnip V, colored with a different color for each group, on an image with a white background based on the position information of the turnip V. This allows the image processing unit 222 to generate an image in which the information indicating the turnip V captured in the image data is colored. Here, as an example, the background color of the image in which the information indicating the turnip V is arranged is white, but this may be changed as appropriate depending on the projection target. The image processing unit 222 may also include information on the estimated size.
[0035] Through the above processing, the image processing unit 222 generates the image shown in Fig. 3. Fig. 3 is a diagram showing the image after processing by the image processing unit 222. Fig. 3 shows information inf1 indicating a plurality of turnips V and size information indicating the size of the turnips V. Furthermore, it can be seen that each group is shown in a different color (represented by a different texture in Fig. 3). As shown in Fig. 3, since the groups can be distinguished by coloring, the size of the information inf1 indicating the turnips V may be the same regardless of the group, or may be different for each group.
[0036] The generation unit 223 generates control information using the image generated by the image processing unit 222. For example, the generation unit 223 generates assortment information including the information inf1 and the size information, and position information indicating the positions of the information inf1 and the size information in the image generated by the image processing unit 222. The generation unit 223 transmits the generated assortment information to the projection device 30 via the communication unit 21 as control information.
[0037] The storage unit 23 stores image data 231. The storage unit 23 is configured using a storage device such as a magnetic storage device or a semiconductor storage device.
[0038] Fig. 4 is a flowchart showing the flow of processing performed by the information providing system 100 in the embodiment. In the explanation of Fig. 4, Fig. 5 will be used as necessary. Fig. 5 is a diagram for explaining the processing of the image processing unit 222 in the embodiment. The imaging device 10 captures an image of the inside of the water storage section 60 (step S101). The imaging device 10 outputs image data obtained by capturing the image to the provision information generating device 20 (step S102). The communication unit 21 of the provision information generating device 20 receives the image data output from the imaging device 10 (step S103). The acquisition unit 221 acquires the image data received by the communication unit 21, and stores the acquired image data in the storage unit 23.
[0039] When the image data is stored in the storage unit 23, the image processing unit 222 reads out the image data stored in the storage unit 23. The image data read out by the image processing unit 222 is shown in FIG. 5(A). As an example, three turnips V are captured in FIG. 5(A). The image processing unit 222 performs image processing on the read out image data. As a result, the image processing unit 222 first acquires information inf indicating the area of the turnip V captured in the image data and position information of the turnip V by using segmentation technology. Next, the image processing unit 222 acquires size information of the turnip V based on the acquired information inf indicating the area of the turnip V. The information inf indicating the area of the turnip V acquired by the image processing unit 222 and the size information of the turnip V are shown in FIG. 5(B). As shown in FIG. 5(B), it can be seen that the information inf indicating the area of the turnip V and size information have been acquired by the image processing unit 222.
[0040] Next, the image processing unit 222 classifies each turnip V into a plurality of groups based on the acquired size information of the turnip V. At this time, the image processing unit 222 colors the information inf1 indicating the turnip V used to identify each group in a different color for each classified group. The classification result of the turnip V is shown in FIG. 5(C). As shown in FIG. 5(C), it can be seen that three turnips V are classified into two groups. After that, as shown in FIG. 5(D), the image processing unit 222 arranges the information inf1 indicating the turnip V colored in a different color for each group on an image of a specific background color (for example, white) based on the position information of the turnip V. Note that the image processing unit 222 may also arrange size information as shown in FIG. 5(D). The image processing unit 222 outputs the image after image processing to the generation unit 223.
[0041] The generation unit 223 generates sorting information including information inf1 indicating the turnip V in the image after image processing output from the image processing unit 222, position information indicating the position of the size information, and information inf1 indicating the turnip V and the size information (step S104). The generation unit 223 transmits the generated sorting information as control information to the projection device 30 via the communication unit 21 (step S105). The projection device 30 receives the control information transmitted from the provision information generation device 20. The projection device 30 projects the sorting information indicated by the received control information into the water storage unit 60 (step S106).
[0042] FIG. 6 is a diagram for explaining the process after projection by the projection device 30 in the embodiment. As shown in FIG. 6, the information inf1 indicating the turnip V and the size information are projected onto each turnip V floating in the water storage section 60. By projecting the information inf1 indicating the turnip V and the size information onto the water storage section 60, the worker can easily determine which turnips V belong to the same group. Note that in FIG. 6, for the sake of simplicity of explanation, the information inf1 indicating the turnip V is projected onto all turnips V, but the information inf1 indicating some turnips V may not be projected onto the turnips V. This is because the turnips V floating on the water storage section 60 may move due to the flow of water. In the present invention, even if the information inf1 indicating the turnip V is not projected onto all turnips V, the information inf1 indicating the turnip V may be projected onto some turnips V, or the turnips V may be easily grouped according to the position of the projected information inf1 indicating the turnips V, so it is not necessary to project the information inf1 indicating the turnip V onto all turnips V.
[0043] The information provision system 100 configured as described above includes an imaging device 10 that captures images of objects to be sorted, a provision information generation device 20 that generates sorting information, which is information for facilitating the sorting work of the objects, based on image data obtained by imaging using the imaging device 10, and a projection device 30 that projects the sorting information generated by the provision information generation device 20 onto the objects.
[0044] In this way, in the information provision system 100, the sorting information is projected onto the agricultural products, which are the objects transported in the conventional configuration. Therefore, it is not necessary to add a new configuration after the transport device 50, which is the conventional configuration. The added configuration is the imaging device 10 and the projection device 30, and since there is no need to add any configuration to the transport device 50, which is the conventional configuration, there is no need to change the configuration. Furthermore, since the sorting information is projected onto the agricultural products, even if the number of types to be classified increases, only the sorting information projected onto the agricultural products increases, and there is no change in the scale of the device. Therefore, the worker can easily grasp the group to be classified by looking at the agricultural products. Therefore, it is possible to facilitate the sorting work with a simple configuration.
[0045] Furthermore, in the information provision system 100, the provision information generating device 20 classifies one or more objects captured in the image data into a plurality of groups, and generates sorting information including information showing a different aspect for each group. In this way, the information projected onto the agricultural products, which are the objects, has a different aspect for each group (different color for each group), so that the worker can easily understand the group into which the agricultural products should be classified by looking at them. This makes it possible to facilitate the sorting work with a simple configuration.
[0046] Furthermore, in the information provision system 100, the provision information generating device 20 classifies one or more objects captured in the image data into a plurality of groups at least according to their sizes. This allows agricultural products of the same grade to be classified into the same group. This makes it possible to facilitate the sorting work with a simple configuration.
[0047] Furthermore, in the information provision system 100, the provision information generating device 20 generates sorting information by coloring areas on the image data that indicate one or more objects that belong to each group in a different color for each group. In this manner, the areas that indicate the agricultural products that belong to each group are colored by image processing. Therefore, the projection device 30 only needs to project the sorting information in response to an instruction from the provision information generating device 20. Therefore, an existing projection device 30 can be used.
[0048] Furthermore, in information provision system 100, one or more objects are located in water storage unit 60 in which water is stored, and projection device 30 projects sorting information onto one or more objects located in water storage unit 60. As a result, the sorting information is projected while the washed agricultural products are located in water storage unit 60. Therefore, the worker can immediately sort the agricultural products transported by transport device 50 on the spot.
[0049] Furthermore, in the information providing system 100, the projection device 30 projects, as sorting information, size information indicating the size of one or more objects and information indicating a group into which the one or more objects are classified (for example, coloring of an area indicating the object) in association with each other onto one or more objects. This allows sorting based on not only color information but also size information. Therefore, even when it is difficult to judge the color, agricultural products can be easily sorted based on size information.
[0050] <Variation 1> In the above-described embodiment, a configuration in which sorting information is projected onto agricultural crops floating in the water storage section 60 has been shown. In contrast, some agricultural crops do not need to be transported to the water storage section 60. An example of an agricultural crop that does not need to be transported to the water storage section 60 is potatoes. In this case, the projection device 30 may be configured to project the sorting information onto agricultural crops located on the transport section 51 of the transport device 50, rather than onto agricultural crops located in the water storage section 60. When configured in this way, the imaging device 10 and the projection device 30 are disposed at positions where they can capture an image of the transport section 51 of the transport device 50 and project the image onto agricultural crops transported on the transport section 51 of the transport device 50. This makes it possible to apply the present invention to agricultural crops that are not transported to the water storage section 60.
[0051] <Variation 2> In the above-described embodiment, the provision information generating device 20 has been configured to group agricultural products, which are targets, based on size. The provision information generating device 20 may group agricultural products using information other than size. As the information other than size, for example, weight information may be used. Note that the provision information generating device 20 may change the information used for grouping depending on the agricultural product.
[0052] <Variation 3> In the above-described embodiment, the provision information generating device 20 has been configured to color each group with a different color. The provision information generating device 20 may indicate each group with information other than color as long as the projection device 30 can project the information so that each group has a different appearance. For example, the provision information generating device 20 may generate sorting information such that a different mark (e.g., star, heart, etc.) is projected onto the crops for each group, or such that information of a different shape (e.g., circle, triangle, square, etc.) is projected onto the crops for each group.
[0053] <Variation 4> It is considered that some agricultural crops are difficult to project colors onto. For example, it is assumed that a worker cannot distinguish green vegetables from green vegetables even if the green color is projected onto the vegetables. Therefore, the provision information generating device 20 may be configured to project different colors depending on the target agricultural crop.
[0054] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]
[0055] REFERENCE SIGNS LIST 10 imaging device, 20 provision information generating device, 21 communication unit, 22 control unit, 23 storage unit, 30 projection device, 100 information providing system, 221 acquisition unit, 222 image processing unit, 223 generation unit
Claims
1. An imaging device that captures images of multiple objects to be sorted that are positioned randomly within an imaging range; a provision information generating device that generates sorting information, which is information for facilitating a sorting operation of the plurality of objects, based on image data obtained by collectively capturing images of the plurality of objects by the imaging device; and a projection device that projects the sorting information generated by the provision information generation device onto the plurality of objects in real time; An information provision system comprising:
2. The provision information generating device includes: classifying the plurality of objects captured in the image data into a plurality of groups, and generating the sorting information including information indicating different aspects for each group; 2. The information providing system according to claim 1.
3. The provision information generating device includes: classifying the plurality of objects captured in the image data into a plurality of groups according to at least a size; The information providing system according to claim 2.
4. The provision information generating device includes: generating the sorting information by coloring information indicating the plurality of objects belonging to each group on the image data in a different color for each group; 4. The information providing system according to claim 2 or 3.
5. The plurality of objects are located in a water storage unit in which water is stored or on a transport device for transporting the objects; The projection device projects the sorting information onto the water storage unit or the plurality of objects positioned on the conveying device. The information providing system according to any one of claims 1 to 3.
6. The projection device is As the sorting information, size information indicating sizes of the plurality of objects and information indicating groups into which the plurality of objects are classified are associated with each other and projected onto the plurality of objects. The information providing system according to any one of claims 1 to 3.
7. The object is a crop. The information providing system according to any one of claims 1 to 3.
8. An acquisition unit that acquires image data of a plurality of objects to be sorted that are positioned randomly within an imaging range of an imaging device; a generation unit that generates sorting information, which is information for facilitating a sorting operation of the plurality of objects, based on the image data acquired by the acquisition unit; a communication unit that transmits the sorting information generated by the generation unit in real time to a projection device that projects the sorting information onto the plurality of objects; A provision information generating device comprising:
9. Taking images of a plurality of objects to be sorted that are positioned randomly within an imaging range; generating sorting information for facilitating a sorting operation of the plurality of objects based on image data obtained by collectively capturing the images of the plurality of objects; projecting the generated sorting information onto the plurality of objects in real time; Information provision method.
Citation Information
Patent Citations
Projection instruction device and projection instruction system
JP2021107268A
Device for sorting agricultural products
JP2021146247A
Agricultural product sorting device
JP2022102614A
