Specifying device

Abstract

[Problem] To provide a specifying device capable of simplifying a device configuration. [Solution] Provided is a product providing device which is a specifying device that specifies an object to be conveyed along a conveyance path. The specifying device comprises: cameras, each of which sequentially photographs each of a plurality of objects to be conveyed passing through each of detection areas on the conveyance path; cameras, each of which photographs a target that has passed through each of the detection areas on the conveyance path after photographing each of the images of the plurality of objects to be conveyed; and a specification means for specifying an object to be conveyed corresponding to the target from among the plurality of objects to be conveyed on the basis of each of the images of the plurality of objects to be conveyed and the image of the target.

Description

Specific device

[0001] The present invention relates to a specific device. More specifically, the present invention relates to a specific device for identifying an object to be conveyed along a conveyance path.

[0002] Conventionally, in a revolving sushi restaurant or the like, products placed on plates are conveyed along a conveyance path to the customer's table. In such a store, since a plurality of products are conveyed in parallel to their respective required tables, it is necessary to identify the products during conveyance. Techniques for identifying products during conveyance are disclosed in, for example, Patent Document 1 below.

[0003] Patent Document 1 below discloses a device for identifying a product during conveyance by providing an RF tag recording an ID for identifying the product on a plate and reading the RF tag.

[0004] Japanese Patent Application Laid-Open No. 2019-107301

[0005] In the device of Patent Document 1, a configuration for recording an ID on the RF tag provided on the plate is required. For this reason, in the device of Patent Document 1, the device configuration has been complicated. The problem of complicating the device configuration was a problem occurring in general specific devices for identifying an object to be conveyed along a conveyance path.

[0006] The present invention is for solving the above problems, and an object thereof is to provide a specific device capable of simplifying the device configuration.

[0007] A specific device according to one aspect of the present invention is a specific device for identifying an object to be conveyed along a conveyance path, including: a first imaging means for sequentially imaging each of a plurality of objects to be conveyed passing through a first area on the conveyance path; a second imaging means for imaging an object that has passed through a second area on the conveyance path after imaging each of the images of the plurality of objects to be conveyed; and a specifying means for specifying which of the plurality of objects to be conveyed the object is based on each of the images of the plurality of objects to be conveyed and the image of the object.

[0008] Preferably, the above-described identification device further includes an identity output means that uses a trained model generated by machine learning to input a specific image of a particular object and an image of a target object from among images of multiple objects to be transported, and outputs a determination result of whether or not the object to be transported in the specific image and the target object are the same. If the identity output means determines that the object to be transported in the specific image and the target object are the same, the identification means identifies the target object as the object to be transported in the specific image.

[0009] Preferably, the specified device includes a placement section including a reference mark and a product placed on the placement section, and further comprises: a classification means for dividing each of the images of a specific image and an object from among the images of a plurality of items to be transported into a portion of the image of the placement section and a portion of the image of the product; a position acquisition means for acquiring the position of the product relative to the reference mark in each of the images of the specific image and the object; a feature point extraction means for extracting feature points from the portion of the image of the product in each of the images of the specific image and the object; and a determination means for determining whether the item to be transported and the object in the specific image are the same based on the position of the product relative to the reference mark in each of the images of the specific image and the object, and the feature points extracted from the portion of the image of the product in each of the images of the specific image and the object. If the determination means determines that the item to be transported and the object in the specific image are the same, the specification means identifies the object as the item to be transported in the specific image.

[0010] Preferably, the above-described specific device further includes a prediction means for predicting the timing at which the object will pass a required position on the transport path.

[0011] Preferably, the above-described specific device includes a plurality of imaging means, including a first and a second imaging means, and when the entire transport path is divided into a plurality of regions including the first and second regions, each of the plurality of imaging means photographs the transported object as it passes through each of the plurality of regions.

[0012] Preferably, the above-described specific device includes a transport path that comprises a circular transport path and a branch path that branches off from a predetermined position on the circular transport path and heads toward the customer who ordered the transported items, and further comprises a first extrusion device that, after identifying which of a plurality of transported items the object is using identification means, pushes the object from the predetermined position on the circular transport path into the branch path.

[0013] Preferably, in the above-described specific device, the transport path further includes a shortcut path that shortens the distance from a branching point on the annular transport path to a merging point on the annular transport path, and further includes a second extrusion device that, after identifying which of a plurality of transported objects the object is using identification means, pushes the object from the annular transport path to the shortcut path at the branching point.

[0014] A identifying device according to another aspect of the present invention is an identifying device for identifying objects being transported along a transport path, comprising: a first imaging means for sequentially photographing each of a plurality of objects passing through a first region on the transport path; a second imaging means for photographing an object that has passed through a second region on the transport path after photographing each of the images of the plurality of objects; an identifying means for identifying which of the plurality of objects the object is; a feature point extraction means for extracting feature points from the portion of the product image in each of the images of the specific object and the image of the object from the images of the plurality of objects; and based on the feature points extracted by the feature point extraction means from the portion of the product image in each of the images of the specific object and the image of the object, the identifying device determines whether the image of the object in the specific image and the image of the object are the same. The system includes a determination means for determining whether or not something exists, a path setting means for setting a path that each of the multiple transported objects should take based on a first region and the transport destination of each of the multiple transported objects, and a prediction means for predicting the timing at which each of the multiple transported objects will pass through the location of the second region based on the path set by the path setting means. If the determination means determines that the transported object in a specific image is the same as the target object, the identification means identifies the target object as the transported object in the specific image. If the determination means determines that none of the images of the multiple transported objects are the same as the image of the target object, the identification means identifies which of the multiple transported objects the target object is based on the timing at which each of the multiple transported objects passes through the location of the second region.

[0015] According to the present invention, it is possible to provide a specific device that can simplify the configuration of the device.

[0016] This is a plan view showing the configuration of a store employing a product dispensing device according to one embodiment of the present invention. This is a block diagram showing the functional configuration of the control device 1. This is a schematic diagram showing the order table immediately after registering order information based on a new order. This is a schematic diagram showing the order table immediately after updating the order information by receiving a predetermined operation on the touch panel 151. This is a schematic diagram showing the order table immediately after adding the next passing position and expected passing time to the order information. This is a diagram showing the path taken by the transported object when the transported object is transported from the supply conveyor 102 to the table TB5. This is a diagram showing the path taken by the transported object to be transported to the table TB4 when the loading space on the transport path RT34 is full. This is a diagram illustrating an example of a method for extracting characteristic information from the transported object. This is a flowchart showing the operation of the control device 1 when the control device 1 receives an order from a customer through the table touch system in one embodiment of the present invention. This is a flowchart showing the operation of the control device 1 when the control device 1 receives a predetermined operation indicating that the loading of the transported object onto the supply conveyor has been completed through the touch panel in one embodiment of the present invention. This is a first flowchart showing the operation of the control device 1 when a target object is detected by any of the cameras 131-137 and 181-184 in one embodiment of the present invention. This is a second flowchart showing the operation of the control device 1 when a target object is detected by any of the cameras 131-137 and 181-184 in one embodiment of the present invention. This is a subroutine for the feature extraction process (S900) in Figures 10 and 11. This is a schematic diagram showing the respective neural networks 500 of the learning models 43 and 45 (Figure 2) in one embodiment of the present invention.

[0017] Figure 1 is a plan view showing the configuration of a store employing a product dispensing device according to one embodiment of the present invention.

[0018] Referring to Figure 1, the product serving device (an example of a specific device) in this embodiment is installed in a store equipped with a transport means that transports products placed on plates along a transport path and circulates them within the dining area, such as a conveyor belt sushi restaurant. The type of product is arbitrary, and is typically food and beverages such as sushi, snacks, or drinks. Multiple tables TB1 to TB6 and seats for customers are provided in the store near the product transport path. Each of the tables TB1 to TB6 and seats is for a group of multiple customers.

[0019] The product serving device in this embodiment is a device that delivers and serves products to each customer using a different transport route than the "transportation means that carries products placed on plates along a transport route and circulates them within the dining area" described above. The product serving device in this embodiment does not circulate products within the dining area, but rather delivers the items to be transported, including products corresponding to orders received from customers, directly from the kitchen area to the customer's table in the dining area. The product serving device identifies the items to be transported along the transport route and delivers those items to the table of the customer who needs them.

[0020] The lower part of Figure 1 is the kitchen area where the chefs prepare the products, and the upper part of Figure 1 is the dining area where customers eat and drink. The product serving device in this embodiment includes transport paths RT1 to RT6, RT21 to 23, RT31 to RT36, and RT41 to RT44 (an example of a transport path), a control device 1, a circulating conveyor 101, supply conveyors 102 to 106, customer conveyors 107 to 109, rollers 110 to 113, cameras 121 to 125, 131 to 137, and 181 to 184, extrusion devices 141 to 147, touch panels 151 to 155, switching levers 161 to 163, and table touch systems (order input devices) 171 to 176. The number and installation location of each of the transport routes, control devices, conveyors, rollers, cameras, extruders, touch panels, table touch systems, switching levers, and table touch systems are arbitrary.

[0021] Each of the transport routes RT1 to RT6, RT21 to RT3, RT31 to RT36, and RT41 to RT44 is a route for transporting the items to be transported. The items to be transported include a tray (an example of a placement area) and the goods placed on the tray. It is preferable that the tray has a reference mark. The goods may be placed on something other than the tray, or they may not be placed on the placement area.

[0022] The transport route RT1 is circular and located within the kitchen area. The transport route RT1 includes a straight lane RT11 that extends linearly on the worker's side and a straight lane RT12 that extends linearly on the customer's side.

[0023] Each of the transport routes RT2 to RT6 is located within the kitchen area. Each of the transport routes RT2 to RT6 extends horizontally in Figure 1 and merges with transport route RT1 at its rightmost end in Figure 1. Between the rightmost end of each of the transport routes RT2 to RT6 in Figure 1 and transport route RT1, transport means (not shown), such as rollers, are provided to supply the transported items that have been transported along each of the transport routes RT2 to RT6 onto transport route RT1.

[0024] Each of the transport routes RT21 to RT23 (an example of a branched route) is straight and branches off from a predetermined position on transport route RT1, heading towards each of the tables TB1 to TB6 in the seating area.

[0025] Each of the transport routes RT31 to RT36 is a location where customers seated at each of the tables TB1 to TB6 receive their goods. The goods being transported, including the items ordered by customers seated at each of the tables TB1 to TB6, arrive at the transport route RT31 to RT36 that corresponds to that table. Each of the transport routes RT31 to RT36 branches off from one of the transport routes RT21 to RT23 and extends to the vicinity of one of the tables TB1 to TB6.

[0026] Each of the transport routes RT41 to RT44 (an example of a shortcut route) takes a shortcut through transport route RT1 from a predetermined branching point to a predetermined merging point.

[0027] Control device 1 controls the entire product dispensing device. Control device 1 consists of a computer such as a PC (Personal Computer) or smartphone, and includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), an operation unit, a display unit, and a network interface. The CPU executes the control program stored in the ROM. The ROM stores various programs executed by the CPU and various fixed data. The RAM is used to temporarily store data necessary when the CPU executes the control program. The operation unit consists of a keyboard and mouse, and accepts various inputs. The display unit displays various information. The network interface communicates with other devices using communication protocols such as TCP / IP.

[0028] The circulating conveyor 101 transports the goods along the transport path RT1. Each of the supply conveyors 102 to 106 transports the goods along each of the transport paths RT2 to RT6 and supplies the goods to the circulating conveyor 101. Each of the customer conveyors 107 to 109 transports the goods along each of the transport paths RT21 to RT23. Each of the rollers 110 to 113 transports the goods along each of the transport paths RT41 to RT44. The circulating conveyor 101, customer conveyors 107 to 109, and rollers 110 to 113 are all driven continuously, while each of the supply conveyors 102 to 106 is driven at the necessary timing under the control of the control device 1.

[0029] Each of the cameras 121-125, 131-137, and 181-184 has detection areas P21-P25, P31-P37, and P81-P84 (areas enclosed by dotted lines in Figure 1) on one of the transport paths RT1-RT6, RT21-23, and RT31-RT36. Each of the cameras 121-125, 131-137, and 181-184 sequentially photographs each of the multiple transported objects passing through their corresponding detection area. In particular, each of the cameras 131-137 and 181-184 photographs the target object that has passed through the corresponding detection area after each of the multiple transported objects has been photographed by one of the cameras 121-125.

[0030] Each of the extrusion devices 141 to 147, under the control of the control device 1, extrudes the material being transported along the transport path RT1 to another transport path as needed. Each of the extrusion devices 141 to 147 may consist of a computer.

[0031] Each of the touch panels 151 to 155, under the control of the control device 1, displays various information to each of the craftsmen SF1 to SF5 and accepts various operations from each of the craftsmen SF1 to SF5. Each of the touch panels 151 to 155 may be a computer.

[0032] Each of the switching levers 161 to 163 switches the destination of the transported object along each of the transport paths RT21 to RT23.

[0033] Each of the table touch systems 171 to 176 is installed on each of the tables TB1 to TB6. Each of the table touch systems 171 to 176 is equipped with a display and input device such as a touch panel display. Under the control of control device 1, each of the table touch systems 171 to 176 displays various information such as menus to the customers at that table and accepts various operations such as orders from the customers at that table. Each of the table touch systems 171 to 176 may consist of a computer.

[0034] Each of the table touch systems 171 to 176 transmits the details of the received order (type and quantity of goods) to the control device 1 when it receives an order from a customer. The goods corresponding to the orders entered into each of the table touch systems 171 to 176 are delivered directly from the kitchen to the customer's table by the goods delivery device in this embodiment.

[0035] At checkout, each of the control device 1 or table touch systems 171-176 transmits checkout information (types and quantities of goods provided and consumed) to a self-checkout register (not shown) based on the order information stored for the customer group corresponding to that table. The self-checkout register calculates the amount based on the transmitted information, presents it to the customer, and also receives payment from the customer and returns change.

[0036] Control device 1 stores the order table. When control device 1 receives order details from any of the table touch systems 171 to 176, it assigns an order ID to the order details and registers them in the order table as new order information.

[0037] Furthermore, if the table touch system that receives the order creates the order information itself, and the control device 1 receives the order information from the table touch system, it may register that order information in the order table.

[0038] Figure 2 is a block diagram showing the functional configuration of the control device 1.

[0039] Referring to Figure 2, the functions of the control device 1 are realized by the CPU executing various programs stored in the ROM. The control device 1 includes a camera control unit 11, an order receiving unit 13, an order information management unit 15, a detection unit 17, an identity output unit 18 (an example of identity output means), a selection unit 19, a division unit 21 (an example of division means), a position acquisition unit 23 (an example of position acquisition means), a feature point extraction unit 25 (an example of feature point extraction means), a discrimination unit 27 (an example of discrimination unit), a specific unit 29 (an example of specific means), an extrusion control unit 31, a route setting unit 33, a prediction unit 35 (an example of prediction means), an operation display unit 37, a communication unit 38, and a storage unit 39.

[0040] The camera control unit 11 controls the shooting operations of each of the cameras 121 to 125, 131 to 137, and 181 to 184 in FIG. 1. The images captured by each of the cameras 121 to 125, 131 to 137, and 181 to 184 may be still images or moving images.

[0041] The order reception unit 13 receives the order content from each of the table touch systems 171 to 176, creates order information with an order ID attached to the order content, and registers the order information in the order table.

[0042] The order information management unit 15 updates the order information registered in the order table at a necessary timing.

[0043] The detection unit 17 detects the management target in each of the detection areas P21 to P25, P31 to P37, and P81 to P84 based on the images captured by each of the cameras 121 to 125, 131 to 137, and 181 to 184 in FIG. 1. The detection unit 17 may detect the management target using the learning model 43.

[0044] The management target is the conveyed object that is the target of the feature extraction process described later. The type of the management target is arbitrary. As an example, the management target may include a conveyed object including a plate and the goods placed on the plate, accessories such as spoons, foreign objects (such as pieces of goods (such as potato fries) dropped from the plate), etc. Also, when the goods providing device is installed in a revolving sushi restaurant, etc., a conveyed object including a circular providing container may be included in the management target, or a conveyed object including a container having a circular planar shape on which a bowl containing soup such as udon or miso soup is placed may be included in the management target. Further, as the plate included in the conveyed object, a conveyed object including a plate for placing sushi, a plate for placing beverages, a plate for placing steamed egg custard, or a plate for providing desserts, etc. may be included in the management target. On the other hand, conveyors and rollers for conveying the conveyed object, and conveyed objects in which a plate with an image for promotion displayed is placed on the plate, etc. may be excluded from the management target.

[0045] The identity output unit 18 uses the learning model 45 to input one image (which may be referred to as a specific image) selected by the selection unit 19 from the images of a plurality of conveyed objects, and an image of a conveyed object (which may be referred to as an object) taken by any one of the cameras 131 to 137 and 181 to 184 in FIG. 1, and outputs a discrimination result as to whether the conveyed object shown in the specific image and the object (the object shown in the image of the object) are the same.

[0046] The selection unit 19 selects one image from each of the images of a plurality of conveyed objects taken by the cameras 121 to 125 (FIG. 1).

[0047] The classification unit 21 classifies each of the specific image and the image of the object among the images of a plurality of conveyed objects into a plate image portion and a product image portion.

[0048] The position acquisition unit 23 acquires the position of the product with respect to the reference mark based on the plate image portion and the product image portion in each of the specific image and the image of the object.

[0049] The feature point extraction unit 25 extracts feature points from the product image portion in each of the specific image and the image of the object.

[0050] The discrimination unit 27 discriminates whether the conveyed object shown in the specific image and the object (the object shown in the image of the object) are the same based on the position of the product with respect to the reference mark in each of the specific image and the image of the object, and the feature points extracted from the product image portion in each of the specific image and the image of the object.

[0051] The specifying unit 29 specifies which of the plurality of conveyed objects the object is based on each of the images of the plurality of conveyed objects taken by the cameras 121 to 125 (FIG. 1) and the image of the object.

[0052] The identification unit 29 may identify the object to be transported based on the determination result by the identity output unit 18, or it may identify the object to be transported based on the determination result by the determination unit 27. Specifically, if the identity output unit 18 determines that the object to be transported in a specific image is the same as the target object, the identification unit 29 may identify the target object as the object to be transported in the specific image. Alternatively, if the determination unit 27 determines that the object to be transported in a specific image is the same as the target object, the identification unit 29 may identify the target object as the object to be transported in the specific image.

[0053] The extrusion control unit 31 controls the operation of each of the extrusion devices 141 to 147 and the switching levers 161 to 163 shown in Figure 1.

[0054] The route setting unit 33 sets the path that the object should take at the necessary timing.

[0055] The prediction unit 35 predicts the expected passing time (an example of passing timing) for the object to pass through the required positions (in this embodiment, the detection area of ​​the camera that the object will next pass through) on the transport paths RT1 to RT6, RT21 to 23, RT31 to RT36, and RT41 to RT44 shown in Figure 1, based on the path set by the path setting unit 33.

[0056] The operation display unit 37 receives various operations and displays various information.

[0057] The communication unit 38 communicates with the circulation conveyor 101, supply conveyors 102-106, customer conveyors 107-109, rollers 110-113, cameras 121-125, 131-137, and 181-184 shown in Figure 1, as well as with the extrusion devices 141-147, touch panels 151-155, switching levers 161-163, and table touch systems 171-176, etc.

[0058] The memory unit 39 stores various information such as the order table 41, the learning model 43, and the learning model 45.

[0059] The learning model 43 is a model generated by machine learning. The learning model 43 takes images taken by cameras 121-125, 131-137, and 181-184 as input and outputs a result of determining whether or not the image contains the object to be transported.

[0060] The learning model 45 is a model generated by machine learning. The learning model 45 takes a specific image and an image of an object as input and outputs a result of determining whether the transported object and the object shown in the specific image are the same.

[0061] Figure 3 schematically shows the order table immediately after registering order information based on a new order.

[0062] Referring to Figures 1 and 3, the order table 41 (Figure 2) contains multiple order entries in the order in which they were received. The order information includes the following items: "Order ID," "Destination," "Product Type," "Order Received Time," "Status," "Feature Information," "Next Passing Location," and "Expected Passing Time." "Order ID" is information that identifies the order. "Destination" is information that identifies the table corresponding to the table touch system that sent the order (the table to which the product will be delivered). "Product Type" is the type of product related to the order. "Order Received Time" is the time when the table touch system received the order. "Status" is the status of the ordered product. "Status" can be one of the following: "Preparing," "Feature Extraction," "Transporting," or "Transportation Completed." "Feature Information" is information about the characteristics of the transported object, including the product being transported. "Next Passing Location" is information indicating the detection area that the transported object, including the product being transported, will pass through next. "Expected Passing Time" is the time when the transported object, including the product being transported, is expected to pass through the next passing location.

[0063] Here, we assume that the control device 1 (Figure 2) receives a new order for "Scallop Sushi" from a customer at table TB5 (Figure 1) via the table touch system 175. Upon receiving this order, the control device 1 associates information identifying the destination of the order (table TB5 of the customer who placed the order), the "product type" (the type of product related to the order), and the "order acceptance time" (the time the order was accepted), and creates new order information with an arbitrary order ID "00119" as shown in Figure 3. The control device 1 registers the created order information at the bottom of the order table. At this time, the "status" field is filled with the status "Preparing," indicating that preparations are being made for the delivery of the product. The "characteristic information," "next passing location," and "expected passing information" fields in the order table are left blank at this point.

[0064] Referring to Figure 1, an example is shown of a kitchen area where five chefs SF1 to SF5 are each preparing food. In front of each chef SF1 to SF5 (upper direction in Figure 1), there are supply conveyors 102 to 106 and touch panels 151 to 155 for the chefs. Here, corresponding to each of the five chefs SF1 to SF5, five supply conveyors 102 to 106 and five touch panels 151 to 155 are installed in front of each chef.

[0065] The circulating conveyor 101 circulates the conveyed items along a ring-shaped transport path RT1 within the kitchen. The rotation direction of the circulating conveyor 101 is counterclockwise, as indicated by the black arrow within the transport path RT1. The rotation direction of the circulating conveyor 101 is arbitrary and may be clockwise.

[0066] Regardless of which of the craftsmen SF1 to SF5 is in charge of preparing the products, the operation of the craftsman and the product dispensing device is the same. Here, we will assume that craftsman SF5 is in charge of preparing the products. When new order information is registered in the order table, the control device 1 displays the order information on the craftsman's touch panel 155 installed in front of the craftsman SF5 in charge of that order, and gives instructions for preparing the product (such as the type of product and the ID of the table to which it will be delivered). Seeing this, craftsman SF5 prepares the product (cooks the product if it is sushi), places the prepared product on a plate, and places the product on the plate onto the supply conveyor belt 106. Hereafter, the object including the plate and the product placed on the plate may be referred to as the conveyed object.

[0067] After placing the items to be transported corresponding to the order onto the supply conveyor 106, the worker SF5 performs a predetermined operation on the touch panel 155 to indicate that the placement of the items has been completed (for example, touching the display area for the order corresponding to that item). Upon receiving this operation, the control device 1 operates the supply conveyor 106. As a result, the items placed on the supply conveyor 106 are supplied to the straight lane RT11 via the transport path RT6, as indicated by the black arrow. Upon receiving this operation, the control device 1 also updates the order information in the order table corresponding to the new order.

[0068] Figure 4 schematically shows the order table 41 (Figure 2) immediately after a predetermined operation is received on the touch panel 155 (Figure 1) and the order information is updated.

[0069] Referring to Figures 1 and 4, when the control device 1 receives a predetermined operation from the craftsman SF5 on the touch panel 151 indicating that the loading of the transported object corresponding to the order information containing the order ID "00119" has been completed, the control device 1 updates the "Status" column of the order information to "Feature Extraction in Progress". "Feature Extraction in Progress" indicates that feature information is being extracted from an image of the transported object.

[0070] Referring to Figure 1, each of the cameras 121 to 125 (an example of the first imaging means) sequentially images each of the multiple transported objects passing through the corresponding detection area. Each of the detection areas P21 to P25 (an example of the first area) may be located on the annular transport path RT1, or on each of the transport paths RT2 to RT6. The control device 1 extracts feature information from the images of each of the multiple transported objects using an extraction method described later. The control device 1 adds the extracted feature information to the order information corresponding to that transported object. As a result, the order information in the order table further includes the feature information of the transported object corresponding to that order information. The feature information of the transported object is used to identify the transported object during transport.

[0071] If the transported object corresponds to order information containing the order ID "00119", the control device 1 extracts characteristic information of the transported object from the image captured by the camera 125 as the transported object passes through the detection area P25. The control device 1 then adds the extracted characteristic information to the order information of the transported object (Figure 5).

[0072] After adding the extracted feature information to the order information, the control device 1 sets the path TE that the transported object should take, based on the detection area P25 (Figure 1) of the camera 125 that photographed the transported object and the position of the destination table TB5 included in the order information. Then, based on the set path TE, the control device 1 identifies the next position the transported object will pass and predicts the expected time of passage. The next position is, for example, one of the detection areas P31-P37 and P81-P84 (an example of a second area) of cameras 131-137 and 181-184. In this case, the detection area P35 adjacent to detection area P25 downstream along path TE is identified as the next position. The expected time of passage may be calculated based on the current time, the distance along path TE from the current detection position to the next detection position, and the speed at which the transported object is being carried.

[0073] Furthermore, for purposes such as product modification, an item being transported along transport path RT1 may be removed by a worker and returned to the transport path. Also, an item being transported may be pushed by another item being transported. When such situations occur, the actual time the item passes the next destination will deviate from the expected passing time.

[0074] Figure 5 schematically shows the order table immediately after the next passing position and estimated passing time have been added to the order information.

[0075] Referring to Figures 1 and 5, after extracting the characteristic information of the transported object, the control device 1 updates the "Status" column of the order information containing the order ID "00119" to "Transporting". "Transporting" indicates that the transported object is being transported along one of the transport routes RT1 to RT6, RT21 to 23, and RT41 to RT44. The control device 1 also adds four pieces of characteristic information, "00119a", "00119b", "00119c", and "00119d", to the "Characteristic Information" column of the order information containing the order ID "00119" in the order table. After identifying the next passing position and predicting the expected passing time, the control device 1 adds the information "P35", which identifies the detection area P35, to the next passing position column, and adds the expected passing time "2024 / 06 / 21 12:55:40".

[0076] In this embodiment, in order to distinguish the four feature information in each order information, the symbols "a", "b", "c", or "d" are added to the end of each order ID as the name of the feature information. The feature information with the symbol "a" added to the end of the order ID (for order information containing the order ID "00119", the feature information is named "00119a") is referred to as the first feature information. The feature information with the symbol "b" added to the end of the order ID (for order information containing the order ID "00119", the feature information is named "00119b") is referred to as the second feature information, and the feature information with the symbol "c" added to the end of the order ID (for order information containing the order ID "00119", the feature information is named "00119c") is referred to as the third feature information. The characteristic information of a name with the symbol "d" appended to the end of the order ID (for example, in order information containing the order ID "00119", the characteristic information would be "00119d") is sometimes referred to as the fourth characteristic information.

[0077] Referring to Figure 1, ordered goods are delivered to the customer's table based on the order information. Cameras 131 to 137 are provided at necessary positions on the annular transport path RT1 to photograph the transported items during transport. In this embodiment, each of the cameras 131 to 137 corresponds to each of the extruders 141 to 147. Each of the cameras 131 to 137 photographs the transported items as they pass through each of the detection areas P31 to P37. The control device 1 extracts feature information from the images of the transported objects captured by each of the cameras 131 to 137, and identifies which transported item it is based on the extracted feature information. The control device 1 then sets the path that the transported item should take based on the destination information included in the order information corresponding to the identified transported item. The set path is basically the same as an already set path, but may differ depending on the situation. Based on the set path, the control device 1 identifies the next passing position and calculates the expected passing time. The control device 1 updates the next passing position and estimated passing time in the order information corresponding to the identified transported item. The control device 1 also controls the operation of each of the extruders 141 to 147 corresponding to the cameras that captured the images, based on the set route. This ensures that the transported item, including the product, is correctly delivered to the customer's table (destination) that ordered the product.

[0078] Each of the transport routes RT21 to RT23 branches off from a predetermined position on the customer-side straight lane RT12. The customer conveyor 107 transports items to table TB1 or TB2 along transport route RT21, and by operating the downstream switching lever 161, goods are transported from transport route RT21 to either transport route RT31 or RT32. Transport route RT31 is the transport route that provides goods to customers at table TB1, and transport route RT32 is the transport route that provides goods to customers at table TB2.

[0079] The customer conveyor 108 transports items to tables TB3 or TB4 along the transport path RT22. By operating the downstream switching lever 162, the goods are transported from transport path RT22 to either transport path RT33 or RT34. Transport path RT33 is the transport path that provides goods to customers at table TB3, and transport path RT34 is the transport path that provides goods to customers at table TB4.

[0080] The customer conveyor 109 transports items to tables TB5 or TB6 along the transport path RT23. By operating the downstream switching lever 163, the goods are transported from transport path RT23 to either transport path RT35 or RT36. Transport path RT35 is the transport path that provides goods to customers at table TB5, and transport path RT36 is the transport path that provides goods to customers at table TB6.

[0081] Extrusion devices 145 to 147 (an example of a first extrusion device) are provided to push the objects being transported along the straight lane RT12 into each of the transport paths RT21 to RT23. When each of the extrusion devices 145 to 147 operates, the objects being transported to a position on the straight lane RT12 that connects to each of the transport paths RT21 to RT23 are pushed out into each of the transport paths RT21 to RT23.

[0082] Each of the transport paths RT41 to RT44 is a shortcut path that shortens the route from a branching point on the annular transport path RT1 to a merging point on the annular transport path RT1. Each of the transport paths RT41 and RT42 is a shortcut path that shortens the route of the transported material from the straight lane RT11 to the straight lane RT12. Each of the transport paths RT43 and RT44 is a shortcut path that shortens the route of the transported material from the straight lane RT12 to the straight lane RT11. Each of the rollers 110 to 113 transports the required transported material along each of the transport paths RT41 to RT44. The transport direction of each of the rollers 110 to 113 is indicated by a black arrow in each of the transport paths RT41 to RT44. Each of the branching points of the transport paths RT41 to RT44 from the annular transport path RT1 is provided with an extruder 141 to 144 (an example of a second extruder).

[0083] When the extrusion device 141 or 142 operates, the object to be transported, which has been transported to the branching point with the transport path RT41 or RT42 on the straight lane RT11, is pushed out onto the transport path RT41 or RT42. The pushed-out object is transported along the transport path RT41 or RT42 by the rollers 110 or 111 and moves to the merging point with the transport path RT41 or RT42 on the straight lane RT11. As a result, the object that has passed through the transport path RT41 or RT42 moves from a predetermined branching point on the straight lane RT11 to a predetermined merging point on the straight lane RT12 without having to go around the annular transport path RT1.

[0084] Figure 6 shows the path taken by an object when it is transported from the supply conveyor 102 to the table TB5.

[0085] Referring to Figure 6, when transporting an object from the supply conveyor 102 to the table TB5 (assuming the loading space on the transport path RT35 is not full), the control device 1 operates the extruder 141, the extruder 147, and the switching lever 163 at the necessary timings. As a result, the object is transported to the transport path RT35 corresponding to the table TB5, passing through the straight lane RT11, the transport path RT41, the straight lane RT12, and the transport path RT23 in that order, as shown by arrow AR1 in Figure 6.

[0086] Referring to Figure 1, when the extrusion device 143 or 144 operates, the object to be transported, which has been transported to the branching point with the transport path RT43 or RT44 on the straight lane RT12, is pushed out onto the transport path RT43 or RT44. The pushed-out object is transported along the transport path RT43 or RT44 by the rollers 112 or 113 and moves to the merging point with the transport path RT43 or RT44 on the straight lane RT11. As a result, the object that has passed through the transport path RT43 or RT44 moves from a predetermined branching point on the straight lane RT12 to a predetermined merging point on the straight lane RT11 without having to go around the annular transport path RT1. Therefore, if the path to the destination is shortened by passing through the transport path RT43 or RT44, the control device 1 sets the path that the object should take to pass through the transport path RT43 or RT44.

[0087] If a transported object accumulates in any of the transport paths RT31 to RT36, the loading space in that transport path becomes full, and it becomes impossible to send the next transported object to that transport path. Therefore, if the control device 1 detects that the loading space in any of the transport paths RT31 to RT36 is full using a sensor (not shown), it sets (changes) the path that the transported object should take to a circulating path and controls the operation of the necessary extruders 145 to 147 according to the set path. This prevents the transported object accumulated in the transport path from being pushed out by newly transported objects. The transported object whose path has been changed will circulate along the annular transport path RT1 until there is space available in the loading space of the destination transport path. The control device 1 detects when the loading space of the transported object in a transport path is full using sensors (not shown) provided in each of the transport paths RT31 to RT36.

[0088] Furthermore, the control device 1 controls the transport to ensure that the transported items do not move too far from the destination table by using each of the transport paths RT41 to RT44, so that the transported items can be immediately transported to the destination table when there is space available on the transport path at the destination.

[0089] Figure 7 shows the path taken by the object to be transported to the table TB4 when the loading space on the transport path RT34 is full.

[0090] Referring to Figure 7, for example, when the loading space of the transport path RT34 corresponding to table TB4 is full, and the camera 136 identifies the transported object as one to be transported to table TB4 based on the image of the transported object, the control device 1 sets (changes) the route in the circulation path. The control device 1 operates the extruder 144 without operating the extruder 146 to push the transported object from the straight lane RT22 to RT11. The control device 1 also operates the extruder 142 to push the transported object from the straight lane RT11 to RT12. As a result, the transported object to be transported to table TB4 can be made to wait on the circulation path, which is composed of the central parts of the straight lanes RT11 and RT12 and the transport paths RT44 and RT42. This circulation path is indicated by arrow AR2 in Figure 7.

[0091] Then, when there is free space in the transport path RT34 corresponding to table TB4, the control device 1 sets (changes) the route so that the objects to be transported waiting on the circulation path move toward transport path RT22. The control device 1 then operates the extrusion device 146 at the necessary timing to transport the objects from the straight lane RT22 to the transport path RT34, as shown by arrow AR3 in Figure 7. While waiting on the circulation path, the objects to be transported to table TB4 are not transported to the transport path RT1 to the left of transport path RT44 or to the transport path RT1 to the right of transport path RT42. Therefore, when there is free space in the transport path RT34 corresponding to table TB4, the objects to be transported can be quickly transported from the circulation path to transport path RT34.

[0092] Referring to Figure 1, each of the cameras 181 to 184 has detection areas P81 to P84, respectively, near the junction with the annular transport path RT1 in the transport paths RT41 to RT44. Each of the cameras 181 to 184 photographs the transported object as it passes through each of the detection areas P81 to P84. The control device 1 extracts feature information from the images of the transported object taken by each of the cameras 181 to 184 and identifies the transported object based on the extracted feature information. Each of the detection areas P81 to P84 may be provided on the annular transport path RT1, or on each of the transport paths RT41 to RT44. By providing each of the cameras 181 to 184, it is possible to detect when the transported object that has been transported on each of the transport paths RT41 to RT44 has returned to the annular transport path RT1, thereby improving transport accuracy.

[0093] When the control device 1 detects, based on an image captured by a camera (not shown), that an item has arrived at the destination transport path (table) among the transport paths RT31 to RT36, it may update the "Status" column of the order information corresponding to that item in the order table to "Transportation Completed". Detecting transport completion using a camera is optional.

[0094] The product dispensing device configured as described above makes it possible to provide products simply and quickly.

[0095] Next, we will explain an example of a method for extracting feature information. The feature information can be any information extracted from an image captured by a camera, and its type and combination are arbitrary.

[0096] Figure 8 illustrates an example of a method for extracting feature information. In this embodiment, an example is shown in which feature information is extracted using an image of the transported object taken from above, but the direction in which the image of the transported object is taken is arbitrary.

[0097] Referring to Figure 8(a), the target image IM1 typically shows a transported object BT1 (an example of a transported object and target object), which includes a plate BT11 (an example of a mounting area) and a product BT12 (an example of a product) placed on the plate BT11. The plate BT11 contains an arbitrary number of reference marks MK (an example of reference marks). The reference marks MK are, for example, the store's logo.

[0098] Referring to Figure 8(b), the control device 1 divides the image IM1 into a portion IM11 of the plate image and a portion IM12 of the product image by performing edge processing on the image IM1. The control device 1 then extracts feature points from the portion IM12 of the product image. Specifically, the control device 1 extracts feature points of the product's shape based on the product's outline and feature points of the product's pattern caused by lines, bumps, etc., from the portion IM12 of the product image. The extracted feature points of the product's shape are set as the first feature information. The feature points of the product's pattern are set as the second feature information. Furthermore, the control device 1 divides the portion IM12 of the product image into multiple regions and extracts color information (for example, numerical information of the R component, G component, and B component, respectively) from each region. The extracted color information is set as the third feature information.

[0099] Referring to Figure 8(c), the control device 1 obtains the position of product BT12 relative to the reference mark MK of plate BT11 in image IM1, based on the portion IM11 of the plate image and the portion IM12 of the product image. The position of product BT12 relative to the reference mark MK in image IM1 can be defined in any way, for example, it may be the direction and distance in which the center CR2 of the product is offset from the center CR1 of the three reference marks MK, or it may be the inclination angle of the product which is approximately rectangular with respect to the straight line connecting the two reference marks MK. The information of the position of the product relative to the reference mark MK in image IM1 is considered the fourth feature information. Note that if the transported object included in the target image does not include a plate, the portion of the plate image does not exist, and therefore the fourth feature information does not need to be extracted.

[0100] In this way, the first to fourth feature information is extracted from the image IM1. If the image IM1 was captured by any of the cameras 121 to 125 (Figure 1), the control device 1 names the extracted first to fourth feature information using the method described with reference to Figure 5 and adds it to the order information corresponding to the transported item in the order table.

[0101] The shape, pattern, and color of the products will vary from one product to another during the manufacturing process. Furthermore, the position of the product relative to the standard mark on the plate will vary from product to product as it is placed on the plate. The first to fourth characteristic information points focus on these variations.

[0102] Next, we will explain how to identify objects passing through each detection area.

[0103] Referring to Figure 1, the control device 1 captures images of detection areas P31-P37 and P81-P84, respectively, with cameras 131-137 and 181-184 (an example of a second imaging means) on the transport path, and detects the object to be managed from the images captured by these cameras. The detection of the object to be managed may be performed using the learning model 43 or using the feature information extraction method described above. When an object to be managed is detected, the control device 1 sets the transported object detected as the object to be managed as the target object. The control device 1 acquires an image of the target object and extracts the feature information of the target object from the image of the target object using the method described above.

[0104] After extracting characteristic information of the object, the control device 1 refers to the order table and extracts order information where the next passing position is the same as the detection area where the object was detected. For example, if the order table is as shown in Figure 5 and the detection area where the object was detected is detection area P35 (Figure 1), the control device 1 extracts five pieces of order information, each containing the order IDs "00115", "00116", "00117", "00118", and "00119". The control device 1 may also extract all order information from the order table regardless of the next passing position information included in the order information, or it may extract all order information whose status is "in transit".

[0105] Next, the control device 1 selects one order from the extracted order information, compares the feature information of the selected order information with the feature information of the target object, and determines whether the transported object corresponding to the feature information of the selected order information and the target object are identical. For example, the control device 1 may determine whether the first feature information of the selected order information and the first feature information of the target object are identical, whether the second feature information of the selected order information and the second feature information of the target object are identical, whether the third feature information of the selected order information and the third feature information of the target object are identical, and whether the fourth feature information of the selected order information and the fourth feature information of the target object are identical. Based on the four determination results obtained in this way, it may be determined whether the transported object corresponding to the feature information of the selected order information (more specifically, the transported object shown in the image from which the feature information included in the selected order information was extracted) and the target object are identical. When performing the four determinations described above, each feature information may be quantified before determining whether they are identical or not.

[0106] After obtaining four discrimination results, for example, if a predetermined number of the four discrimination results are identical, it may be determined that the transported object corresponding to the characteristic information of the selected order information and the target object are identical. Alternatively, only some of the first to fourth characteristic information may be extracted from the image, and the determination of whether or not they are identical may be made based on the extracted characteristic information. As a characteristic information different from the first to fourth characteristic information, characteristic information indicating the gloss of the product may be further extracted from the image, and the determination of whether or not they are identical may be made based on the extracted characteristic information. In particular, if the product is perishable, the product will dry out over time, and the gloss of the product will change.

[0107] In this way, the control device 1 selects order information one by one from the extracted order information and determines whether the image corresponding to the feature information of the selected order information is the same as the image of the object. If it is determined that they are the same, the control device 1 identifies the object as the transported object corresponding to the selected order information.

[0108] Furthermore, the position of the product on the plate or the shape of the product may change during transport. In particular, if the product is sushi, the toppings placed on top of the rice may fall apart during transport. In addition, foreign objects (typically objects that have fallen from the product being transported, such as salmon roe grains in the case of salmon roe sushi) may be generated along the transport path and detected as the target object. When such changes occur, the system may determine that none of the images corresponding to the characteristic information of the extracted order information are identical to the image of the target object, making it impossible to identify the target object.

[0109] Therefore, if the control device 1 determines that none of the images corresponding to the characteristic information of the extracted order information are identical to the image of the object, it may notify the touch panels 151 to 155, etc., of an error indicating an abnormality in the transported object. This notification may also include information from the camera that photographed the object. This allows each of the craftsmen SF1 to SF5 to take necessary actions, such as removing the transported object before it arrives at the destination table.

[0110] Furthermore, if the control device 1 determines that none of the images corresponding to the characteristic information of the extracted order information are identical to the image of the target object, it may identify the target object based on the next detection position and expected passage time of the order information registered in the order table. For example, if the order table is as shown in Figure 5, and the detection area where the target object was detected is detection area P35, the control device 1 may identify the transported object as the target object, which corresponds to the order information with the order ID "00115" whose next passage position is the same as the detection area where the target object was detected and which has the earliest expected passage time.

[0111] Next, a flowchart illustrating the operation of the product dispensing device in this embodiment will be described.

[0112] Figure 9 is a flowchart showing the operation of the control device 1 when it receives an order from a customer via a table touch system, according to one embodiment of the present invention.

[0113] Referring to Figure 9, the control device 1 determines whether or not it has received an order from a customer through one of the table touch systems 171 to 176 in Figure 1 (S101). The control device 1 repeats the process in step S101 until it determines that it has received an order from a customer.

[0114] In step S101, if it is determined that an order has been received from a customer (YES in S101), the control device 1 assigns an order ID to the order, creates new order information including the order ID, delivery destination, product type, order acceptance time, and status (S103), and registers the new order information in the order table (S105). Next, it selects from the touch panels 151 to 155 (Figure 1) which touch panel will display the order (the touch panel for the craftsman preparing the product) (S106). Subsequently, the control device 1 updates the screen of the selected touch panel so that the new order information is displayed (S107), and then terminates the process.

[0115] Figure 10 is a flowchart showing the operation of the control device 1 in one embodiment of the present invention when the control device 1 receives a predetermined operation via a touch panel indicating that the placement of the object to be transported onto the supply conveyor has been completed.

[0116] Referring to Figure 10, the control device 1 determines whether or not it has received a predetermined operation indicating that the loading of the object to be transported onto the supply conveyor has been completed, via one of the touch panels 151 to 155 (Figure 1) (S201). The control device 1 repeats the process of step S201 until it determines that it has received a predetermined operation indicating that the loading of the object to be transported onto the supply conveyor has been completed.

[0117] In step S201, if the control device 1 determines that a predetermined operation indicating the completion of placing the object to be transported onto the supply conveyor has been received (YES in S201), the control device 1 updates the screen of the touch panel that received the predetermined operation so that the completion of placing the object to be transported is reflected (S203), and updates the status of the order information corresponding to the object to be transported (S204). Subsequently, the control device 1 starts transporting the object by operating the supply conveyor corresponding to the touch panel that received the predetermined operation among the supply conveyors 102 to 106 (Figure 1) (S205). Subsequently, the control device 1 determines whether or not it has detected a target for management based on the image captured by the camera corresponding to the touch panel that received the predetermined operation among the cameras 121 to 125 (Figure 1) (S207). The control device 1 repeats the process in step S207 until it determines that it has detected a target for management.

[0118] In step S207, if it is determined that a managed object has been detected (YES in S207), the control device 1 extracts feature information from the image of the transported object by performing a feature extraction process (S900) described later on the image of the transported object. Next, the control device 1 adds the extracted feature information to the order information of the transported object (S208) and updates the status of the order information (S209). Next, the control device 1 sets the path that the transported object should take based on the location where the transported object was detected and the destination information in the order information of the transported object (S210). Next, the control device 1 identifies the next passing location based on the set path (S211) and calculates the estimated passing time (S213). Next, the control device 1 adds the identified detection area and the calculated estimated passing time to the order information corresponding to the transported object (S215) and terminates the process.

[0119] Figures 11 and 12 are flowcharts showing the operation of the control device 1 in one embodiment of the present invention when a target is detected by any of the cameras 131-137 and 181-184 in Figure 1.

[0120] Referring to Figure 11, the control device 1 determines whether or not a target object has been detected by any of the cameras 131-137 and 181-184 based on the images captured by each of the cameras 131-137 and 181-184 (S301). The control device 1 repeats the process in step S301 until it determines that a target object has been detected by any of the cameras 131-137 and 181-184.

[0121] In step S301, if it is determined that a transported object that is subject to management has been detected (YES in S301), the control device 1 extracts the characteristic information of the object by performing a feature extraction process (S900) described later on the image of the detected object that is subject to management. Next, the control device 1 extracts order information from the order table in which the detection area where the object was detected is the next passing position (S303). Next, the control device 1 selects one order information from the extracted order information (S305). The control device 1 compares the characteristic information of the selected order information with the characteristic information of the object (S307) and determines whether the transported object corresponding to the selected order information and the object are the same (S311).

[0122] In step S311, if it is determined that the object to be transported corresponding to the selected order information is the same as the target object (YES in S311), the control device 1 identifies the object to be transported corresponding to the selected order information as the target object (S313), and proceeds to the process in step S321 in Figure 12.

[0123] In step S311, if it is determined that the object to be transported and the target object corresponding to the selected order information are not the same (NO in S311), the control device 1 determines whether all of the extracted order information has been selected or not (S315). In step S315, if it is determined that none of the extracted order information has been selected (NO in S315), the control device 1 proceeds to the process in step S305.

[0124] If, in step S315, it is determined that all extracted order information has been selected (YES in S315), the control device 1 determines whether or not the target item can be identified based on the order information registered in the order table (S317).

[0125] If it is determined in step S317 that the object can be identified (YES in S317), the control device 1 identifies the object (S313) and proceeds to the process in step S321 in Figure 12.

[0126] If, in step S317, it is determined that the object cannot be identified (NO in S317), the control device 1 notifies an error (S319) and terminates the process.

[0127] Referring to Figure 12, in step S321, the control device 1 refers to the order information of the identified object (S321) and sets the path that the object should take (S325). The path may be set based on the position of the detection area of ​​the camera that captured the image of the object, the destination, and whether or not the loading space on the loading path at the destination is full (detection result of a sensor (not shown) that detects the loading space). Next, the control device 1 determines whether or not the camera that captured the image of the object has an extruder (S327). Specifically, if the camera that captured the image of the object is one of cameras 131 to 137 (Figure 1), it is determined that the camera has an extruder. On the other hand, if the camera that captured the image of the object is one of cameras 181 to 184 (Figure 1), it is determined that the camera does not have an extruder.

[0128] In step S327, if the camera that captured the image of the object has an extruder (YES in S327), the control device 1 determines whether or not it is necessary to have the extruder perform an operation based on the set path (S329).

[0129] If, in step S329, it is determined that it is necessary to operate the extruder (YES in S329), the control device 1 causes the extruder to perform the extrusion operation (S331) and proceeds to the process in step S333.

[0130] If, in step S327, it is determined that the camera that captured the image of the object does not have a corresponding extruder (NO in S327), or if, in step S329, it is determined that there is no need to operate the extruder (NO in S329), the control device 1 proceeds to the process in step S333.

[0131] In step S333, the control device 1 identifies the next passing position of the object based on the set path and calculates the predicted passing time for passing the identified next passing position (S333). Subsequently, the control device 1 updates the next passing position and predicted passing time in the order information of the object (S335) and terminates the process.

[0132] Figure 13 shows the subroutine for the feature extraction process (S900) in Figures 10 and 11.

[0133] Referring to Figure 13, in the feature extraction process of step S900, the control device 1 divides the target image into a plate image portion and a product image portion by performing edge processing on the target image (S901). Next, the control device 1 extracts feature points of the product shape from the product image portion (S903) and extracts feature points of the product pattern from the product image portion (S905). Subsequently, the control device 1 divides the product image portion into multiple regions and extracts color information (product color information) from each region (S907). Next, the control device 1 obtains the position of the product relative to a reference mark in the target image (S909). Subsequently, the control device 1 creates feature information that includes the feature points of the product shape, the feature points of the product pattern, the product color information, and the position of the product relative to the reference mark as the first to fourth feature information, respectively (S911), and returns it.

[0134] Figure 14 is a schematic diagram showing the respective neural networks 500 of the learning models 43 and 45 (Figure 2) in one embodiment of the present invention.

[0135] Referring to Figure 14, each of the neural networks 500 in learning models 43 and 45 (Figure 2) is a so-called hierarchical neural network, in which a large number of artificial neurons (shown as circles in Figure 14) are connected in a hierarchical structure. A hierarchical neural network comprises artificial neurons for input, artificial neurons for processing, and artificial neurons for output.

[0136] The problem data 510 is the target of processing by the neural network 500. The problem data 510 is acquired by artificial neurons for input in the input layer 501. The input layer 501 is composed of artificial neurons for input arranged in parallel. The problem data 510 is distributed to the artificial neurons for processing.

[0137] Processing artificial neurons are connected to input artificial neurons. The processing artificial neurons are arranged in parallel to form a hidden layer 502. The hidden layer 502 may consist of multiple layers. A neural network with three or more layers that includes a hidden layer 502 is called a deep neural network.

[0138] The neural network may be a so-called convolutional neural network. A convolutional neural network is a deep neural network constructed by alternately connecting convolutional layers and pooling layers.

[0139] The output artificial neurons output the training data 511 to the outside. The output artificial neurons constitute the output layer 503. The neural network 500 is trained so that when the problem data 510 is input, the training data 511 is output (specifically, the parameters of the training models 43 and 45 (Figure 2) are adjusted).

[0140] The training data used to create the learning model 43 may include multiple sets of sample images and the result of determining whether or not the sample image contains the object to be detected. In this case, the sample image becomes the problem data 510, and the result of determining whether or not the sample image contains the object to be detected becomes the training data 511. Through machine learning using this training data, the learning model 43 will output a result of determining whether or not the image contains the object to be detected when an image taken by each of the cameras 121-125, 131-137, and 181-184 in Figure 1 is input.

[0141] The training data used to create the learning model 45 may include multiple sets of pairs of two sample images and the result of determining whether the transported object depicted in those two sample images is the same or not. In this case, the two sample images become the problem data 510, and the result of determining whether the transported object depicted in those two sample images is the same or not becomes the training data 511. Through machine learning using this training data, the learning model 45 will output a result of determining whether the transported object depicted in a specific image and the target object are the same or not when given a specific image and an image of the target object as input.

[0142] (Effects of the embodiment)

[0143] In the above-described embodiment, the object is identified as one of the multiple transported objects based on images of each of the multiple transported objects passing through the first region on the transport path and an image of the object that has passed through the second region on the transport path. According to the above-described embodiment, there is no need to record an ID on the RF tag to identify the object, and the device configuration can be simplified.

[0144] In particular, when the goods being transported are sushi, there are times when multiple transported items, each carrying the same type of sushi topping, are transported along the same transport route at the same time. Despite these transported items having different appearances, they need to be transported to different destinations. According to the embodiment described above, even in such cases, each of the multiple transported items, each carrying the same type of sushi topping, can be distinguished based on images and transported to the required destination.

[0145] (others)

[0146] The characteristic information included in the order information may be extracted from images of the transported object as it passes through any of the detection areas P31-P37 and P81-P84 in Figure 1. The control device 1 may update the characteristic information of the order information corresponding to the object with the characteristic information extracted from the image of the object each time it identifies an object in any of the detection areas P31-P37 and P81-P84.

[0147] The identification of the target object may be performed by the extruder corresponding to the camera instead of the control device 1. In this case, the extruder may extract order information from the order table stored by the control device 1, or each extruder may store an order table that is synchronized at a predetermined timing.

[0148] The transport paths RT1-RT6, RT21-23, RT31-RT36, and RT41-RT44 of the product dispensing device in Figure 1 may be divided into multiple detection areas, and each of these detection areas may be equipped with multiple imaging means (cameras) to photograph the transported object as it passes through. This allows for the management of the state of the transported object throughout the entire transport paths RT1-RT6, RT21-23, RT31-RT36, and RT41-RT44. As a result, if an object on the transport path is removed by a craftsman or other person due to a product defect or other reason, this removal can be immediately reflected in the order information, enabling more advanced transport control.

[0149] The identification device of this embodiment may be used in devices other than product dispensing devices. For example, it may be used in an inspection device that transports intermediate or finished products in a factory and inspects the transported products in transit, as a device for identifying the transported products in transit.

[0150] The processing in the above-described embodiments and modifications may be performed by software or by hardware circuits. Furthermore, a program for performing the processing in the above-described embodiments may be provided, or the program may be recorded on a recording medium such as a CD-ROM, flexible disk, hard disk, ROM, RAM, or memory card and provided to the user. The program is executed by a computer such as a CPU. Alternatively, the program may be downloaded to the device via a communication line such as the Internet.

[0151] The embodiments and variations described above can be combined as appropriate.

[0152] The embodiments and modifications described above should be considered in all respects as illustrative and not restrictive. The scope of the invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope of the claims are intended to be included.

[0153] 1 Control device 11 Camera control unit 13 Order receiving unit 15 Order information management unit 17 Detection unit 18 Identity output unit (example of identity output means) 19 Selection unit 21 Classification unit (example of classification means) 23 Position acquisition unit (example of position acquisition means) 25 Feature point extraction unit (example of feature point extraction means) 27 Discrimination unit (example of discrimination unit) 29 Identification unit (example of identification means) 31 Extrusion control unit 33 Route setting unit 35 Prediction unit (example of prediction means) 37 Operation display unit 38 Communication unit 39 Storage unit 41 Order table 43, 45 Learning model 101 Circulation conveyor 102-106 Supply conveyor 107-109 Customer conveyor 110-113 Rollers 121-125, 131-137, 181-184 Camera (Examples of first and second shooting means) 141-147 Extrusion device (Examples of first and second extrusion devices) 151-155 Touch panel 161-163 Switching lever 171-176 Table touch system 500 Neural network 501 Input layer 502 Hidden layer 503 Output layer 510 Problem data 511 Training data BT1 Transported object (Example of transported object and target object) BT11 Plate (Example of mounting part) BT12 Product (Example of product) CR1, CR2 Center IM1 Image from which feature information is to be extracted IM11 Part of the plate image IM12 Part of the product image MK Reference mark (Example of reference mark) P21-P25, P31-P37, P81-P84 Detection area (Examples of first and second areas) RT1-RT6, RT21-RT23, RT31-RT36, RT41-RT44 Transport routes (example of transport route) RT11, RT12 Straight lanes SF1-SF5 Workers TB1-TB6 Tables TE Set route

Claims

1. A device for identifying objects being transported along a transport path, comprising: a first imaging means for sequentially photographing each of a plurality of objects being transported as they pass through a first region on the transport path; a second imaging means for photographing an object that has passed through a second region on the transport path after photographing each of the images of the plurality of objects; and a device for identifying which of the plurality of objects the object is based on each of the images of the plurality of objects and the image of the object.

2. The identification device according to claim 1, further comprising a trained model generated by machine learning, which takes a specific image from among the images of a plurality of transported objects and an image of the target object as input, and outputs a determination result of whether or not the transported object in the specific image and the target object are the same, wherein the identification means determines that the transported object in the specific image and the target object are the same, and the identification means identifies the target object as the transported object in the specific image.

3. The transported object includes a placement section including a reference mark and a product placed on the placement section, and further comprises: a division means for dividing each of the images of a plurality of transported objects into a portion of the image of the placement section and a portion of the image of the product; a position acquisition means for acquiring the position of the product with respect to the reference mark in each of the images of the specific object and the product; a feature point extraction means for extracting feature points from the portion of the image of the product in each of the images of the specific object and the product; and a determination means for determining whether the transported object of the specific image and the product are the same, based on the position of the product with respect to the reference mark in each of the images of the specific image and the product and the feature points extracted from the portion of the image of the product in each of the images of the specific image and the product, wherein the determination means determines that the transported object of the specific image and the product are the same, the determination means identifies the product as the transported object of the specific image.

4. The identification device according to claim 1, further comprising a prediction means for predicting the timing at which the object passes a required position on the transport path.

5. The identification device according to claim 1, comprising a plurality of imaging means including the first and second imaging means, wherein when the entire transport path is divided into a plurality of regions including the first and second regions, each of the plurality of imaging means photographs the transported object as it passes through each of the plurality of regions.

6. The identification device according to claim 1, wherein the transport path includes an annular transport path and a branch path that branches off from a predetermined position on the annular transport path and proceeds toward the customer who ordered the transported items, and the identification means identifies which of the plurality of transported items the object is, and then pushes the object from the predetermined position on the annular transport path toward the branch path.

7. The identification device according to claim 6, wherein the transport path further includes a shortcut path that shortens the distance from a branching point on the annular transport path to a merging point on the annular transport path, and after the identification means identifies which of the plurality of transported objects the object is, the identification device further includes a second extrusion device that pushes the object from the annular transport path to the shortcut path at the branching point.

8. A device for identifying objects to be transported along a transport path, comprising: a first imaging means for sequentially photographing each of a plurality of objects as they pass through a first region on the transport path; a second imaging means for photographing an object that has passed through a second region on the transport path after photographing each of the images of the plurality of objects; an identification means for identifying which of the plurality of objects the object is; a feature point extraction means for extracting feature points from the image portion of the product in each of the images of the specific object and the image of the object; a determination means for determining whether the object in the specific image and the image of the object are the same based on the feature points extracted by the feature point extraction means from the image portion of the product in each of the images of the specific object and the image of the object; a path setting means for setting a path that each of the plurality of objects should take based on the first region and the transport destination of each of the plurality of objects; and a prediction means for predicting the timing at which each of the plurality of objects will pass through the second region based on the path set by the path setting means. If the determination means determines that the object to be transported in the specific image is the same as the object to be transported, the determination means identifies the object to be transported in the specific image. If the determination means determines that none of the images of the plurality of objects to be transported are the same as the image of the object to be transported, the determination means identifies which of the plurality of objects the object to be transported is based on the timing at which each of the plurality of objects to be transported passes through the position of the second region.

Images