Specific device

Abstract

To provide a specific device that can simplify the configuration of the device. [Solution] The product dispensing device is a identification device that identifies objects being transported along a transport route. It comprises cameras 121 to 125 that sequentially photograph each of multiple objects being transported as they pass through each of the detection areas P21 to P25 on the transport route, cameras 131 to 137 and 181 to 184 that, after photographing each of the images of the multiple objects being transported, photograph the object as it passes through each of the detection areas P31 to P37 and P81 to P84 on the transport route, and identification means that identifies which of the multiple objects the object is based on each of the images of the multiple objects being transported and the image of the object.

Description

Technical Field

[0001] The present invention relates to a specific device. More specifically, the present invention relates to a specific device that identifies an object to be conveyed along a conveyance path.

Background Art

[0002] Conventionally, in a revolving sushi restaurant or the like, products placed on plates are conveyed along a conveyance path to a customer's table. In such a store, since a plurality of products are conveyed in parallel to their respective required tables, it is necessary to identify the products being conveyed. Techniques for identifying products being conveyed are disclosed in, for example, Patent Document 1 below.

[0003] Patent Document 1 below discloses a device that identifies a product being conveyed by providing an RF tag recording an ID for identifying the product on the plate and reading the RF tag.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the device of Patent Document 1, a configuration for recording an ID on the RF tag provided on the plate is required. Therefore, in the device of Patent Document 1, the device configuration is complicated. The problem of complicating the device configuration was a problem occurring in general specific devices that identify an object to be conveyed along a conveyance path.

[0006] The present invention is for solving the above problems, and its object is to provide a specific device capable of simplifying the device configuration.

Means for Solving the Problems

[0007] A identifying device according to one aspect of the present invention is an identifying device for identifying an object being transported along a transport path, the object being transported includes a product, and comprises: a first imaging means for sequentially photographing each of a plurality of objects being transported as they pass through a first region on the transport path; a second imaging means for photographing an object that has passed through a second region on the transport path after photographing each of the images of the plurality of objects; an identifying means for identifying which of the plurality of objects the object is; and a feature point extraction means for extracting feature points from the portion of the product image in each of the images of the specific object and the image of the object from among the images of the plurality of objects. The system further includes a determination means that determines whether the object being transported in the specific image and the object depicted in the object image are the same, based on feature points extracted by a feature point extraction means from the portion of the product image in each of the specific image and the object image. If the determination means determines that the object being transported in the specific image and the object depicted in the object image are the same, the determination means identifies the object as the object being transported in the specific image. If the determination means determines that none of the multiple objects being transported are the same as the object depicted in the object image, the system further includes a notification means that notifies an error indicating an abnormality in the transported object.

[0008] Preferably, in the above-described specific device, the notification means also notifies information from the second imaging means that photographed the object. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a specific device that can simplify the configuration of the device. [Brief explanation of the drawing]

[0010] [Figure 1] This is a plan view showing the configuration of a store employing a product dispensing device according to one embodiment of the present invention. [Figure 2] This is a block diagram showing the functional configuration of the control device 1. [Figure 3] This diagram schematically shows the order table immediately after registering order information based on a new order. [Figure 4]This diagram schematically shows the order table immediately after a predetermined operation is received on the touch panel 151 and the order information is updated. [Figure 5] This diagram schematically shows the order table immediately after the next passing position and estimated passing time have been added to the order information. [Figure 6] This diagram shows the path taken by an object when it is transported from the supply conveyor 102 to the table TB5. [Figure 7] This diagram shows the path taken by the object to be transported to table TB4 when the loading space on the transport path RT34 is full. [Figure 8] This diagram illustrates an example of a method for extracting characteristic information from a transported object. [Figure 9] This flowchart shows the operation of the control device 1 when it receives an order from a customer via a table touch system, according to one embodiment of the present invention. [Figure 10] This flowchart shows the operation of the control device 1 in one embodiment of the present invention when the control device 1 receives a predetermined operation via a touch panel indicating that the placement of the object to be transported onto the supply conveyor has been completed. [Figure 11] This is a first flowchart showing the operation of the control device 1 when a target is detected by any of the cameras 131-137 and 181-184 in one embodiment of the present invention. [Figure 12] This is a second flowchart showing the operation of the control device 1 when a target is detected by any of the cameras 131-137 and 181-184 in one embodiment of the present invention. [Figure 13] This is the subroutine for the feature extraction process (S900) shown in Figures 10 and 11. [Figure 14] This figure schematically shows the neural networks 500 of learning models 43 and 45 (Figure 2) in one embodiment of the present invention. [Modes for carrying out the invention]

[0011] FIG. 1 is a plan view showing the configuration of a store that employs the merchandise providing apparatus according to an embodiment of the present invention.

[0012] Referring to FIG. 1, the merchandise providing apparatus (an example of a specific apparatus) in the present embodiment is provided in a store equipped with conveying means for conveying merchandise placed on a plate along a conveyance path and circulating it within a dining area, such as a conveyor belt sushi restaurant. The type of merchandise is arbitrary and typically food and drink items such as sushi, light meals, or beverages. A plurality of tables TB1 to TB6, which are customer tables, and seats are provided along the vicinity of the merchandise conveyance path within the store. Each of tables TB1 to TB6 and the seats is for a group of multiple customers.

[0013] The merchandise providing apparatus in the present embodiment is an apparatus that conveys and provides merchandise to each customer using a conveyance path different from the above-described "conveying means for conveying merchandise placed on a plate along a conveyance path and circulating it within a dining area". The merchandise providing apparatus in the present embodiment does not circulate merchandise within the dining area, but directly delivers an object to be conveyed, including the merchandise corresponding to an order received from a customer, from the kitchen area to the customer's table in the dining area. The merchandise providing apparatus identifies the object to be conveyed that is conveyed along the conveyance path and conveys the object to be conveyed to the necessary customer's table.

[0014] In Fig. 1, the lower side is the kitchen area where a craftsman cooks goods, and the upper side in Fig. 1 is the dining area where customers eat and drink. The goods providing apparatus in the present embodiment includes conveyance paths RT1 to RT6, RT21 to 23, RT31 to RT36, and RT41 to RT44 (an example of conveyance paths), a control device 1, a circulation conveyor 101, supply conveyors 102 to 106, customer conveyors 107 to 109, rollers 110 to 113, cameras 121 to 125, 131 to 137, and 181 to 184, extrusion devices 141 to 147, touch panels 151 to 155, switching levers 161 to 163, and table touch systems (order input devices) 171 to 176. Note that the number and installation positions of each of the conveyance paths, control device, conveyors, rollers, cameras, extrusion devices, touch panels, table touch systems, switching levers, and table touch systems are arbitrary.

[0015] Each of the conveyance paths RT1 to RT6, RT21 to 23, RT31 to RT36, and RT41 to RT44 is a path for conveying an object to be conveyed. The object to be conveyed includes a plate (an example of a placement part) and goods placed on the plate. It is preferable that a reference mark is given to the plate. The goods may be placed on something other than the plate or may not be placed on the placement part.

[0016] The conveyance path RT1 is annular and provided within the kitchen area. The conveyance path RT1 includes a linear lane RT11 extending linearly on the craftsman side and a linear lane RT12 extending linearly on the customer side.

[0017] Each of the conveyance paths RT2 to RT6 is provided within the kitchen area. Each of the conveyance paths RT2 to RT6 extends in the horizontal direction in Fig. 1 and merges with the conveyance path RT1 at the right end in Fig. 1. Between the right end in Fig. 1 of each of the conveyance paths RT2 to RT6 and the conveyance path RT1, conveying means (not shown) such as rollers for supplying the object to be conveyed conveyed along each of the conveyance paths RT2 to RT6 onto the conveyance path RT1 are provided.

[0018] Each of the transport routes RT21 to RT23 (an example of a branched route) is straight and branches off from a predetermined position on transport route RT1, heading towards each of the tables TB1 to TB6 in the seating area.

[0019] Each of the transport routes RT31 to RT36 is a location where customers seated at each of the tables TB1 to TB6 receive their goods. For goods ordered by customers seated at each of the tables TB1 to TB6, the transport route corresponding to that table among the transport routes RT31 to RT36 is the destination. Each of the transport routes RT31 to RT36 branches off from one of the transport routes RT21 to RT23 and extends to the vicinity of one of the tables TB1 to TB6.

[0020] Each of the transport routes RT41 to RT44 (an example of a shortcut route) shortcuts transport route RT1 from a predetermined branching point to a predetermined merging point.

[0021] Control device 1 controls the entire product dispensing system. Control device 1 consists of a computer such as a PC (Personal Computer) or smartphone, and includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), an operation unit, a display unit, and a network interface. The CPU executes the control program stored in ROM. ROM stores various programs executed by the CPU and various fixed data. RAM is used to temporarily store data necessary when the CPU executes the control program. The operation unit consists of a keyboard and mouse, and accepts various inputs. The display unit displays various information. The network interface communicates with other devices using communication protocols such as TCP / IP.

[0022] The circulating conveyor 101 transports the goods along the transport path RT1. Each of the supply conveyors 102 to 106 transports the goods along each of the transport paths RT2 to RT6 and supplies the goods to the circulating conveyor 101. Each of the customer conveyors 107 to 109 transports the goods along each of the transport paths RT21 to RT23. Each of the rollers 110 to 113 transports the goods along each of the transport paths RT41 to RT44. The circulating conveyor 101, customer conveyors 107 to 109, and rollers 110 to 113 are all driven continuously, while each of the supply conveyors 102 to 106 is driven at the required timing under the control of the control device 1.

[0023] Cameras 121-125, 131-137, and 181-184 each have detection areas P21-P25, P31-P37, and P81-P84 (areas enclosed by dotted lines in Figure 1) on one of the transport paths RT1-RT6, RT21-23, and RT31-RT36, respectively. Cameras 121-125, 131-137, and 181-184 each sequentially photograph each of the multiple transported objects passing through their corresponding detection area. In particular, cameras 131-137 and 181-184 each photograph the target object that has passed through their corresponding detection area after having captured images of each of the multiple transported objects with one of the cameras 121-125.

[0024] Each of the extrusion devices 141 to 147, under the control of the control device 1, extrudes the material being transported along the transport path RT1 to another transport path as needed. Each of the extrusion devices 141 to 147 may consist of a computer.

[0025] Each of the touch panels 151 to 155, under the control of the control device 1, displays various information to each of the craftsmen SF1 to SF5 and accepts various operations from each of the craftsmen SF1 to SF5. Each of the touch panels 151 to 155 may be a computer.

[0026] Each of the switching levers 161 to 163 switches the destination of the transported object along each of the transport paths RT21 to RT23.

[0027] Each of the table touch systems 171 to 176 is installed on each of the tables TB1 to TB6. Each of the table touch systems 171 to 176 is equipped with a display and input device such as a touch panel display. Each of the table touch systems 171 to 176, under the control of control device 1, displays various information such as menus to the customers at that table and accepts various operations such as orders from the customers at that table. Each of the table touch systems 171 to 176 may consist of a computer.

[0028] Each of the table touch systems 171 to 176 transmits the details of the received order (type and quantity of goods) to the control device 1 when it receives an order from a customer. The goods corresponding to the orders entered into each of the table touch systems 171 to 176 are delivered directly from the kitchen to the customer's table by the goods serving device in this embodiment.

[0029] At checkout, each of the control device 1 or table touch systems 171-176 transmits checkout information (types and quantities of goods provided and consumed) to the self-checkout register (not shown) based on the order information stored for the customer group corresponding to that table. The self-checkout register calculates the amount based on the transmitted information, presents it to the customer, and also receives payment from the customer and returns change.

[0030] Control device 1 stores the order table. When control device 1 receives order details from any of the table touch systems 171 to 176, it assigns an order ID to the order details and registers them in the order table as new order information.

[0031] Furthermore, if the table touch system that receives the order creates the order information itself, and the control device 1 receives the order information from the table touch system, it may register that order information in the order table.

[0032] Figure 2 is a block diagram showing the functional configuration of the control device 1.

[0033] Referring to Figure 2, the functions of the control device 1 are realized by the CPU executing various programs stored in the ROM. The control device 1 includes a camera control unit 11, an order receiving unit 13, an order information management unit 15, a detection unit 17, an identity output unit 18 (an example of identity output means), a selection unit 19, a classification unit 21 (an example of classification means), a position acquisition unit 23 (an example of position acquisition means), a feature point extraction unit 25 (an example of feature point extraction means), a discrimination unit 27 (an example of discrimination unit), a specific unit 29 (an example of specific means), an extrusion control unit 31, a route setting unit 33, a prediction unit 35 (an example of prediction means), an operation display unit 37, a communication unit 38, and a storage unit 39.

[0034] The camera control unit 11 controls the shooting operation of each of the cameras 121-125, 131-137, and 181-184 shown in Figure 1. The images captured by each of the cameras 121-125, 131-137, and 181-184 may be still images or videos.

[0035] The order receiving unit 13 receives the order details from each of the table touch systems 171 to 176, creates order information by assigning an order ID to the order details, and registers that order information in the order table.

[0036] The order information management unit 15 updates the order information registered in the order table at the necessary time.

[0037] The detection unit 17 detects the target object in each of the detection regions P21-P25, P31-P37, and P81-P84, respectively, based on the images captured by each of the cameras 121-125, 131-137, and 181-184 in Figure 1. The detection unit 17 may also use the learning model 43 to detect the target object.

[0038] The managed object is the transported object that is the subject of the feature extraction process described later. The type of managed object is arbitrary. For example, the managed object may include not only the transported object including the plate and the product placed on the plate, but also accessories such as spoons, foreign objects (such as pieces of product that have fallen from the plate, such as french fries), etc. Furthermore, if the product serving device is installed in a conveyor belt sushi restaurant, the managed object may include the transported object including a circular serving container, or the transported object may include a container with a circular planar shape on which a bowl for soup such as udon or miso soup is placed. In addition, the managed object may include plates that are used to place sushi, beverages, chawanmushi (steamed egg custard), or desserts. On the other hand, conveyors and rollers that transport the transported object, and transported objects such as plates with promotional images placed on them, may be excluded from the managed object.

[0039] The identity output unit 18 uses the learning model 45 to take as input one image (sometimes referred to as a specific image) selected by the selection unit 19 from among multiple images of transported objects, and an image of the transported object (sometimes referred to as an object) captured by one of the cameras 131-137 and 181-184 in Figure 1, and outputs a determination result of whether the transported object shown in the specific image and the object (the object shown in the image of the object) are the same.

[0040] The selection unit 19 selects one image from among the images of multiple objects to be transported, each captured by cameras 121 to 125 (Figure 1).

[0041] The division unit 21 divides each of the images of multiple transported objects into a portion of the plate image and a portion of the product image.

[0042] The position acquisition unit 23 acquires the position of the product relative to a reference mark based on the portion of the plate image and the portion of the product image in each of the specific image and the image of the object.

[0043] The feature point extraction unit 25 extracts feature points from the product image in both the specific image and the image of the object.

[0044] The discrimination unit 27 determines whether the transported object shown in the specific image and the object (the object shown in the image of the object) are the same, based on the position of the product relative to a reference mark in each of the specific image and the image of the object, and feature points extracted from the portion of the product image in each of the specific image and the image of the object.

[0045] The identification unit 29 identifies which of the multiple transported objects the object is based on each of the images of the multiple transported objects captured by cameras 121 to 125 (Figure 1) and the image of the target object.

[0046] The identification unit 29 may identify the transported object based on the determination result by the identity output unit 18, or it may identify the transported object based on the determination result by the determination unit 27. Specifically, if the identity output unit 18 determines that the transported object in a specific image and the target object are the same, the identification unit 29 may identify the target object as the transported object shown in the specific image. Alternatively, if the determination unit 27 determines that the transported object shown in a specific image and the target object are the same, the identification unit 29 may identify the target object as the transported object shown in the specific image.

[0047] The extrusion control unit 31 controls the operation of each of the extrusion devices 141-147 and the switching levers 161-163 shown in Figure 1.

[0048] The route setting unit 33 sets the path that the object should take at the necessary timing.

[0049] The prediction unit 35 predicts the expected passing time (an example of passing timing) for the object to pass through the required positions (in this embodiment, the detection area of ​​the camera that the object will next pass through) on the transport paths RT1~RT6, RT21~23, RT31~RT36, and RT41~RT44 shown in Figure 1, based on the path set by the path setting unit 33.

[0050] The operation display unit 37 receives various operations and displays various information.

[0051] The communication unit 38 communicates with the circulation conveyor 101, supply conveyors 102-106, customer conveyors 107-109, rollers 110-113, cameras 121-125, 131-137, and 181-184 shown in Figure 1, as well as with the extrusion devices 141-147, touch panels 151-155, switching levers 161-163, and table touch systems 171-176, etc.

[0052] The memory unit 39 stores various information such as the order table 41, the learning model 43, and the learning model 45.

[0053] The learning model 43 is a model generated by machine learning. The learning model 43 takes images taken by cameras 121-125, 131-137, and 181-184 as input and outputs a result of determining whether or not the image contains the object to be transported.

[0054] The learning model 45 is a model generated by machine learning. The learning model 45 takes a specific image and an image of an object as input and outputs a result of determining whether the transported object and the object shown in the specific image are the same.

[0055] Figure 3 schematically shows the order table immediately after registering order information based on a new order.

[0056] Referring to Figures 1 and 3, the order table 41 (Figure 2) contains multiple order entries in the order in which they were received. The order information includes the following items: "Order ID," "Destination," "Product Type," "Order Received Time," "Status," "Feature Information," "Next Passing Location," and "Expected Passing Time." "Order ID" is information that identifies the order. "Destination" is information that identifies the table corresponding to the table touch system that sent the order (the table to which the product will be delivered). "Product Type" is the type of product related to the order. "Order Received Time" is the time when the table touch system received the order. "Status" is the state of the ordered product. "Status" can be one of the following: "Preparing," "Feature Extraction," "Transporting," or "Transportation Completed." "Feature Information" is information about the characteristics of the transported object, including the product being transported. "Next Passing Location" is information indicating the detection area that the transported object, including the product being transported, will pass through next. "Expected Passing Time" is the time when the transported object, including the product being transported, is expected to pass through the next passing location.

[0057] Here, we assume that the control device 1 (Figure 2) receives a new order for "Scallop Sushi" from a customer at table TB5 (Figure 1) via the table touch system 175. Upon receiving this order, the control device 1 associates the information identifying the destination of the order (table TB5 of the customer who placed the order), the "product type" (the type of product related to the order), and the "order acceptance time" (the time the order was accepted), and creates new order information with an arbitrary order ID of "00119" as shown in Figure 3. The control device 1 registers the created order information at the bottom of the order table. At this time, the "Status" field is filled with the status "Preparing," indicating that preparations are being made for the delivery of the product. The "Feature Information," "Next Passing Position," and "Expected Passing Information" fields in the order table are left blank at this point.

[0058] Referring to Figure 1, an example is shown of a kitchen area where five chefs SF1 to SF5 are each preparing food. In front of each chef SF1 to SF5 (upper direction in Figure 1), there are supply conveyors 102 to 106 and touch panels 151 to 155 for the chefs. Here, corresponding to each of the five chefs SF1 to SF5, there are five supply conveyors 102 to 106 and five touch panels 151 to 155, each installed in front of the chef.

[0059] The circulating conveyor 101 circulates the conveyed items along the annular transport path RT1 within the kitchen. The rotation direction of the circulating conveyor 101 is counterclockwise, as indicated by the black arrow within the transport path RT1. The rotation direction of the circulating conveyor 101 is arbitrary and may be clockwise.

[0060] Regardless of which of the craftsmen SF1 to SF5 is in charge of preparing the products, the operation of the craftsman and the product dispensing equipment is the same. Here, we will assume that craftsman SF5 is in charge of preparing the products. When new order information is registered in the order table, the control device 1 displays the order information on the craftsman's touch panel 155 installed in front of the assigned craftsman SF5, thereby issuing instructions for preparing the product (such as the type of product and the ID of the destination table). Seeing this, craftsman SF5 prepares the product (cooks the product if it is sushi), places the prepared product on a plate, and places the product on the plate onto the supply conveyor belt 106. Hereafter, the plate and the product placed on it may be referred to as the conveyed object.

[0061] After placing the items to be transported corresponding to the order onto the supply conveyor 106, the worker SF5 performs a predetermined operation on the touch panel 155 to indicate that the placement of the items has been completed (for example, touching the display area for the order corresponding to the items). Upon receiving this operation, the control device 1 operates the supply conveyor 106. As a result, the items placed on the supply conveyor 106 are supplied to the straight lane RT11 via the transport path RT6, as indicated by the black arrow. Upon receiving this operation, the control device 1 also updates the order information in the order table corresponding to the new order.

[0062] Figure 4 schematically shows the order table 41 (Figure 2) immediately after a predetermined operation is received on the touch panel 155 (Figure 1) and the order information is updated.

[0063] Referring to Figures 1 and 4, when the control device 1 receives a predetermined operation from the craftsman SF5 on the touch panel 151 indicating that the loading of the transported object corresponding to the order information containing the order ID "00119" has been completed, the control device 1 updates the "Status" column of the order information to "Feature Extraction in Progress". "Feature Extraction in Progress" indicates that feature information is being extracted from the image of the transported object.

[0064] Referring to Figure 1, each of the cameras 121 to 125 (an example of the first imaging means) sequentially images each of the multiple transported objects passing through the corresponding detection area. Each of the detection areas P21 to P25 (an example of the first area) may be located on the annular transport path RT1, or on each of the transport paths RT2 to RT6. The control device 1 extracts feature information from the images of each of the multiple transported objects using the extraction method described later. The control device 1 adds the extracted feature information to the order information corresponding to that transported object. As a result, the order information in the order table further includes the feature information of the transported object corresponding to that order information. The feature information of the transported object is used to identify the transported object during transport.

[0065] If the transported item corresponds to order information containing the order ID "00119", the control device 1 extracts characteristic information of the transported item from the image captured by the camera 125 as the transported item passes through the detection area P25. The control device 1 then adds the extracted characteristic information to the order information of the transported item (Figure 5).

[0066] After adding the extracted feature information to the order information, the control device 1 sets the path TE that the transported object should take, based on the detection area P25 (Figure 1) of the camera 125 that photographed the transported object and the position of the destination table TB5 included in the order information. Then, based on the set path TE, the control device 1 identifies the next position the transported object will pass and predicts the estimated time of passage. The next position is, for example, one of the detection areas P31-P37 and P81-P84 (an example of a second area) of cameras 131-137 and 181-184. In this case, detection area P35, which is adjacent to detection area P25 downstream along path TE, is identified as the next position. The estimated time of passage may be calculated based on the current time, the distance along path TE from the current detection position to the next detection position, and the speed at which the transported object is being carried.

[0067] Furthermore, for purposes such as product modification, a transported item may be removed by a worker while being transported along transport route RT1 and returned to the transport route. Also, a transported item may be pushed by other transported items. When such situations occur, the actual time when the transported item passes the next destination will deviate from the expected passing time.

[0068] Figure 5 schematically shows the order table immediately after the next passing position and estimated passing time have been added to the order information.

[0069] Referring to Figures 1 and 5, after extracting the characteristic information of the transported object, the control device 1 updates the "Status" column of the order information containing the order ID "00119" to "Transporting". "Transporting" indicates that the transported object is being transported along one of the transport routes RT1-RT6, RT21-23, and RT41-RT44. The control device 1 also adds four pieces of characteristic information, "00119a", "00119b", "00119c", and "00119d", to the "Characteristic Information" column of the order information containing the order ID "00119" in the order table. After identifying the next passing position and predicting the expected passing time, the control device 1 adds the information "P35", which identifies the detection area P35, to the next passing position column, and adds the expected passing time "2024 / 06 / 21 12:55:40".

[0070] In this embodiment, in order to distinguish the four feature information in each order information, the symbols "a", "b", "c", or "d" are added to the end of each order ID as the name of the feature information. The feature information with the symbol "a" added to the end of the order ID (for order information containing the order ID "00119", the feature information is named "00119a") is referred to as the first feature information. The feature information with the symbol "b" added to the end of the order ID (for order information containing the order ID "00119", the feature information is named "00119b") is referred to as the second feature information, and the feature information with the symbol "c" added to the end of the order ID (for order information containing the order ID "00119", the feature information is named "00119c") is referred to as the third feature information. The characteristic information of a name with the symbol "d" appended to the end of the order ID (for example, the characteristic information "00119d" in order information containing the order ID "00119") is sometimes referred to as the fourth characteristic information.

[0071] Referring to Figure 1, ordered goods are delivered to the customer's table based on the order information. Cameras 131-137 are provided at necessary positions on the annular transport path RT1 to photograph the transported items during transport. In this embodiment, each of the cameras 131-137 corresponds to each of the extruders 141-147. Each of the cameras 131-137 photographs the transported items as they pass through each of the detection areas P31-P37. The control device 1 extracts feature information from the images of the transported objects captured by each of the cameras 131-137, and identifies which transported item it is based on the extracted feature information. The control device 1 then sets the path that the transported item should take based on the destination information included in the order information corresponding to the identified transported item. The set path is basically the same as an already set path, but may differ depending on the situation. Based on the set path, the control device 1 identifies the next passing position and calculates the expected passing time. The control device 1 updates the next passing position and estimated passing time in the order information corresponding to the identified transported item. Furthermore, the control device 1 controls the operation of each of the extruders 141-147 corresponding to the cameras that captured the images, based on the set route. This ensures that the transported item, including the product, is correctly delivered to the customer's table (destination) that ordered the product.

[0072] Each of the transport routes RT21 to RT23 branches off from a predetermined position on the customer-side straight lane RT12. The customer conveyor 107 transports items to table TB1 or TB2 along transport route RT21, and by operating the downstream switching lever 161, goods are transported from transport route RT21 to either transport route RT31 or RT32. Transport route RT31 is the transport route that provides goods to customers at table TB1, and transport route RT32 is the transport route that provides goods to customers at table TB2.

[0073] The customer conveyor 108 transports items to table TB3 or TB4 along the transport path RT22. By operating the downstream switching lever 162, the goods are transported from transport path RT22 to either transport path RT33 or RT34. Transport path RT33 is the transport path that provides goods to customers at table TB3, and transport path RT34 is the transport path that provides goods to customers at table TB4.

[0074] The customer conveyor 109 transports items to tables TB5 or TB6 along the transport path RT23. By operating the downstream switching lever 163, the goods are transported from transport path RT23 to either transport path RT35 or RT36. Transport path RT35 is the transport path that delivers goods to customers at table TB5, and transport path RT36 is the transport path that delivers goods to customers at table TB6.

[0075] Extruders 145 to 147 (an example of a first extruder) are provided to push the conveyed objects, which are being transported along the straight lane RT12, into the respective transport paths RT21 to RT23. When each of the extruders 145 to 147 operates, the conveyed objects that have been transported to a position on the straight lane RT12 that connects to the respective transport paths RT21 to RT23 are pushed out into the respective transport paths RT21 to RT23.

[0076] Each of the transport paths RT41 to RT44 is a shortcut path that takes a shortcut from a branching point on the annular transport path RT1 to a merging point on the annular transport path RT1. Each of the transport paths RT41 and RT42 is a shortcut path that takes the transported material from straight lane RT11 to straight lane RT12. Each of the transport paths RT43 and RT44 is a shortcut path that takes the transported material from straight lane RT12 to straight lane RT11. Each of the rollers 110 to 113 transports the required transported material along each of the transport paths RT41 to RT44. The transport direction of each of the rollers 110 to 113 is indicated by a black arrow in each of the transport paths RT41 to RT44. Each of the branching points of the transport paths RT41 to RT44 from the annular transport path RT1 is provided with an extruder 141 to 144 (an example of a second extruder).

[0077] When the extrusion device 141 or 142 operates, the object to be transported, which has been transported to the branching point with the transport path RT41 or RT42 on the straight lane RT11, is pushed out onto the transport path RT41 or RT42. The pushed-out object is transported along the transport path RT41 or RT42 by the rollers 110 or 111 and moves to the merging point with the transport path RT41 or RT42 on the straight lane RT11. As a result, the object that has passed through the transport path RT41 or RT42 moves from a predetermined branching point on the straight lane RT11 to a predetermined merging point on the straight lane RT12 without having to go around the annular transport path RT1.

[0078] Figure 6 shows the path taken by an object when it is transported from the supply conveyor 102 to the table TB5.

[0079] Referring to Figure 6, when transporting an object from the supply conveyor 102 to the table TB5 (assuming the loading space on the transport path RT35 is not full), the control device 1 operates the extruder 141, the extruder 147, and the switching lever 163 at the necessary timings. As a result, the object is transported to the transport path RT35 corresponding to the table TB5, passing through the straight lane RT11, the transport path RT41, the straight lane RT12, and the transport path RT23 in that order, as shown by arrow AR1 in Figure 6.

[0080] Referring to Figure 1, when the extruder 143 or 144 operates, the object to be transported, which has been transported to the branching point with the transport path RT43 or RT44 on the straight lane RT12, is pushed out onto the transport path RT43 or RT44. The pushed-out object is transported along the transport path RT43 or RT44 by the rollers 112 or 113 and moves to the merging point with the transport path RT43 or RT44 on the straight lane RT11. As a result, the object that has passed through the transport path RT43 or RT44 moves from a predetermined branching point on the straight lane RT12 to a predetermined merging point on the straight lane RT11 without having to go around the annular transport path RT1. Therefore, if the path to the destination is shortened by passing through the transport path RT43 or RT44, the control device 1 sets the path that the object should take to pass through the transport path RT43 or RT44.

[0081] If a transported object accumulates in any of the transport paths RT31 to RT36, the loading space in that transport path becomes full, and it becomes impossible to send the next transported object to that transport path. Therefore, if the control device 1 detects that the loading space in any of the transport paths RT31 to RT36 is full using a sensor (not shown), it sets (changes) the path that the transported object should take to a circulating path and controls the operation of the necessary extruders 145 to 147 according to the set path. This prevents the transported object accumulated in the transport path from being pushed out by the newly transported object. The transported object whose path has been changed will circulate along the annular transport path RT1 until there is space available in the loading space of the destination transport path. The control device 1 detects when the loading space of the transported object in a transport path is full using sensors (not shown) provided in each of the transport paths RT31 to RT36.

[0082] Furthermore, the control device 1 controls the transport to ensure that the transported item does not move far from the destination table by using each of the transport paths RT41 to RT44, so that the transported item can be immediately transported to the destination table when there is space available on the transport path at the destination.

[0083] Figure 7 shows the path taken by the object to be transported to the table TB4 when the loading space on the transport path RT34 is full.

[0084] Referring to Figure 7, for example, when the loading space of the transport path RT34 corresponding to table TB4 is full, and the camera 136 identifies the transported object as one to be transported to table TB4, the control device 1 sets (changes) the route in the circulation path. The control device 1 operates the extruder 144 without operating the extruder 146 to push the transported object from the straight lane RT22 to RT11. The control device 1 also operates the extruder 142 to push the transported object from the straight lane RT11 to RT12. This allows the transported object to be transported to table TB4 to wait on the circulation path, which consists of the central parts of the straight lanes RT11 and RT12 and the transport paths RT44 and RT42. This circulation path is indicated by arrow AR2 in Figure 7.

[0085] Then, when there is free space on the transport path RT34 corresponding to table TB4, the control device 1 sets (changes) the route so that the objects to be transported waiting on the circulation path move toward the transport path RT22. The control device 1 then operates the extruder 146 at the necessary timing to transport the objects from the straight lane RT22 to the transport path RT34, as shown by arrow AR3 in Figure 7. While waiting on the circulation path, objects to be transported to table TB4 are not transported to the transport path RT1 to the left of the transport path RT44 or to the transport path RT1 to the right of the transport path RT42. Therefore, when there is free space on the transport path RT34 corresponding to table TB4, the objects to be transported can be quickly transported from the circulation path to the transport path RT34.

[0086] Referring to Figure 1, each of the cameras 181 to 184 has detection areas P81 to P84 near the junction with the annular transport path RT1 in the transport paths RT41 to RT44. Each of the cameras 181 to 184 photographs the transported object as it passes through each of the detection areas P81 to P84. The control device 1 extracts feature information from the images of the transported object captured by each of the cameras 181 to 184 and identifies the transported object based on the extracted feature information. Each of the detection areas P81 to P84 may be provided on the annular transport path RT1 or on each of the transport paths RT41 to RT44. By providing each of the cameras 181 to 184, it is possible to detect when the transported object that has been transported on each of the transport paths RT41 to RT44 has returned to the annular transport path RT1, thereby improving transport accuracy.

[0087] When the control device 1 detects, based on an image captured by a camera (not shown), that an item has arrived at the destination transport route (table) among the transport routes RT31 to RT36, it may update the "Status" column of the order information corresponding to that item in the order table to "Transportation Completed". Detecting transport completion using a camera is optional.

[0088] The product dispensing device configured as described above makes it possible to provide products simply and quickly.

[0089] Next, we will explain an example of a method for extracting feature information. The feature information can be any information extracted from an image captured by a camera, and its type and combination are arbitrary.

[0090] Figure 8 illustrates an example of a method for extracting feature information. In this embodiment, an example is shown in which feature information is extracted using an image of the transported object taken from above, but the direction in which the image of the transported object is taken is arbitrary.

[0091] Referring to Figure 8(a), the target image IM1 typically shows a transported object BT1 (an example of a transported object and target object), which includes a plate BT11 (an example of a mounting area) and a product BT12 (an example of a product) placed on the plate BT11. The plate BT11 contains an arbitrary number of reference marks MK (an example of reference marks). Reference marks MK are, for example, the store's logo.

[0092] Referring to Figure 8(b), the control device 1 divides image IM1 into a portion of the plate image IM11 and a portion of the product image IM12 by edge processing. The control device 1 then extracts feature points from the portion of the product image IM12. Specifically, the control device 1 extracts feature points of the product's shape based on the product's outline and feature points of the product's pattern caused by lines, bumps, etc., from the portion of the product image IM12. The extracted feature points of the product's shape are set as the first feature information. The feature points of the product's pattern are set as the second feature information. Furthermore, the control device 1 divides the portion of the product image IM12 into multiple regions and extracts color information (for example, numerical information of the R, G, and B components) from each region. The extracted color information is set as the third feature information.

[0093] Referring to Figure 8(c), the control device 1 obtains the position of product BT12 relative to the reference mark MK of plate BT11 in image IM1, based on the portion IM11 of the plate image and the portion IM12 of the product image. The position of product BT12 relative to the reference mark MK in image IM1 can be defined in any way, for example, it may be the direction and distance of the displacement of the product's center CR2 relative to the center CR1 of the three reference marks MK, or it may be the inclination angle of the product which is approximately rectangular with respect to the straight line connecting the two reference marks MK. The information of the product's position relative to the reference mark MK in image IM1 is considered the fourth feature information. Note that if the object to be transported included in the target image does not include a plate, the portion of the plate image does not exist, and therefore the fourth feature information does not need to be extracted.

[0094] In this way, the first to fourth feature information is extracted from image IM1. If image IM1 was captured by any of the cameras 121 to 125 (Figure 1), the control device 1 names the extracted first to fourth feature information using the method described with reference to Figure 5 and adds it to the order information corresponding to the transported item in the order table.

[0095] The shape, pattern, and color of the products will vary from one product to another during the manufacturing process. Furthermore, the position of the product relative to the standard mark on the plate will vary from product to product as it is placed on the plate. The first to fourth characteristic information points focus on these variations.

[0096] Next, we will explain how to identify objects passing through each detection area.

[0097] Referring to Figure 1, the control device 1 captures images of detection areas P31-P37 and P81-P84, respectively, with cameras 131-137 and 181-184 (an example of a second imaging means) on the transport path, and detects the object to be managed from the images captured by these cameras. The detection of the object to be managed may be performed using the learning model 43 or using the feature information extraction method described above. When an object to be managed is detected, the control device 1 sets the transported object detected as the object to be managed as the target object. The control device 1 acquires an image of the target object and extracts the feature information of the target object from the image of the target object using the method described above.

[0098] After extracting characteristic information of the object, the control device 1 refers to the order table and extracts order information where the next passing position is the same as the detection area where the object was detected. For example, if the order table is as shown in Figure 5, and the detection area where the object was detected is detection area P35 (Figure 1), the control device 1 extracts five pieces of order information, each containing the order IDs "00115", "00116", "00117", "00118", and "00119". The control device 1 may also extract all order information from the order table regardless of the next passing position information included in the order information, or it may extract all order information whose status is "in transit".

[0099] Next, the control device 1 selects one order from the extracted order information, compares the feature information of the selected order information with the feature information of the target object, and determines whether the transported object corresponding to the feature information of the selected order information is the same as the target object. For example, the control device 1 may determine whether the first feature information of the selected order information is the same as the first feature information of the target object, whether the second feature information of the selected order information is the same as the second feature information of the target object, whether the third feature information of the selected order information is the same as the third feature information of the target object, and whether the fourth feature information of the selected order information is the same as the fourth feature information of the target object. Based on the four determination results obtained in this way, the control device 1 may determine whether the transported object corresponding to the feature information of the selected order information (more specifically, the transported object shown in the image from which the feature information included in the selected order information was extracted) is the same as the target object. When performing the four determinations described above, each feature information may be quantified before determining whether they are the same or not.

[0100] After obtaining four discrimination results, for example, if a predetermined number of the four discrimination results are identical, it may be determined that the transported object corresponding to the characteristic information of the selected order information and the target object are identical. Alternatively, only some of the first to fourth characteristic information may be extracted from the image, and the determination of whether or not they are identical may be made based on the extracted characteristic information. Further characteristic information different from the first to fourth characteristic information, such as characteristic information indicating the gloss of the product, may be extracted from the image, and the determination of whether or not they are identical may be made based on the extracted characteristic information. In particular, if the product is perishable, the product will dry out over time, and the gloss of the product will change.

[0101] In this way, the control device 1 selects order information one by one from the extracted order information and determines whether the image corresponding to the feature information of the selected order information is the same as the image of the object. If it is determined that they are the same, the control device 1 identifies the object as the transported object corresponding to the selected order information.

[0102] Furthermore, the position of the product on the plate or the shape of the product may change during transport. In particular, if the product is sushi, the toppings placed on top of the rice may fall apart during transport. In addition, foreign objects (typically objects that have fallen from the product being transported, such as salmon roe grains in the case of salmon roe sushi)) may be generated along the transport path and detected as the target object. When such changes occur, the system may determine that none of the images corresponding to the feature information of the extracted order information are identical to the image of the target object, making it impossible to identify the target object.

[0103] Therefore, if the control device 1 determines that none of the images corresponding to the characteristic information of the extracted order information are identical to the image of the object, it may notify the touch panels 151-155 or the like of an error indicating an abnormality in the transported object. This notification may also include information from the camera that photographed the object. This allows each of the craftsmen SF1-SF5 to take necessary actions, such as removing the transported object before it arrives at the destination table.

[0104] Furthermore, if the control device 1 determines that none of the images corresponding to the characteristic information of the extracted order information are identical to the image of the target object, it may identify the target object based on the next detection position and expected passage time of the order information registered in the order table. For example, if the order table is as shown in Figure 5, and the detection area where the target object was detected is detection area P35, the control device 1 may identify the transported object as the target object, which corresponds to the order information with the order ID "00115" whose next passage position is the same as the detection area where the target object was detected and which has the earliest expected passage time.

[0105] Next, a flowchart illustrating the operation of the product dispensing device in this embodiment will be described.

[0106] Figure 9 is a flowchart showing the operation of the control device 1 when it receives an order from a customer via a table touch system, according to one embodiment of the present invention.

[0107] Referring to Figure 9, the control device 1 determines whether or not it has received an order from a customer through one of the table touch systems 171 to 176 in Figure 1 (S101). The control device 1 repeats the process in step S101 until it determines that it has received an order from a customer.

[0108] In step S101, if it is determined that an order has been received from a customer (YES in S101), the control device 1 assigns an order ID to the order, creates new order information including the order ID, delivery destination, product type, order acceptance time, and status (S103), and registers the new order information in the order table (S105). Next, it selects from touch panels 151 to 155 (Figure 1) which touch panel will display the order (the touch panel for the craftsman preparing the product) (S106). Subsequently, the control device 1 updates the screen of the selected touch panel so that the new order information is displayed (S107), and then terminates the process.

[0109] Figure 10 is a flowchart showing the operation of the control device 1 in one embodiment of the present invention when the control device 1 receives a predetermined operation via a touch panel indicating that the placement of the object to be transported onto the supply conveyor has been completed.

[0110] Referring to Figure 10, the control device 1 determines whether or not it has received a predetermined operation indicating that the loading of the object to be transported onto the supply conveyor has been completed, via one of the touch panels 151 to 155 (Figure 1) (S201). The control device 1 repeats the process of step S201 until it determines that it has received a predetermined operation indicating that the loading of the object to be transported onto the supply conveyor has been completed.

[0111] In step S201, if the control device 1 determines that a predetermined operation indicating the completion of placing the object to be transported onto the supply conveyor has been received (YES in S201), the control device 1 updates the screen of the touch panel that received the predetermined operation so that the completion of placing the object to be transported is reflected (S203), and updates the status of the order information corresponding to the object to be transported (S204). Subsequently, the control device 1 starts transporting the object by operating the supply conveyor corresponding to the touch panel that received the predetermined operation among the supply conveyors 102 to 106 (Figure 1) (S205). Subsequently, the control device 1 determines whether or not a target object has been detected based on the image captured by the camera corresponding to the touch panel that received the predetermined operation among the cameras 121 to 125 (Figure 1) (S207). The control device 1 repeats the process in step S207 until it determines that a target object has been detected.

[0112] In step S207, if it is determined that a managed object has been detected (YES in S207), the control device 1 extracts feature information from the image of the transported object by performing a feature extraction process (S900) described later on the image of the transported object. Next, the control device 1 adds the extracted feature information to the order information of the transported object (S208) and updates the status of the order information (S209). Next, the control device 1 sets the path that the transported object should take based on the location where the transported object was detected and the destination information in the order information of the transported object (S210). Next, the control device 1 identifies the next passing location based on the set path (S211) and calculates the estimated passing time (S213). Finally, the control device 1 adds the identified detection area and the calculated estimated passing time to the order information corresponding to the transported object (S215) and terminates the process.

[0113] Figures 11 and 12 are flowcharts showing the operation of the control device 1 in one embodiment of the present invention when a target is detected by any of the cameras 131-137 and 181-184 in Figure 1.

[0114] Referring to Figure 11, the control device 1 determines whether or not a target object has been detected by any of the cameras 131-137 and 181-184 based on the images captured by each of the cameras 131-137 and 181-184 (S301). The control device 1 repeats the process in step S301 until it determines that a target object has been detected by any of the cameras 131-137 and 181-184.

[0115] In step S301, if it is determined that a transported object that is subject to management has been detected (YES in S301), the control device 1 extracts the characteristic information of the object by performing a feature extraction process (S900) described later on the image of the detected object that is subject to management. Next, the control device 1 extracts order information from the order table in which the detection area where the object was detected is the next passing position (S303). Next, the control device 1 selects one order from the extracted order information (S305). The control device 1 compares the characteristic information of the selected order information with the characteristic information of the object (S307) and determines whether the transported object corresponding to the selected order information and the object are the same (S311).

[0116] In step S311, if it is determined that the transported object corresponding to the selected order information is the same as the target object (YES in S311), the control device 1 identifies the transported object corresponding to the selected order information as the target object (S313) and proceeds to the process in step S321 in Figure 12.

[0117] In step S311, if it is determined that the object to be transported and the target object corresponding to the selected order information are not the same (NO in S311), the control device 1 determines whether all of the extracted order information has been selected or not (S315). In step S315, if it is determined that none of the extracted order information has been selected (NO in S315), the control device 1 proceeds to the process in step S305.

[0118] If, in step S315, it is determined that all extracted order information has been selected (YES in S315), the control device 1 determines whether or not the target item can be identified based on the order information registered in the order table (S317).

[0119] If it is determined in step S317 that the object can be identified (YES in S317), the control device 1 identifies the object (S313) and proceeds to the process in step S321 in Figure 12.

[0120] If, in step S317, it is determined that the object cannot be identified (NO in S317), the control device 1 notifies an error (S319) and terminates the process.

[0121] Referring to Figure 12, in step S321, the control device 1 refers to the order information of the identified object (S321) and sets the path that the object should take (S325). The path may be set based on the position of the detection area of ​​the camera that captured the image of the object, the destination, and whether or not the loading space on the loading path at the destination is full (detection result of a sensor (not shown) that detects the loading space). Next, the control device 1 determines whether or not the camera that captured the image of the object has an extruder (S327). Specifically, if the camera that captured the image of the object is one of the cameras 131 to 137 (Figure 1), it is determined that the camera has an extruder. On the other hand, if the camera that captured the image of the object is one of the cameras 181 to 184 (Figure 1), it is determined that the camera does not have an extruder.

[0122] In step S327, if the camera that captured the image of the object has an extruder (YES in S327), the control device 1 determines whether or not it is necessary to have the extruder perform an operation based on the set path (S329).

[0123] If, in step S329, it is determined that it is necessary to operate the extruder (YES in S329), the control device 1 causes the extruder to perform the extrusion operation (S331) and proceeds to the process in step S333.

[0124] If, in step S327, it is determined that the camera that captured the image of the object does not have a corresponding extruder (NO in S327), or if, in step S329, it is determined that it is not necessary to operate the extruder (NO in S329), the control device 1 proceeds to the process in step S333.

[0125] In step S333, the control device 1 identifies the next passing position of the object based on the set path and calculates the predicted passing time for passing the identified next passing position (S333). Subsequently, the control device 1 updates the next passing position and predicted passing time in the order information of the object (S335) and terminates the process.

[0126] Figure 13 shows the subroutine for the feature extraction process (S900) in Figures 10 and 11.

[0127] Referring to Figure 13, in the feature extraction process of step S900, the control device 1 divides the target image into a plate image portion and a product image portion by performing edge processing on the target image (S901). Next, the control device 1 extracts feature points of the product shape from the product image portion (S903) and extracts feature points of the product pattern from the product image portion (S905). Subsequently, the control device 1 divides the product image portion into multiple regions and extracts color information (product color information) from each region (S907). Next, the control device 1 obtains the position of the product relative to a reference mark in the target image (S909). Subsequently, the control device 1 creates feature information that includes the feature points of the product shape, the feature points of the product pattern, the product color information, and the position of the product relative to the reference mark as the first to fourth feature information (S911), and returns it.

[0128] Figure 14 is a schematic diagram showing the respective neural networks 500 of learning models 43 and 45 (Figure 2) in one embodiment of the present invention.

[0129] Referring to Figure 14, each of the neural networks 500 in learning models 43 and 45 (Figure 2) is a so-called hierarchical neural network, in which a large number of artificial neurons (shown as circles in Figure 14) are connected in a hierarchical structure. A hierarchical neural network comprises artificial neurons for input, artificial neurons for processing, and artificial neurons for output.

[0130] Problem data 510 is the target of processing by the neural network 500. Problem data 510 is acquired by artificial neurons for input in the input layer 501. The input layer 501 is composed of artificial neurons arranged in parallel. Problem data 510 is distributed to the artificial neurons for processing.

[0131] Processing artificial neurons are connected to input artificial neurons. The processing artificial neurons are arranged in parallel to form a hidden layer 502. The hidden layer 502 may consist of multiple layers. A neural network with three or more layers and a hidden layer 502 is called a deep neural network.

[0132] The neural network may be a so-called convolutional neural network. A convolutional neural network is a deep neural network constructed by alternately connecting convolutional layers and pooling layers.

[0133] The output artificial neurons output the training data 511 to the outside. The output artificial neurons constitute the output layer 503. The neural network 500 is trained so that when the problem data 510 is input, the training data 511 is output (specifically, the parameters of the training models 43 and 45 (Figure 2) are adjusted).

[0134] The training data used to create the learning model 43 may include multiple sets of sample images and the result of determining whether or not the sample image contains the object to be detected. In this case, the sample image becomes the problem data 510, and the result of determining whether or not the sample image contains the object to be detected becomes the training data 511. Through machine learning using this training data, the learning model 43 will output a result of determining whether or not the object to be detected contains the object to be detected when an image taken by cameras 121-125, 131-137, and 181-184 in Figure 1 is input.

[0135] The training data used to create the learning model 45 may include multiple sets of pairs of two sample images and the result of determining whether the transported object depicted in those two sample images is the same or not. In this case, the two sample images become the problem data 510, and the result of determining whether the transported object depicted in those two sample images is the same or not becomes the training data 511. Through machine learning using this training data, the learning model 45 will output a result of determining whether the transported object depicted in a specific image and the target object are the same or not when given a specific image and an image of the target object as input.

[0136] (Effects of the embodiment)

[0137] In the above-described embodiment, the object is identified as one of the multiple transported objects based on images of each of the multiple transported objects passing through the first region on the transport path and an image of the object that has passed through the second region on the transport path. According to the above-described embodiment, there is no need to record an ID on the RF tag to identify the object, and the device configuration can be simplified.

[0138] In particular, when the goods being transported are sushi, there are times when multiple transported items, each carrying the same type of sushi topping, are transported along the same transport route at the same time. Despite these transported items having different appearances, they need to be transported to different destinations. According to the embodiment described above, even in such cases, each of the multiple transported items, each carrying the same type of sushi topping, can be distinguished based on images and transported to the required destination.

[0139] (others)

[0140] The characteristic information included in the order information may be extracted from images of the transported object as it passes through any of the detection areas P31-P37 and P81-P84 in Figure 1. The control device 1 may update the characteristic information of the order information corresponding to the object with the characteristic information extracted from the image of the object each time it identifies an object in any of the detection areas P31-P37 and P81-P84.

[0141] The identification of the target object may be performed by the extruder corresponding to the camera instead of the control device 1. In this case, the extruder may extract order information from the order table stored by the control device 1, or each extruder may store an order table that is synchronized at a predetermined timing.

[0142] The transport paths RT1-RT6, RT21-23, RT31-RT36, and RT41-RT44 of the product dispensing device in Figure 1 may be divided into multiple detection areas, and each of these detection areas may be equipped with multiple imaging means (cameras) to photograph the transported object as it passes through. This allows for the management of the state of the transported object throughout the entire transport paths RT1-RT6, RT21-23, RT31-RT36, and RT41-RT44. As a result, if an object on the transport path is removed by a craftsman or other person due to a product defect or other reason, this removal can be immediately reflected in the order information, enabling more advanced transport control.

[0143] The identification device of this embodiment may be used in devices other than product dispensing devices. For example, it may be used in an inspection device that transports intermediate or finished products in a factory and inspects the transported products in transit, as a device for identifying the transported products in transit.

[0144] The processing in the above-described embodiments and modifications may be performed by software or by hardware circuits. Furthermore, a program for performing the processing in the above-described embodiments may be provided, or the program may be recorded on a recording medium such as a CD-ROM, flexible disk, hard disk, ROM, RAM, or memory card and provided to the user. The program is executed by a computer such as a CPU. Alternatively, the program may be downloaded to the device via a communication line such as the Internet.

[0145] The embodiments and variations described above can be combined as appropriate.

[0146] The embodiments and modifications described above should be considered in all respects as illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of symbols]

[0147] 1. Control device 11 Camera Control Unit 13 Order Reception Department 15. Order Information Management Department 17 Detection unit 18 Identity Output Unit (An Example of Identity Output Means) 19 Selection Section 21. Segmentation section (an example of a segmentation means) 23 Position acquisition unit (an example of a position acquisition means) 25 Feature Point Extraction Unit (An Example of Feature Point Extraction Means) 27. Discrimination Unit (An Example of a Discrimination Unit) 29. Specific section (an example of a specific means) 31 Extrusion Control Unit 33 Route setting section 35 Prediction Unit (An Example of Prediction Means) 37 Operation display section 38 Communications Department 39 Memory section 41 Order Table 43,45 Learning Models 101 Circulating conveyor 102-106 Supply conveyor 107-109 Customer conveyor 110-113 Rollers 121-125, 131-137, 181-184 Camera (Examples of first and second shooting means) 141-147 Extrusion apparatus (Examples of the first and second extrusion apparatuses) 151-155 Touch panel 161-163 Switching lever 171-176 Table Touch System 500 Neural Networks 501 Input Layer 502 Middle Class 503 Output Layer 510 Problem Data 511 Training Data BT1 Transported object (Example of transported object and target object) BT11 Plate (Example of mounting section) BT12 Products (Examples of Products) CR1,CR2 center Image from which IM1 feature information will be extracted. IM11 Plate image section IM12 Product image section MK Standard Mark (Example of a standard mark) P21-P25, P31-P37, P81-P84 Detection area (Examples of the first and second areas) Transport routes for RT1-RT6, RT21-RT23, RT31-RT36, and RT41-RT44 (example of a transport route) RT11, RT12 Straight Lane SF1~SF5 Craftsmen TB1~TB6 Table TE configured route

Claims

1. A device for identifying objects to be transported along a transport route, The transported object includes goods, A first imaging means for sequentially photographing each of the multiple objects being transported as they pass through a first region on the transport path, After capturing images of each of the plurality of objects to be transported, a second imaging means captures an object that has passed through a second region on the transport path, A means for identifying which of the plurality of transported objects the object in question is, A feature point extraction means for extracting feature points from the portion of the product image in each of the images of a specific object and the image of the target object, among the images of the plurality of transported objects, The system includes a determination means for determining whether the object being transported in the specific image and the object depicted in the image of the object are the same, based on feature points extracted by the feature point extraction means from the portion of the product image in each of the specific image and the image of the object, respectively. If the determination means determines that the object being transported in the specific image and the object depicted in the image of the object are the same, the determination means identifies the object as the object being transported in the specific image. The identification device further comprises a notification means for notifying an error indicating an abnormality of a transported object when the determination means determines that none of the plurality of transported objects are identical to the object captured in the image of the object.

2. The identification device according to claim 1, wherein the notification means also notifies information of the second imaging means that has photographed the object.

Images