Specific devices

Abstract

A specific device capable of simplifying the device configuration is provided. [Solution] The product supply device is an identification device that identifies transported objects transported along a transport route. It includes cameras 121-125 that sequentially capture images of multiple transported objects passing through detection areas P21-P25 on the transport route, cameras 131-137 and 181-184 that capture images of objects after capturing images of the multiple transported objects and then capture images of objects that have passed through detection areas P31-P37 and P81-P84 on the transport route, and an identification unit that identifies which of the multiple transported objects the object is based on each of the images of the multiple transported objects and an image of the object.

Description

[Technical Field]

[0001] The present invention relates to an identification device, and more particularly to an identification device that identifies an object being conveyed along a conveyance path. [Background technology]

[0002] Conventionally, in conveyor belt sushi restaurants and the like, products placed on plates are transported along a transport route to customer tables. In such restaurants, multiple products are transported in parallel to the required tables, so it is necessary to identify the products being transported. Technology for identifying products being transported is disclosed, for example, in Patent Document 1 listed below.

[0003] The following Patent Document 1 discloses a device that identifies products being conveyed by attaching an RF tag, on which an ID for identifying the product is recorded, to a tray and reading the RF tag. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-107301 Summary of the Invention [Problem to be solved by the invention]

[0005] The device of Patent Document 1 requires a configuration for recording an ID on an RF tag attached to a tray. This results in a complicated device configuration. The problem of complicated device configuration is a problem that occurs in all identification devices that identify transported objects transported along a transport path.

[0006] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a specific device that can simplify the device configuration. [Means for solving the problem]

[0007] An identification device according to one aspect of the present invention is an identification device for identifying a transported object transported along a transport route, and includes a first imaging means for sequentially photographing each of a plurality of transported objects passing through a first area on the transport route, a second imaging means for photographing an object that has passed through a second area on the transport route after photographing each of the images of the plurality of transported objects, and an identification means for identifying which of the plurality of transported objects the object is based on each of the images of the plurality of transported objects and the image of the object. The transported object includes a loading section and a product placed on the loading section, and further includes a feature extraction means for extracting feature points indicating the shape, pattern, and gloss of the product from the product image portion in each of the specific image and the image of the object among the multiple images of the transported object, and a discrimination means for discriminating whether the transported object in the specific image and the object are the same based on the feature points extracted by the feature extraction means from the product image portion in each of the specific image and the image of the object, and when the discrimination means determines that the transported object in the specific image and the object are the same, the identification means identifies the object as the transported object in the specific image.

[0009] In the above-mentioned specific device, preferably, The mounting portion includes a reference mark, a dividing means for dividing each of the specific image and the image of the object into a portion of the image of the placement section and a portion of the image of the product; and a position acquiring means for acquiring the position of the product relative to the reference mark in each of the specific image and the image of the object. The determination means further comprises: The location of the item relative to the fiducial marks in each of the particular images and the image of the object. Furthermore Based on the image, it is determined whether the transported object in the specific image is the same as the target object. do.

[0010] In the above-mentioned specific device, preferably, a route setting means for setting a route through which each of the plurality of transported objects should pass based on the first area and the destination of each of the plurality of transported objects; and a route setting means for setting a route through which each of the plurality of transported objects should pass based on the route set by the route setting means. and a prediction means for predicting the timing of passing through the If the discrimination means determines that none of the images of the multiple transported objects is identical to the image of the target object, the identification means identifies which of the multiple transported objects the target object is based on the timing at which each of the multiple transported objects passes through the position of the second area.

[0011] Preferably, the above-mentioned identification device is provided with a plurality of photographing means including first and second photographing means, and when the entire conveying route is divided into a plurality of areas including the first and second areas, each of the plurality of photographing means photographs the conveyed object as it passes through each of the plurality of areas.

[0012] In the above-mentioned identification device, preferably, the conveying route includes a circular conveying route and a branch route that branches off from a predetermined position on the circular conveying route and heads toward the customer who has ordered the conveyed item, and the device further includes a first extrusion device that, after identifying which of the multiple conveyed items the object is, pushes the object from a predetermined position on the circular conveying route into the branch route.

[0013] In the above-mentioned identification device, preferably, the conveying path further includes a shortcut path that takes a shortcut from a branching position on the circular conveying path to a merging position on the circular conveying path, and further includes a second extrusion device that, after identifying which of the multiple conveyed objects the object is, pushes the object from the circular conveying path to the shortcut path at the branching position. [Effects of the Invention]

[0014] According to the present invention, it is possible to provide a specific device that can simplify the device configuration. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a plan view showing the configuration of a store that employs a product providing device according to an embodiment of the present invention. [Figure 2] 2 is a block diagram showing the functional configuration of a control device 1. FIG. [Figure 3] FIG. 10 is a diagram schematically illustrating the order table immediately after order information based on a new order is registered. [Figure 4] 10 is a diagram showing a schematic view of the order table immediately after a predetermined operation is received on touch panel 151 and order information is updated. FIG. [Figure 5] FIG. 10 is a diagram schematically illustrating the order table immediately after the next passing position and the estimated passing time are added to the order information. [Figure 6] 10 is a diagram showing a path along which an object passes when it is transported from a supply conveyor 102 to a table TB5. FIG. [Figure 7] 10 is a diagram showing a path along which an object to be transported to table TB4 passes when the placement space of transport route RT34 is fully loaded. FIG. [Figure 8] 10A and 10B are diagrams illustrating an example of a method for extracting feature information from a transported object. [Figure 9]10 is a flowchart showing the operation of the control device 1 when the control device 1 receives an order from a customer through the table touch system in one embodiment of the present invention. [Figure 10] 10 is a flowchart showing the operation of the control device 1 in the embodiment of the present invention when the control device 1 receives a predetermined operation via the touch panel indicating that the placement of the transported objects on the supply conveyor has been completed. [Figure 11] 10 is a first flowchart showing the operation of the control device 1 when a managed object is detected by any of the cameras 131 to 137 and 181 to 184 in the embodiment of the present invention. [Figure 12] 10 is a second flowchart showing the operation of the control device 1 when a managed object is detected by any of the cameras 131 to 137 and 181 to 184 in the embodiment of the present invention. [Figure 13] This is a subroutine of the feature extraction process (S900) in FIGS. [Figure 14] 5 is a diagram schematically illustrating a neural network 500 of each of learning models 43 and 45 (FIG. 2) in one embodiment of the present invention. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0016] FIG. 1 is a plan view showing the configuration of a store that employs a product providing device according to an embodiment of the present invention.

[0017] Referring to Fig. 1, a product provision device (an example of a specific device) in this embodiment is installed in a store, such as a conveyor belt sushi restaurant, that has a conveyor means for conveying products placed on plates along a conveyor path and circulating them within the eating and drinking area. The types of products are arbitrary, and are typically food and drink such as sushi, snacks, or beverages. Within the store, a plurality of tables TB1-TB6 for customers and seats are provided along the vicinity of the product conveyor path. Each of the tables TB1-TB6 and seats is for a group of multiple customers.

[0018] The product providing device in this embodiment is a device that transports and provides products to each customer using a transport route different from the above-mentioned "transport means that transports products placed on plates along a transport route and circulates them within the eating and drinking area." The product providing device in this embodiment does not circulate products within the eating and drinking area, but rather delivers transported objects including products corresponding to orders received from customers directly from the kitchen area to the customer's table in the eating and drinking area. The product providing device identifies the transported object being transported along the transport route and transports the transported object to the table of the customer who needs it.

[0019] The lower side of Fig. 1 is the kitchen area where chefs prepare the products, and the upper side of Fig. 1 is the eating area where customers eat and drink. The product providing device in this embodiment includes conveying routes RT1-RT6, RT21-23, RT31-RT36, and RT41-RT44 (examples of conveying routes), a control device 1, a circulating conveyor 101, supply conveyors 102-106, customer conveyors 107-109, rollers 110-113, cameras 121-125, 131-137, and 181-184, push-out devices 141-147, touch panels 151-155, switching levers 161-163, and table touch systems (order input devices) 171-176. The number and installation positions of the conveying routes, control devices, conveyors, rollers, cameras, push-out devices, touch panels, table touch systems, switching levers, and table touch systems are arbitrary.

[0020] Each of the transport routes RT1 to RT6, RT21 to RT23, RT31 to RT36, and RT41 to RT44 is a route for transporting transported objects. The transported objects include plates (an example of a placement unit) and products placed on the plates. It is preferable that reference marks are provided on the plates. The products may be placed on something other than plates, and may not be placed on a placement unit.

[0021] The transport route RT1 is circular and is provided in the kitchen area. The transport route RT1 includes a straight lane RT11 that extends straight on the craftsman side and a straight lane RT12 that extends straight on the customer side.

[0022] Each of the transport routes RT2 to RT6 is provided in the kitchen area. Each of the transport routes RT2 to RT6 extends horizontally in Fig. 1 and merges with the transport route RT1 at its right end in Fig. 1. Between the right end in Fig. 1 of each of the transport routes RT2 to RT6 and the transport route RT1, a transport means (not shown) such as a roller is provided to supply the transported objects transported along each of the transport routes RT2 to RT6 onto the transport route RT1.

[0023] Each of the transport routes RT21 to RT23 (an example of a branch route) is linear, branches off from a predetermined position on the transport route RT1, and heads toward each of the tables TB1 to TB6 in the seating area.

[0024] Each of the transport routes RT31 to RT36 is a location where customers seated at each of the tables TB1 to TB6 can receive their products. Transported objects, including products ordered by customers seated at each of the tables TB1 to TB6, arrive at the transport route RT31 to RT36 that corresponds to that table. Each of the transport routes RT31 to RT36 branches off from one of the transport routes RT21 to RT23 and extends to the vicinity of one of the tables TB1 to TB6.

[0025] Each of the transport routes RT41 to RT44 (an example of a shortcut route) takes a shortcut along the transport route RT1 from a predetermined branching position on the transport route RT1 to a predetermined joining position.

[0026] The control device 1 controls the entire product provision device. The control device 1 is made up of a computer such as a PC (Personal Computer) or smartphone, and includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), an operation unit, a display unit, a network interface, etc. The CPU executes the control program stored in the ROM. The ROM stores various programs executed by the CPU and various fixed data. The RAM is used to temporarily store data required when the CPU executes the control program. The operation unit consists of a keyboard and mouse, etc., and accepts various inputs. The display unit displays various information. The network interface communicates with other devices using communication protocols such as TCP / IP.

[0027] The circulating conveyor 101 transports the objects along the transport path RT1. The supply conveyors 102-106 transport the objects along the transport paths RT2-RT6, respectively, and supply products to the circulating conveyor 101. The customer conveyors 107-109 transport the objects along the transport paths RT21-RT23, respectively. The rollers 110-113 transport the objects along the transport paths RT41-RT44, respectively. The circulating conveyor 101, the customer conveyors 107-109, and the rollers 110-113 are all driven at all times, and the supply conveyors 102-106 are driven at the required timing under the control of the control device 1.

[0028] Each of cameras 121-125, 131-137, and 181-184 has a detection area P21-P25, P31-P37, and P81-P84 (areas surrounded by dotted lines in FIG. 1 ) on one of conveyance routes RT1-RT6, RT21-RT23, and RT31-RT36. Each of cameras 121-125, 131-137, and 181-184 sequentially captures an image of each of the multiple conveyed objects passing through the corresponding detection area. In particular, each of cameras 131-137 and 181-184 captures an image of the target conveyed object that has passed through the corresponding detection area after one of cameras 121-125 captures an image of each of the multiple conveyed objects.

[0029] Each of the pushers 141 to 147 pushes the object being transported along the transport path RT1 to another necessary transport path under the control of the control device 1. Each of the pushers 141 to 147 may be implemented by a computer.

[0030] Each of the touch panels 151 to 155 displays various information to each of the craftsmen SF1 to SF5 and accepts various operations from each of the craftsmen SF1 to SF5 under the control of the control device 1. Each of the touch panels 151 to 155 may be made up of a computer.

[0031] Each of the switching levers 161 to 163 switches the destination of the transported object transported along each of the transport routes RT21 to RT23.

[0032] Each of the table touch systems 171 to 176 is provided at each of the tables TB1 to TB6. Each of the table touch systems 171 to 176 is equipped with a display and input device such as a touch panel display. Under the control of the control device 1, each of the table touch systems 171 to 176 displays various information such as menus to customers at that table and accepts various operations such as orders from customers at that table. Each of the table touch systems 171 to 176 may be composed of a computer.

[0033] When each of the table touch systems 171 to 176 receives an order for a product from a customer, it transmits the details of the received order (type and quantity of product) to the control device 1. The product corresponding to the order input to each of the table touch systems 171 to 176 is delivered directly from the kitchen to the table of the customer who ordered it by the product provision device in this embodiment.

[0034] At the time of payment, the control device 1 or each of the table touch systems 171-176 transmits payment information (type and quantity of products provided / consumed) to the self-checkout register (not shown) based on the order information accumulated for the customer group corresponding to that table. The self-checkout register calculates the amount based on the transmitted information, presents it to the customer, receives the payment from the customer, returns the change, etc.

[0035] The control device 1 stores an order table. When the control device 1 receives the details of an order from any of the table touch systems 171 to 176, it assigns an order ID to the details of the order and registers them in the order table as new order information.

[0036] It should be noted that the table touch system that has accepted the order may itself create order information, and when the control device 1 receives the order information from the table touch system, the control device 1 may register the order information in the order table.

[0037] FIG. 2 is a block diagram showing the functional configuration of the control device 1. As shown in FIG.

[0038] 2, the functions of the control device 1 are realized by the CPU executing various programs stored in the ROM. The control device 1 includes a camera control unit 11, an order receiving unit 13, an order information management unit 15, a detection unit 17, an identity output unit 18 (an example of identity output means), a selection unit 19, a classification unit 21 (an example of classification means), a position acquisition unit 23 (an example of position acquisition means), a feature point extraction unit 25 (an example of feature point extraction means), a discrimination unit 27 (an example of discrimination unit), an identification unit 29 (an example of identification means), an extrusion control unit 31, a path setting unit 33, a prediction unit 35 (an example of prediction means), an operation display unit 37, a communication unit 38, and a storage unit 39.

[0039] Camera control unit 11 controls the shooting operation of each of cameras 121 to 125, 131 to 137, and 181 to 184 in Fig. 1. The images shot by each of cameras 121 to 125, 131 to 137, and 181 to 184 may be still images or moving images.

[0040] The order receiving unit 13 receives the order details from each of the table touch systems 171 to 176, creates order information by adding an order ID to the order details, and registers the order information in the order table.

[0041] The order information management unit 15 updates the order information registered in the order table at the necessary timing.

[0042] The detection unit 17 detects the managed object in each of the detection areas P21 to P25, P31 to P37, and P81 to P84 based on the images captured by the cameras 121 to 125, 131 to 137, and 181 to 184 in Fig. 1. The detection unit 17 may detect the managed object using a learning model 43.

[0043] The managed object is a transported object that is the target of the feature extraction process described below. The type of managed object is arbitrary. For example, the managed object may include a plate and the product placed on the plate, as well as accessories such as spoons and foreign objects (such as pieces of a product (such as French fries) that have fallen from the plate). Furthermore, if the product serving device is installed in a conveyor-belt sushi restaurant, the managed object may include a transported object including a circular serving container, or a container with a circular planar shape for holding a bowl of soup such as udon or miso soup. Furthermore, the managed object may include plates for placing sushi, beverages, chawanmushi, desserts, and the like. Conveyors and rollers that transport the transported object, as well as transported objects with plates displaying promotional images placed on them, may be excluded from the managed object.

[0044] The identity output unit 18 uses the learning model 45 to input one image (sometimes referred to as a specific image) selected by the selection unit 19 from among multiple images of the transported object and an image of the transported object (sometimes referred to as the target object) taken by one of the cameras 131 to 137 and 181 to 184 in Figure 1, and outputs a determination result as to whether the transported object shown in the specific image and the target object (the object shown in the image of the target object) are the same.

[0045] The selection unit 19 selects one image from among the images of the plurality of transported objects taken by the cameras 121 to 125 (FIG. 1).

[0046] The dividing unit 21 divides each of the specific images and the target object images among the images of the plurality of transported objects into a portion of a plate image and a portion of a product image.

[0047] The position acquisition unit 23 acquires the position of the product relative to the reference mark based on the portion of the image of the dish and the portion of the image of the product in each of the specific image and the image of the object.

[0048] The feature point extraction unit 25 extracts feature points from the product image portion in each of the specific image and the image of the object.

[0049] The discrimination unit 27 determines whether the transported object shown in the specific image and the object (the object shown in the image of the object) are the same based on the position of the product relative to the reference mark in each of the specific image and the image of the object, and feature points extracted from the image of the product in each of the specific image and the image of the object.

[0050] Identification unit 29 identifies which of the plurality of transported objects the target object is based on each of the images of the plurality of transported objects taken by cameras 121 to 125 (FIG. 1) and the image of the target object.

[0051] The identification unit 29 may identify the transported object based on the determination result by the identity output unit 18, or may identify the transported object based on the determination result by the discrimination unit 27. Specifically, when the identity output unit 18 determines that the transported object in the specific image and the target object are the same, the identification unit 29 may identify the target object as the transported object shown in the specific image. Also, when the discrimination unit 27 determines that the transported object shown in the specific image and the target object are the same, the identification unit 29 may identify the target object as the transported object shown in the specific image.

[0052] The extrusion control unit 31 controls the operations of the extrusion devices 141 to 147 and the switching levers 161 to 163 shown in FIG.

[0053] The path setting unit 33 sets a path along which the object should pass at the required timing.

[0054] Based on the route set by the route setting unit 33, the prediction unit 35 predicts the expected passing time (an example of the passing timing) at which the object will pass through the required positions (in this embodiment, the detection area of the camera through which the object will next pass) on the conveying routes RT1 to RT6, RT21 to RT23, RT31 to RT36, and RT41 to RT44 in Figure 1.

[0055] The operation display unit 37 accepts various operations and performs various displays.

[0056] The communication unit 38 communicates with the circulating conveyor 101, supply conveyors 102-106, customer conveyors 107-109, rollers 110-113, cameras 121-125, 131-137, and 181-184 in Figure 1, as well as the extrusion devices 141-147, touch panels 151-155, switching levers 161-163, and table touch systems 171-176.

[0057] The memory unit 39 stores various information such as an order table 41, a learning model 43, and a learning model 45.

[0058] Learning model 43 is a model generated by machine learning. Learning model 43 receives images captured by cameras 121 to 125, cameras 131 to 137, and 181 to 184 as input, and outputs a determination result as to whether or not a transported object is included in the image.

[0059] The learning model 45 is a model generated by machine learning. The learning model 45 receives a specific image and an image of an object as input, and outputs a determination result as to whether the transported object shown in the specific image is the same as the object.

[0060] FIG. 3 is a diagram schematically showing the order table immediately after order information based on a new order is registered.

[0061] 1 and 3, the order table 41 (FIG. 2) registers multiple pieces of order information in the order received order order. The order information includes the following fields: "Order ID," "Delivery Destination," "Product Type," "Order Acceptance Time," "Status," "Characteristic Information," "Next Passage Location," and "Expected Passage Time." The "Order ID" is information identifying the order. The "Delivery Destination" is information identifying the table corresponding to the table touch system that sent the order (the table to which the product will be delivered). The "Product Type" is the type of product related to the order. The "Order Acceptance Time" is the time when the table touch system accepted the order. The "Status" is the status of the ordered product. One of the following states is entered in the "Status" field: "Preparing," "Feature Extracting," "Transporting," or "Transport Completed." The "Characteristic Information" is information related to the characteristics of the transported object, including the product being transported. The "Next Passage Location" is information indicating the detection area through which the transported object, including the product being transported, will next pass. The "Expected Passage Time" is the time when the transported object, including the product being transported, is expected to pass the next pass location.

[0062] Here, it is assumed that the control device 1 (Fig. 2) receives a new order for "scallop sushi" from a customer in table TB5 (Fig. 1) via the table touch system 175. When this order is received, the control device 1 associates information specifying the delivery destination of this order (table TB5 of the customer who placed the order), the "type of product" (type of product related to the order), and the "time of order acceptance" (time of acceptance), and creates new order information with an arbitrary order ID of "00119" in Fig. 3. The control device 1 registers the created order information at the bottom of the order table. At this time, the "status" field is filled in with "preparing," indicating that the product is being prepared for delivery. Note that the "characteristic information," "next passing position," and "expected passing information" fields in the order table are left blank at this point.

[0063] 1 illustrates an example of a situation in which five chefs SF1 to SF5 are cooking in a kitchen area. In front of each of the chefs SF1 to SF5 (upward in FIG. 1) are provided supply conveyors 102 to 106 and touch panels 151 to 155 for the chefs. Here, five supply conveyors 102 to 106 and five touch panels 151 to 155 are provided in front of each of the five chefs SF1 to SF5.

[0064] The circulating conveyor 101 circulates the objects along a circular transport path RT1 in the kitchen. The rotation direction of the circulating conveyor 101 is counterclockwise as indicated by the black arrow within the transport path RT1. The rotation direction of the circulating conveyor 101 is optional and may be clockwise.

[0065] Regardless of which of the chefs SF1 to SF5 is in charge of preparing the products, the operation of the chef and the product supply device is the same. Here, it is assumed that the chef SF5 is in charge of preparing the products. When new order information is registered in the order table, the control device 1 displays the order information on the chef's touch panel 155 installed in front of the chef SF5 in charge, and gives instructions for cooking the products (such as the type of product and the ID of the table to which the product is to be delivered). Upon seeing this, the chef SF5 prepares the product (cooks the product if the product is sushi), places the prepared product on a plate, and places the product on the plate on the supply conveyor 106. Hereinafter, the object including the plate and the product placed on the plate may be referred to as the transported object.

[0066] After placing the transported item corresponding to the order on the supply conveyor 106, the craftsman SF5 performs a predetermined operation on the touch panel 155 to indicate that the transported item has been placed (for example, by touching the area displaying the order corresponding to that item). Upon receiving this operation, the control device 1 operates the supply conveyor 106. As a result, the transported item placed on the supply conveyor 106 is supplied to the straight lane RT11 via the transport route RT6, as indicated by the black arrow. Furthermore, upon receiving this operation, the control device 1 updates the order information corresponding to the new order in the order table.

[0067] FIG. 4 is a diagram schematically showing order table 41 (FIG. 2) immediately after a predetermined operation is accepted on touch panel 155 (FIG. 1) to update the order information.

[0068] 1 and 4, when craftsman SF5 performs a predetermined operation on touch panel 151 indicating that the placement of the transported object corresponding to the order information including the order ID "00119" has been completed, control device 1 updates the "status" field of the order information to "feature extraction in progress." "Feature extraction in progress" indicates the state in which feature information is being extracted from the image of the transported object.

[0069] Referring to FIG. 1, each of cameras 121 to 125 (an example of a first photographing means) sequentially photographs each of the multiple transported objects passing through the corresponding detection area. Each of detection areas P21 to P25 (an example of a first area) may be provided on the circular transport route RT1, or may be provided on each of the transport routes RT2 to RT6. The control device 1 extracts characteristic information from the captured image of each of the multiple transported objects using an extraction method described below. The control device 1 adds the extracted characteristic information to the order information corresponding to the transported object. As a result, the order information in the order table further includes characteristic information of the transported object corresponding to the order information. The characteristic information of the transported object is used to identify the transported object during transport.

[0070] In the case of a transported object corresponding to order information including the order ID "00119", the control device 1 extracts characteristic information of the transported object from an image of the transported object taken by the camera 125 while the transported object is passing through the detection area P25. Then, the control device 1 adds the extracted characteristic information to the order information of the product of the transported object (FIG. 5).

[0071] After adding the extracted feature information to the order information, the control device 1 sets a route TE through which the transported object should pass, based on the detection area P25 (FIG. 1) of the camera 125 that captured the image of the transported object and the location of the table TB5, which is the destination of the transported object and is included in the order information. Based on the set route TE, the control device 1 then identifies the next pass position of the transported object and predicts the expected passing time. The next pass position is, for example, one of the detection areas P31-P37 and P81-P84 (an example of a second area) of the cameras 131-137 and 181-184. In this example, the detection area P35, adjacent to the detection area P25 on the downstream side along the route TE, is identified as the next pass position. The expected passing time may be calculated based on the current time, the distance from the current detection position to the next detection position along the route TE, the transport speed of the transported object, and the like.

[0072] In addition, for the purpose of repairing the product, a transported object being transported along the transport route RT1 may be removed by a craftsman and returned to the transport route. Also, a transported object may be pushed by another transported object. When such an event occurs, the time at which the transported object actually passes the next passing position will deviate from the predicted passing time.

[0073] FIG. 5 is a diagram schematically showing the order table immediately after the next passing position and the estimated passing time are added to the order information.

[0074] 1 and 5, when the control device 1 extracts the characteristic information of the transported object, it updates the "status" column of the order information including the order ID "00119" to "in transport." "In transport" indicates a state in which the transported object is being transported along one of the transport routes RT1 to RT6, RT21 to RT23, and RT41 to RT44. The control device 1 also adds four pieces of characteristic information, "00119a," "00119b," "00119c," and "00119d," to the "characteristic information" column of the order information including the order ID "00119" in the order table. When the control device 1 identifies the next passing position and predicts the expected passing time, it adds information "P35" that identifies the detection area P35 to the next passing position column and adds the expected passing time of "2024 / 06 / 21 12:55:40."

[0075] In this embodiment, to distinguish between the four pieces of characteristic information in each piece of order information, the name of the characteristic information is determined by adding the symbol "a," "b," "c," or "d" to the end of the order ID. Feature information with the name "a" added to the end of the order ID (for order information including the order ID "00119," the characteristic information named "00119a") is referred to as the first characteristic information. Feature information with the name "b" added to the end of the order ID (for order information including the order ID "00119," the characteristic information named "00119b") is referred to as the second characteristic information. Feature information with the name "c" added to the end of the order ID (for order information including the order ID "00119," the characteristic information named "00119c") is referred to as the third characteristic information. Feature information with a name that has the symbol "d" added to the end of the order ID (for order information that includes the order ID "00119", the feature information is "00119d") may be referred to as the fourth feature information.

[0076] Referring to FIG. 1, ordered products are delivered to the table of the customer who placed the order based on order information. Cameras 131-137 are provided at necessary positions on the circular conveyance route RT1 to capture images of the conveyed objects during conveyance. In this embodiment, each of the cameras 131-137 corresponds to a corresponding one of the pushers 141-147. Each of the cameras 131-137 captures an image of the conveyed object passing through each of the detection areas P31-P37. The control device 1 extracts characteristic information from the image of the object captured by each of the cameras 131-137 and identifies which conveyed object the object is based on the extracted characteristic information. The control device 1 then sets a route for the conveyed object to pass based on destination information included in the order information corresponding to the identified conveyed object. The set route is generally the same as the previously set route, but may differ depending on the situation. The control device 1 identifies the next passing position and calculates the expected passing time based on the set route. The control device 1 updates the next passing position and the expected passing time in the order information corresponding to the identified transported object. Furthermore, the control device 1 controls the operation of each of the pushers 141-147 corresponding to the camera that took the photograph based on the set route. This allows the transported object including the product to be correctly transported to the table (destination) of the customer who ordered the product.

[0077] Each of the transport routes RT21 to RT23 branches off from a predetermined position on the customer-side straight lane RT12. The customer conveyor 107 transports items along the transport route RT21 to table TB1 or TB2, and by operating the downstream switching lever 161, transports products from the transport route RT21 to either the transport route RT31 or RT32. The transport route RT31 is a transport route for providing products to customers at table TB1, and the transport route RT32 is a transport route for providing products to customers at table TB2.

[0078] The customer conveyor 108 transports items along the transport path RT22 to table TB3 or TB4, and by operating the downstream switching lever 162, transports products from the transport path RT22 to either the transport path RT33 or RT34. The transport path RT33 is a transport path for providing products to customers at table TB3, and the transport path RT34 is a transport path for providing products to customers at table TB4.

[0079] Customer conveyor 109 transports items along transport path RT23 to table TB5 or TB6, and by operating downstream switch lever 163, transports products from transport path RT23 to either transport path RT35 or RT36. Transport path RT35 is a transport path for providing products to customers at table TB5, and transport path RT36 is a transport path for providing products to customers at table TB6.

[0080] Pushing devices 145-147 (an example of a first pushing device) are provided to push out objects being transported along the straight lane RT12 onto the respective transport routes RT21-RT23. When each of the pushing devices 145-147 operates, the object that has been transported to a position on the straight lane RT12 where it connects with the respective transport routes RT21-RT23 is pushed out onto each of the transport routes RT21-RT23.

[0081] Each of the conveying routes RT41 to RT44 is a shortcut route that takes a shortcut from a branching position on the circular conveying route RT1 to a merging position on the circular conveying route RT1. Each of the conveying routes RT41 and RT42 is a shortcut route for conveyed objects from the straight lane RT11 to the straight lane RT12. Each of the conveying routes RT43 and RT44 is a shortcut route for conveyed objects from the straight lane RT12 to the straight lane RT11. Each of the rollers 110 to 113 conveys the necessary conveyed objects along each of the conveying routes RT41 to RT44. The conveying direction of each of the rollers 110 to 113 is indicated by a black arrow on each of the conveying routes RT41 to RT44. Each of the conveying routes RT41 to RT44 is provided at a branching position from the circular conveying route RT1 with each of the extrusion devices 141 to 144 (an example of a second extrusion device).

[0082] When push-out device 141 or 142 operates, an object that has been conveyed to a branch position on straight lane RT11 with conveying path RT41 or RT42 is pushed onto conveying path RT41 or RT42. The pushed-out object is conveyed along conveying path RT41 or RT42 by rollers 110 or 111, and moves to a junction position on straight lane RT11 with conveying path RT41 or RT42. As a result, the object that has passed through conveying path RT41 or RT42 moves from a predetermined branch position on straight lane RT11 to a predetermined junction position on straight lane RT12 without making a full circuit of circular conveying path RT1.

[0083] FIG. 6 is a diagram showing the path along which the transported object passes when it is transported from the supply conveyor 102 to the table TB5.

[0084] 6, when an object is transported from supply conveyor 102 to table TB5 (assuming that the placement space of transport path RT35 is not fully filled), control device 1 operates push-out device 141, push-out device 147, and switching lever 163 at the required timing. As a result, the object is transported to transport path RT35 corresponding to table TB5 via straight lane RT11, transport path RT41, straight lane RT12, and transport path RT23 in that order, as shown by arrow AR1 in FIG.

[0085] 1, when push-out device 143 or 144 operates, an object conveyed to a branch position on straight lane RT12 with conveying path RT43 or RT44 is pushed onto conveying path RT43 or RT44. The pushed object is conveyed along conveying path RT43 or RT44 by rollers 112 or 113 and moves to a junction position on straight lane RT11 with conveying path RT43 or RT44. As a result, an object that has passed through conveying path RT43 or RT44 moves from a predetermined branch position on straight lane RT12 to a predetermined junction position on straight lane RT11 without making a full circuit of circular conveying path RT1. Therefore, if passing through conveying path RT43 or RT44 would shorten the path to the destination, control device 1 sets the path that the object should take to be a path that passes through conveying path RT43 or RT44.

[0086] When objects accumulate on one of the transport routes RT31-RT36, the placement space on that route becomes full, making it impossible to send the next object to that route. Therefore, when a sensor (not shown) or the like detects that the placement space on one of the transport routes RT31-RT36 is full, the control device 1 sets (changes) the route through which the objects should pass to a circulation route and controls the operation of the necessary pushers 145-147 according to the set route. This prevents objects accumulated on the transport route from being pushed out by newly transported objects. The objects whose route has been changed will circulate on the circular transport route RT1 until a placement space becomes available on the destination transport route. The control device 1 detects that the placement space on the transport route is full using sensors (not shown) provided on each of the transport routes RT31-RT36.

[0087] Furthermore, the control device 1 controls the transporting path RT41 to RT44 so that the transported item does not move too far from the destination table, so that the transported item can be transported to the destination table immediately when a space becomes available on the destination transporting path.

[0088] FIG. 7 is a diagram showing a path along which an object to be transferred to the table TB4 passes when the placement space of the transfer route RT34 is fully loaded.

[0089] Referring to FIG. 7, for example, when the placement space of the transport path RT34 corresponding to table TB4 is full, and the control device 1 identifies the transported object as the object to be transported to table TB4 based on an image of the transported object captured by camera 136, the control device 1 sets (changes) the route to the circulation path. The control device 1 does not operate push-out device 146, but operates push-out device 144 to push the transported object from straight lane RT22 to RT11. The control device 1 also operates push-out device 142 to push the transported object from straight lane RT11 to RT12. This allows the transported object to be transported to table TB4 to wait on the circulation path formed by the centers of each of straight lanes RT11 and RT12 and transport paths RT44 and RT42. This circulation path is indicated by arrow AR2 in FIG. 7.

[0090] When a free space becomes available in the placement space of the transfer path RT34 corresponding to table TB4, the control device 1 sets (changes) the path so that the object waiting on the circulation path is directed to the transfer path RT22. The control device 1 then operates the push-out device 146 at the necessary timing to transfer the object from the straight lane RT22 to the transfer path RT34, as shown by the arrow AR3 in FIG. 7. While waiting on the circulation path, the object to be transferred to table TB4 is not transferred to the transfer path RT1 to the left of the transfer path RT44 or the transfer path RT1 to the right of the transfer path RT42. Therefore, when a free space becomes available in the placement space of the transfer path RT34 corresponding to table TB4, the object can be quickly transferred from the circulation path to the transfer path RT34.

[0091] Referring to FIG. 1, each of the cameras 181-184 has a detection area P81-P84 near the junction of the conveying routes RT41-RT44 with the circular conveying route RT1. Each of the cameras 181-184 captures an image of the conveyed object passing through each of the detection areas P81-P84. The control device 1 extracts characteristic information from the image of the conveyed object captured by each of the cameras 181-184 and identifies the conveyed object based on the extracted characteristic information. Each of the detection areas P81-P84 may be provided on the circular conveying route RT1 or on each of the conveying routes RT41-RT44. By providing each of the cameras 181-184, it is possible to detect that the conveyed object conveyed on each of the conveying routes RT41-RT44 has returned to the circular conveying route RT1, thereby improving conveying accuracy.

[0092] When the control device 1 detects, based on an image taken by a camera (not shown), that the transported object has arrived at the destination transport route (table) among the transport routes RT31 to RT36, the control device 1 may update the "status" column of the order information corresponding to the transported object in the order table to "transport completed." Detection of transport completion using a camera is optional.

[0093] The product providing device configured as described above makes it possible to provide products easily and quickly.

[0094] Next, an example of a method for extracting feature information will be described. The feature information may be any information extracted from an image captured by a camera, and the type and combination of the information may be arbitrary.

[0095] 8 is a diagram illustrating an example of a method for extracting feature information. In this embodiment, an example is shown in which feature information is extracted using an image of the transported object captured from above, but the direction in which the image of the transported object is captured is arbitrary.

[0096] 8(a), the target image IM1 typically shows a conveyed object BT1 (an example of a conveyed object and a target object) including a plate BT11 (an example of a placement portion) and a product BT12 (an example of a product) placed on the plate BT11. The plate BT11 includes any number of reference marks MK (an example of a reference mark). The reference marks MK are, for example, a store logo.

[0097] Referring to FIG. 8(b), the control device 1 performs edge processing on the image IM1 to divide the image IM1 into a portion IM11 of an image of a plate and a portion IM12 of an image of a product. The control device 1 then extracts feature points from the portion IM12 of the product image. Specifically, the control device 1 extracts, from the portion IM12 of the product image, feature points of the product shape based on the product's outline, for example, and feature points of the product's pattern due to the product's streaks, unevenness, etc. The extracted feature points of the product shape are set as first feature information. The feature points of the product pattern are set as second feature information. Furthermore, the control device 1 divides the portion IM12 of the product image into multiple regions and extracts color information (for example, information obtained by numerically converting the R component, G component, and B component) for each region. The extracted color information is set as third feature information.

[0098] Referring to FIG. 8(c), the control device 1 acquires the position of the product BT12 relative to the reference mark MK of the plate BT11 in image IM1 based on the portion IM11 of the image of the plate and the portion IM12 of the image of the product. The position of the product BT12 relative to the reference mark MK in image IM1 may be determined by any method, such as the direction and distance by which the center CR2 of the product is shifted relative to the center CR1 of the three reference marks MK, or the tilt angle of the approximately rectangular product relative to the line connecting the two reference marks MK. Information about the position of the product relative to the reference mark MK in image IM1 is treated as fourth feature information. Note that if the object included in the target image does not include a plate, there is no image of the plate, and therefore the fourth feature information does not need to be extracted.

[0099] In this way, the first to fourth characteristic information are extracted from the image IM1. If the image IM1 was captured by one of the cameras 121 to 125 (Fig. 1), the control device 1 assigns names to the extracted first to fourth characteristic information using the method described with reference to Fig. 5, and adds them to the order information corresponding to the transported item in the order table.

[0100] The shape, pattern, and color of each product vary from product to product during manufacturing. Also, the position of each product relative to the reference mark on the plate varies from product to product when the product is placed on the plate. The first to fourth characteristic information focus on these differences.

[0101] Next, a method for identifying an object passing through each detection area will be described.

[0102] Referring to FIG. 1, the control device 1 captures images of the detection areas P31 to P37 and P81 to P84 using cameras 131 to 137 and 181 to 184 (examples of second imaging means) on the conveyance route, respectively, and detects the management target from the images captured by these cameras. The management target may be detected using the learning model 43 or the above-described method for extracting feature information. When the management target is detected, the control device 1 sets the transported object detected as the management target as the target object. The control device 1 acquires an image of the target object and extracts feature information of the target object from the image of the target object using the above-described method.

[0103] After extracting the characteristic information of the object, the control device 1 refers to the order table and extracts order information whose next passing position is the same as the detection area where the object was detected. For example, if the order table has the contents shown in FIG. 5 and the detection area where the object was detected is detection area P35 (FIG. 1), the control device 1 extracts five pieces of order information each containing the order IDs "00115," "00116," "00117," "00118," and "00119," respectively. Note that the control device 1 may extract all order information in the order table, regardless of the next passing position information included in the order information, or may extract all order information with a status of "in transit."

[0104] Next, the control device 1 selects one piece of order information from the extracted order information, compares the characteristic information of the selected order information with the characteristic information of the object, and determines whether the transported object corresponding to the characteristic information of the selected order information is the same as the object. For example, the control device 1 may determine whether the first characteristic information of the selected order information is the same as the first characteristic information of the object, whether the second characteristic information of the selected order information is the same as the second characteristic information of the object, whether the third characteristic information of the selected order information is the same as the third characteristic information of the object, and whether the fourth characteristic information of the selected order information is the same as the fourth characteristic information of the object. Based on the four determination results obtained in this manner, it may be determined whether the transported object corresponding to the characteristic information of the selected order information (more specifically, the transported object shown in the image from which the characteristic information included in the selected order information was extracted) is the same as the object. When making the above four determinations, each piece of characteristic information may be quantified before determining whether they are identical.

[0105] After obtaining four discrimination results, for example, if a predetermined number of the four discrimination results are identical, it may be determined that the transported item corresponding to the characteristic information of the selected order information is the same as the target object. Alternatively, only some of the first to fourth characteristic information may be extracted from the image, and a determination as to whether the items are identical may be made based on the extracted characteristic information. As characteristic information different from the first to fourth characteristic information, characteristic information indicating the glossiness of the product may be further extracted from the image, and a determination as to whether the items are identical may be made based on the extracted characteristic information. In particular, if the product is perishable, the product will dry out over time, causing the glossiness of the product to change.

[0106] In this way, the control device 1 selects order information one by one from the extracted order information, and determines whether the image corresponding to the feature information of the selected order information is the same as the image of the object. If it is determined that they are the same, the control device 1 identifies the object as the transported object corresponding to the selected order information.

[0107] It should be noted that the position of the product relative to the plate may change or the shape of the product may change while the product is being transported. In particular, if the product is sushi, the toppings placed on top of the rice may crumble during transport. Also, a foreign object (typically something that has fallen from the product being transported (such as a piece of salmon roe if the product is salmon roe sushi)) may appear on the transport path and be detected as the target object. When such changes occur, it may be determined that none of the images corresponding to the extracted feature information of the order information are identical to the image of the target object, and the target object may not be identified.

[0108] Therefore, if it is determined that none of the images corresponding to the characteristic information of the extracted order information are identical to the image of the object, the control device 1 may notify the touch panels 151-155 or the like of an error indicating an abnormality in the transported object. This notification may include information from the camera that captured the object. This allows each of the craftsmen SF1-SF5 to take necessary measures, such as removing the transported object with the abnormality, before it arrives at the destination table.

[0109] Furthermore, if it is determined that none of the images corresponding to the feature information of the extracted order information are identical to the image of the target object, the control device 1 may identify the target object based on the next detection position and expected passage time of the order information registered in the order table. For example, if the order table has the content shown in Fig. 5 and the detection area in which the target object was detected is detection area P35, the control device 1 may identify as the target object the transported object corresponding to the order information with order ID "00115", which is the order information with the earliest expected passage time, among the order information whose next passage position is identical to the detection area in which the target object was detected.

[0110] Next, a flowchart showing the operation of the product providing device according to this embodiment will be described.

[0111] FIG. 9 is a flowchart showing the operation of the control device 1 when the control device 1 receives an order from a customer through the table touch system in one embodiment of the present invention.

[0112] 9, the control device 1 determines whether or not an order from a customer has been accepted through any one of the table touch systems 171 to 176 in Fig. 1 (S101). The control device 1 repeats the process of step S101 until it is determined that an order from a customer has been accepted.

[0113] In step S101, if it is determined that an order from a customer has been accepted (YES in S101), the control device 1 assigns an order ID to the order, creates new order information including the order ID, delivery destination, product type, order acceptance time, and status (S103), and registers the new order information in the order table (S105). Next, the control device 1 selects a touch panel (a touch panel for the craftsman preparing the product) to display the order from among the touch panels 151 to 155 (FIG. 1) (S106). The control device 1 then updates the screen of the selected touch panel so that the new order information is displayed (S107), and ends the process.

[0114] FIG. 10 is a flowchart showing the operation of the control device 1 in one embodiment of the present invention when the control device 1 receives a predetermined operation via the touch panel indicating that the placement of the transported object on the supply conveyor has been completed.

[0115] 10, the control device 1 determines whether or not a predetermined operation indicating that the placement of the transported objects on the supply conveyor has been completed has been received through any one of the touch panels 151 to 155 (FIG. 1) (S201). The control device 1 repeats the process of step S201 until it determines that a predetermined operation indicating that the placement of the transported objects on the supply conveyor has been completed has been received.

[0116] In step S201, if it is determined that a predetermined operation indicating that the object has been placed on the supply conveyor has been received (YES in S201), the control device 1 updates the screen of the touch panel that received the predetermined operation to reflect that the object has been placed (S203), and updates the status of the order information corresponding to the object (S204). Next, the control device 1 starts conveying the object by operating the supply conveyor 102-106 (FIG. 1) corresponding to the touch panel that received the predetermined operation (S205). Next, the control device 1 determines whether or not the object to be managed has been detected based on an image captured by the camera 121-125 (FIG. 1) corresponding to the touch panel that received the predetermined operation (S207). The control device 1 repeats the process of step S207 until it determines that the object to be managed has been detected.

[0117] If it is determined in step S207 that a management target object has been detected (YES in S207), the control device 1 extracts feature information from the captured image of the transported object by performing a feature extraction process (S900) described below on the captured image of the transported object. The control device 1 then adds the extracted feature information to the order information of the transported object (S208) and updates the status of the order information (S209). The control device 1 then sets a route for the transported object to pass based on the detected position of the transported object and the destination information in the order information of the transported object (S210). The control device 1 then identifies the next passing position based on the set route (S211) and calculates the expected passing time (S213). The control device 1 then adds the identified detection area and the calculated expected passing time to the order information corresponding to the transported object (S215), and ends the process.

[0118] 11 and 12 are flowcharts showing the operation of the control device 1 when a managed object is detected by any of the cameras 131 to 137 and 181 to 184 in FIG. 1 in one embodiment of the present invention.

[0119] 11, the control device 1 determines whether or not a managed object has been detected by any of the cameras 131 to 137 and 181 to 184 based on images captured by each of the cameras 131 to 137 and 181 to 184 (S301). The control device 1 repeats the process of step S301 until it determines that a managed object has been detected by any of the cameras 131 to 137 and 181 to 184.

[0120] In step S301, if it is determined that a transported object to be managed has been detected (YES in S301), the control device 1 extracts feature information of the object by performing a feature extraction process (S900) described below on the image of the detected object to be managed. Next, the control device 1 extracts order information from the order table, where the detection area in which the object was detected is the next passing position (S303). Next, the control device 1 selects one piece of order information from the extracted order information (S305). The control device 1 compares the feature information of the selected order information with the feature information of the object (S307), and determines whether the transported object corresponding to the selected order information is the same as the object (S311).

[0121] In step S311, if it is determined that the transported item corresponding to the selected order information is the same as the target object (YES in S311), the control device 1 identifies the transported item corresponding to the selected order information as the target object (S313) and proceeds to processing of step S321 in Figure 12.

[0122] In step S311, if it is determined that the transported object corresponding to the selected order information is not the same as the target object (NO in S311), the control device 1 determines whether or not all of the extracted order information has been selected (S315). In step S315, if it is determined that all of the extracted order information has not been selected (NO in S315), the control device 1 proceeds to the processing of step S305.

[0123] In step S315, if it is determined that all of the extracted order information has been selected (YES in S315), the control device 1 determines whether or not the target object can be identified based on the order information registered in the order table (S317).

[0124] In step S317, if it is determined that the object can be identified (YES in S317), the control device 1 identifies the object (S313), and proceeds to the processing of step S321 in FIG.

[0125] In step S317, if it is determined that the object cannot be identified (NO in S317), the control device 1 notifies an error (S319) and ends the process.

[0126] Referring to FIG. 12, in step S321, the control device 1 refers to the order information of the identified object (S321) and sets a route through which the object should pass (S325). The route may be set based on the position of the detection area of the camera that captured the image of the object, the delivery destination, and whether the placement space on the delivery path at the delivery destination is full (detection result of a sensor (not shown) that detects the placement space). Next, the control device 1 determines whether the camera that captured the image of the object has a push-out device (S327). Specifically, if the camera that captured the image of the object is one of cameras 131 to 137 (FIG. 1), it is determined that the camera has a push-out device. On the other hand, if the camera that captured the image of the object is one of cameras 181 to 184 (FIG. 1), it is determined that the camera does not have a push-out device.

[0127] In step S327, if it is determined that the camera that captured the image of the object has an extrusion device (YES in S327), the control device 1 determines whether or not it is necessary to have the extrusion device perform an operation based on the set path (S329).

[0128] In step S329, when it is determined that the extrusion device needs to be operated (YES in S329), the control device 1 causes the extrusion device to perform the extrusion operation (S331), and proceeds to the process of step S333.

[0129] If it is determined in step S327 that the camera that captured the image of the object does not have a corresponding extrusion device (NO in S327), or if it is determined in step S329 that there is no need to execute the operation of the extrusion device (NO in S329), the control device 1 proceeds to processing in step S333.

[0130] In step S333, the control device 1 identifies the next pass position of the object based on the set route, and calculates the predicted pass time when the object will pass the identified next pass position (S333).The control device 1 then updates the next pass position and predicted pass time in the order information of the object (S335), and ends the process.

[0131] FIG. 13 shows a subroutine of the feature extraction process (S900) in FIGS.

[0132] Referring to FIG. 13, in the feature extraction process of step S900, the control device 1 performs edge processing on the target image to divide the target image into a plate image portion and a product image portion (S901). Next, the control device 1 extracts feature points of the product shape from the product image portion (S903) and extracts feature points of the product pattern from the product image portion (S905). Next, the control device 1 divides the product image portion into a plurality of regions and extracts color information (product color information) for each region (S907). Next, the control device 1 obtains the position of the product relative to the reference mark in the target image (S909). Next, the control device 1 creates feature information including the feature points of the product shape, the feature points of the product pattern, the product color information, and the product position relative to the reference mark as first to fourth feature information, respectively (S911), and returns.

[0133] FIG. 14 is a diagram schematically illustrating neural networks 500 of the learning models 43 and 45 (FIG. 2) in one embodiment of the present invention.

[0134] 14, the neural network 500 of each of the learning models 43 and 45 (FIG. 2) is a so-called hierarchical neural network in which a large number of artificial neurons (shown as circles in FIG. 14) are connected to form a hierarchy. The hierarchical neural network includes input artificial neurons, processing artificial neurons, and output artificial neurons.

[0135] Problem data 510 is the target of processing by the neural network 500. The problem data 510 is acquired by input artificial neurons in the input layer 501. The input artificial neurons are arranged in parallel to form the input layer 501. The problem data 510 is distributed to the processing artificial neurons.

[0136] The processing artificial neurons are connected to the input artificial neurons. The processing artificial neurons are arranged in parallel to form the hidden layer 502. The hidden layer 502 may have multiple layers. A neural network with three or more layers and hidden layers 502 is called a deep neural network.

[0137] The neural network may be a so-called convolutional neural network, which is a deep neural network consisting of alternating convolutional layers and pooling layers.

[0138] The output artificial neuron outputs training data 511 to the outside. The output artificial neuron constitutes the output layer 503. The neural network 500 is trained so that training data 511 is output when problem data 510 is input (specifically, the parameters of the training models 43 and 45 (FIG. 2) are adjusted).

[0139] The learning data used to create learning model 43 may include multiple pairs of sample images and determination results as to whether the sample images contain the detection target. In this case, the sample images become question data 510, and the determination results as to whether the sample images contain the managed object become training data 511. Through machine learning using this learning data, learning model 43 outputs a determination result as to whether the managed object is included in the images captured by each of cameras 121 to 125, cameras 131 to 137, and 181 to 184 in FIG. 1 when the images are input.

[0140] The learning data used to create learning model 45 may include multiple pairs of two sample images and a determination result as to whether the transported object shown in the two sample images is the same. In this case, the two sample images become problem data 510, and the determination result as to whether the transported object shown in the two sample images is the same becomes training data 511. Through machine learning using this learning data, when a specific image and an image of an object are input, learning model 45 outputs a determination result as to whether the transported object shown in the specific image is the same as the object.

[0141] (Effects of the embodiment)

[0142] In the above-described embodiment, it is possible to identify which of the multiple transported objects the object is based on each of the images of the multiple transported objects passing through a first area on the transport path and an image of the object that has passed through a second area on the transport path. According to the above-described embodiment, it is not necessary to record an ID in an RF tag to identify the object, and the device configuration can be simplified.

[0143] In particular, when the transported items are sushi, there are cases where multiple transported items with the same type of topping are transported along the transport path at the same time. Even though these transported items have different appearances, they need to be transported to different destinations. According to the above-described embodiment, even in such cases, each of the multiple transported items with the same type of topping can be distinguished based on the image, and they can be transported to the required destination.

[0144] (others)

[0145] The feature information included in the order information may be extracted from an image of the transported object passing through any one of the detection areas P31 to P37 and P81 to P84 in Fig. 1. Every time the control device 1 identifies an object in any one of the detection areas P31 to P37 and P81 to P84, it may update the feature information in the order information corresponding to that object with the feature information extracted from the image of the object.

[0146] The target object may be identified by an extrusion device corresponding to the camera instead of the control device 1. In this case, the extrusion device may extract the order information from an order table stored in the control device 1, or each extrusion device may store an order table that is synchronized at a predetermined timing.

[0147] The entire conveying routes RT1-RT6, RT21-23, RT31-RT36, and RT41-RT44 provided in the product supply device of Fig. 1 may be divided into a plurality of detection areas, and a plurality of imaging means (cameras) may be provided to capture images of conveyed objects passing through each of the plurality of detection areas. This makes it possible to manage the status of conveyed objects being conveyed along the entire conveying routes RT1-RT6, RT21-23, RT31-RT36, and RT41-RT44. As a result, if a conveyed object on a conveying route is removed by a craftsman or the like due to a product abnormality or the like, this removal can be immediately reflected in the order information, enabling more advanced conveying control.

[0148] The identification device of this embodiment may be used in devices other than product supply devices, such as in a factory, in an inspection device that transports transported objects such as intermediate products or finished products and inspects the transported objects during transport, and may be used as a device for identifying the transported objects during transport.

[0149] The processes in the above-described embodiments and variations may be performed by software or hardware circuits. A program for executing the processes in the above-described embodiments may be provided, or the program may be recorded on a recording medium such as a CD-ROM, flexible disk, hard disk, ROM, RAM, or memory card and provided to the user. The program is executed by a computer such as a CPU. The program may also be downloaded to a device via a communication line such as the Internet.

[0150] The above-described embodiments and modifications can be combined as appropriate.

[0151] The above-described embodiments and modifications should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0152] 1. Control device 11 Camera control unit 13 Order Reception Department 15 Order Information Management Department 17 Detection unit 18 Identity output unit (an example of identity output means) 19 Selection section 21 Division section (an example of division means) 23 Position acquisition unit (an example of a position acquisition means) 25 Feature point extraction unit (an example of feature point extraction means) 27 Discrimination unit (example of discrimination unit) 29 Identification unit (an example of identification means) 31 Extrusion control section 33 Route setting section 35 Prediction unit (an example of a prediction means) 37 Operation display section 38 Communications Department 39 Memory section 41 Order Table 43,45 Learning Model 101 Circulating conveyor 102~106 Supply conveyor 107~109 Customer conveyor 110~113 Roller 121-125, 131-137, 181-184 Camera (an example of the first and second photographing means) 141-147 Extrusion device (examples of the first and second extrusion devices) 151~155 Touch panel 161~163 Switch lever 171~176 Table Touch System 500 Neural Networks 501 Input Layer 502 Middle Class 503 Output Layer 510 Problem Data 511 Teacher Data BT1 Transported object (an example of a transported object and target object) BT11 Plate (Example of a mounting part) BT12 product (example of product) CR1,CR2 center IM1 Image from which feature information is extracted IM11 Plate image part IM12 Product image section MK fiducial mark (example of fiducial mark) P21-P25, P31-P37, P81-P84 detection areas (examples of the first and second areas) RT1~RT6, RT21~RT23, RT31~RT36, RT41~RT44 Transport route (example of transport route) RT11,RT12 straight lane SF1~SF5 Craftsman Tables TB1~TB6 TE Established Route

Claims

1. An identification device for identifying an object to be conveyed along a conveyance path, a first photographing means for photographing each of the plurality of transported objects in turn while the objects are passing through a first area on the transport path; a second photographing means for photographing an object that has passed through a second area on the conveyance path after photographing each of the images of the plurality of conveyed objects; an identification means for identifying which of the plurality of transported objects the target object is based on each of the images of the plurality of transported objects and an image of the target object; the transported object includes a placement section and a product placed on the placement section, a feature point extraction means for extracting feature points indicating the shape, pattern, and gloss of the product from a portion of the product image in each of a specific image among the plurality of images of the transported objects and the image of the target object; a determination means for determining whether the transported object in the specific image is the same as the object based on feature points extracted by the feature point extraction means from the image of the product in each of the specific image and the image of the object, When the determination means determines that the transported object in the specific image is the same as the target object, the identification means identifies the target object as the transported object in the specific image.

2. The mounting portion includes a reference mark, a dividing means for dividing each of the specific image and the image of the object into an image portion of the placement section and an image portion of the commodity; a position acquisition means for acquiring a position of the product relative to the reference mark in each of the specific image and the image of the object; The identification device described in claim 1, wherein the discrimination means further determines whether the transported object in the specific image is the same as the object based on the position of the product relative to the reference mark in each of the specific image and the image of the object.

3. A route setting means for setting a route through which each of the plurality of transported objects should pass based on the first area and the destination of each of the plurality of transported objects; a prediction unit that predicts a timing at which each of the plurality of transported objects will pass through a position in the second area based on the route set by the route setting unit, The identification device described in claim 1, wherein when the discrimination means determines that none of the images of the multiple transported objects is identical to the image of the target object, the identification means identifies which of the multiple transported objects the target object is based on the timing at which each of the multiple transported objects passes through the position of the second area.

4. a plurality of image capturing means including the first and second image capturing means; 2. The identification device according to claim 1, wherein when the entire conveying path is divided into a plurality of areas including the first and second areas, each of the plurality of photographing means photographs the conveyed object passing through each of the plurality of areas.

5. the conveying route includes a circular conveying route and a branching route that branches off from a predetermined position on the circular conveying route and heads toward a customer who has ordered the conveyed item, The identification device described in claim 1, further comprising a first extrusion device that extrudes the object from the predetermined position on the circular conveying path to the branch path after the identification means identifies which of the multiple conveyed objects the object is.

6. the conveying path further includes a shortcut path that takes a shortcut from a branching position on the annular conveying path to a joining position on the annular conveying path, The identification device described in claim 5, further comprising a second extrusion device that extrudes the object from the circular conveying path to the shortcut path at the branching position after the identification means identifies which of the multiple conveyed objects the object is.

Images