Item management device, item management method, item management program, and item management system

Abstract

To provide an item management device excellent in collecting learning data for item recognition.SOLUTION: An item management device according to an embodiment includes an information acquisition interface, a processor, and a communication interface. The information acquisition interface acquires item management information. The processor selects item information that specifies an item to be imaged based on the item management information, and generates a shipping instruction for the item specified by the item information selected as the item to be imaged. The communication interface communicates with a transport control device that controls a transport device that transports an item, and a picking control device that controls a picking device that recognizes and picks up an item based on a captured image of the item transported by the transport device, to output at least a leaving instruction to the transport control device and receive a photographed image acquired in response to the leaving instruction.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] An embodiment of the present invention relates to an item management device, an item management method, an item management program, and an item management system. [Background technology]

[0002] In recent years, the volume of various products (hereinafter referred to as "goods") handled at distribution centers and other locations in logistics has increased, and automation of goods processing has been promoted. For example, in a warehouse, various automated devices work together, and each device delivers goods or cases or equipment containing goods according to instructions from a higher-level device. As automated devices, conveying devices and picking devices are known, and the retrieval work of goods stored in a warehouse is automated by these devices.

[0003] For example, the higher-level device instructs the out-of-stock of an item based on the out-of-stock list. The transport device transports the item based on the out-of-stock instruction. The picking device photographs the item to be transported, recognizes the item from the photographed image by machine learning, and picks up the recognized item. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2022-181623 A Summary of the Invention [Problem to be solved by the invention]

[0005] As described above, when machine learning is used for item recognition processing, it is necessary to create a recognition dictionary based on the collection of learning data, etc. For example, in advance, in a learning data collection area, images of items to be picked are captured to collect learning data, and a recognition dictionary is created. In this case, if the lighting conditions of the picking area where the picking device is actually placed are different from the lighting conditions of the learning data collection area, there is a risk that the recognition accuracy will decrease, and improvements are required. In addition, in the logistics industry, new items may be added each season, and there is a demand to reduce the burden of collecting learning data, etc.

[0006] An object of the present invention is to provide an article management device, an article management method, an article management program, and an article management system that are excellent in collecting learning data for article recognition. [Means for solving the problem]

[0007] An item management device according to an embodiment includes an information acquisition interface, a processor, and a communication interface. The information acquisition interface acquires item management information. The processor selects item information specifying an item for which an image is to be acquired based on the item management information, and generates an outgoing instruction for the item specified by the item information selected as the image acquisition target. The communication interface communicates with a transport control device that controls a transport device that transports items based on the outgoing instruction, and a picking control device that controls a picking device that recognizes and picks the items based on a captured image acquired from the item transported by the transport device, and outputs an outgoing instruction to at least the transport control device, and receives the captured image acquired in response to the outgoing instruction. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a conceptual diagram showing a logistics system, which is an example of an article processing system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a WES according to an embodiment. [Diagram 3]FIG. 3 is a block diagram illustrating an example of a transport control device according to the embodiment. [Figure 4] FIG. 4 is a block diagram illustrating an example of a picking control device according to the embodiment. [Diagram 5] FIG. 5 is a sequence diagram showing an example of an item retrieval process according to the embodiment. [Figure 6] FIG. 6 is a sequence diagram illustrating an example of the first article image acquisition process according to the embodiment. [Figure 7] FIG. 7 is a sequence diagram illustrating an example of the second article image acquisition process according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Each embodiment will be described below with reference to the drawings. [composition] FIG. 1 is a conceptual diagram showing a logistics system, which is an example of an article processing system according to an embodiment. As shown in FIG. 1, a logistics system S includes a warehouse management system (WMS) 1 and a warehouse processing system 2.

[0010] The warehouse processing system 2 includes a warehouse execution system (WES) 30, a transport control device 40, a transport device (transport robot) 41, a picking control device 50, a picking device (picking robot) 51, and a recognition camera 60. For example, one transport control device 40 controls one or more transport devices 41, and one picking control device 50 controls one or more picking devices 51.

[0011] The WMS1 can be configured with one or more computers, i.e., a processor, a memory, an interface, etc. The processor is a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), etc. The WMS1 receives item management information such as an order list from a higher-level server and transmits it to the WES30.

[0012] An order list is a list of items stored on a warehouse shelf or the like, and specifies one or more items. For example, an order list includes item information, ordering information, delivery information, etc. Item information includes the number of items, item identification information, item name, etc. Order information includes the order date and time, orderer, etc. Delivery information includes the delivery destination, delivery date and time, recipient, etc. Furthermore, the order list includes a flag indicating first entry, a flag indicating that the image is not registered or does not match the recognition dictionary, and the success rate of grasping the item.

[0013] WES30 (server) is an example of an item management device, and is composed of one or more general-purpose computers. WES30 communicates with WMS1, and also communicates with the transport control device 40 and the picking control device 50. WES30 receives item management information, such as an order list, sent from WMS1, and generates and outputs various instructions based on the item management information. WES30 generates an outgoing instruction to have the transport device 41 transport an item, and outputs the outgoing instruction to the transport control device 40. WES30 also generates a picking instruction to have the picking device 51 pick up an item transported by the transport device 41, and outputs the picking instruction to the picking control device 50.

[0014] The transport control device 40 is composed of one or more general-purpose computers. The transport control device 40 communicates with the WES 30 and also with the transport device 41. The transport control device 40 receives an outgoing instruction transmitted from the WES 30, generates a transport control signal for controlling the transport device 41 based on the outgoing instruction, transmits the transport control signal to the transport device 41, and controls the transport of items by the transport device 41. Note that instead of a picking instruction from the WES 30, the transport control device 40 may generate a picking instruction for causing a picking device 51 to pick up an item transported by the transport device 41 based on the outgoing instruction transmitted from the WES 30, and output the picking instruction to the picking control device 50.

[0015] The transport device 41 communicates with the transport control device 40, receives a transport control signal transmitted from the transport control device 40, and transports an article or a case containing an article from a storage position to a destination position based on the transport control signal. The transport device 41 is a transport robot of various types, such as an Auto Guided Forklift (AGF) that picks up a case containing an article from a high-rise position and transports it, an Auto Guided Vehicle (AGV) that transports a shelf or the like that stores cases containing articles, or an Autonomous Case-handling Robot (ACR) that transports cases containing articles.

[0016] The picking control device 50 is composed of one or more general-purpose computers. The picking control device 50 communicates with the WES 30 and also with the picking device 51. The picking control device 50 receives a picking instruction transmitted from the WES 30 or the transport control device 40, transmits a picking control signal to the picking device 51, and controls the picking of an item by the picking device 51. For example, the picking control device 50 recognizes an item image included in the captured image data from the recognition camera 60 based on an accumulated recognition dictionary, and generates a picking control signal that controls the picking device 51 based on the recognition result.

[0017] The picking device 51 communicates with the picking control device 50, receives a picking control signal transmitted from the picking control device 50, and grips an item and moves it from the gripping position to the target position based on the picking control signal. The picking control signal may be a signal that simply instructs the start of picking, in which case the picking device 51 grips an item that arrives at the gripping position or an item contained in a case based on a detection signal of an item or case arriving at the gripping position and moves it from the gripping position to the target position.

[0018] The recognition camera 60 is an RGB camera, a 3D camera, or a combination thereof that captures an image of the gripping position, and outputs the captured image data to the picking control device 50. The picking control device 50 uploads the captured image data to the WES 30. The picking control device 50 also recognizes the item image included in the image captured by the recognition camera 60 based on an accumulated recognition dictionary, and generates a picking control signal based on the recognition result. The picking control device 50 also periodically receives dictionary update data generated based on the uploaded captured image data from the WES 30, and updates the recognition dictionary.

[0019] FIG. 2 is a block diagram illustrating an example of a WES according to an embodiment. 2, the WES 30 includes a processor 301, a memory 302, an information acquisition interface 303, and a communication interface 304. Note that the information acquisition interface 303 and the communication interface 304 may be a common interface or may be independent interfaces.

[0020] The processor 301 is a CPU, an MPU, a DSP, or the like. The processor 301 operates based on a program stored in the memory 302. The processor 301 includes an input unit 3011, a selection unit 3012, a generation unit 3013, and an output unit 3014. The processor 301 executes the program stored in the memory 302 to realize the functions of each unit.

[0021] The input unit 3011 inputs an order list such as item management information. The selection unit 3012 selects item information specifying an item to be imaged based on the item management information. The generation unit 3013 generates a shipping instruction for the item specified by the item information selected as the image to be acquired. The output unit 3014 outputs the generated shipping instruction.

[0022] Memory 302 is a non-volatile memory, and stores an operating program for processor 301. Memory 302 also stores an inventory management database, an entry history, and an exit history, etc. The inventory management database is data indicating the storage location of each item, and includes warehouse map data (coordinate data), position identification information associated with the map data, storage position information associated with the position identification information, and item information associated with the storage position information. The entry history includes the entry date and time and item information of the entered items. The exit history includes the exit date and time and item information of the exited items.

[0023] The information acquisition interface 303 acquires an order list such as item management information sent from the WMS 1 .

[0024] The communication interface 304 communicates with a transport control device 40 that controls a transport device 41 that transports items, and a picking control device 50 that controls a picking device 51 that recognizes and picks items based on captured images of the items transported by the transport device 41. The communication interface 304 outputs a shipping instruction to the transport control device 40.

[0025] FIG. 3 is a block diagram illustrating an example of a transport control device according to the embodiment. As shown in FIG. 3, the transport control device 40 includes a processor 401, a memory 402, a communication interface 403, and the like.

[0026] The processor 401 is a CPU, an MPU, a DSP, or the like. The processor 401 operates based on a program stored in the memory 402. The processor 401 executes the program stored in the memory 402 to generate a control signal for controlling the conveying device 41 based on a delivery instruction from the WES 30.

[0027] The memory 402 is a non-volatile memory, and stores the operating program of the processor 401 and the like.

[0028] The communication interface 403 receives a shipping instruction from the WES 30 and transmits a control signal to the transport device 41 .

[0029] FIG. 4 is a block diagram illustrating an example of a picking control device according to the embodiment. As shown in FIG. 4, the picking control device 50 includes a processor 501, a memory 502, a communication interface 503, and the like.

[0030] The processor 501 is a CPU, an MPU, a DSP, or the like. The processor 501 operates based on a program stored in the memory 502. The processor 501 executes the program stored in the memory 502 to generate a control signal for controlling the picking device 51 based on a picking instruction from the WES 30 or the transport control device 40. The processor 501 also includes a recognition processing unit 5011. The recognition processing unit 5011 recognizes an article image included in an image captured by the recognition camera 60 based on a recognition dictionary stored in the memory 502.

[0031] The memory 502 is a non-volatile memory, and stores the operation program of the processor 501. The memory 502 also includes a storage area 5021 that stores a recognition dictionary.

[0032] The communication interface 503 receives a picking instruction from the WES 30 or the transport control device 40 and transmits a control signal to the picking device 51 .

[0033] Below, the item retrieval processing will be explained with reference to the sequence diagram shown in Figure 5, the first item image acquisition processing will be explained with reference to the sequence diagram shown in Figure 6, and the second item image acquisition processing will be explained with reference to the sequence diagram shown in Figure 7.

[0034] The item out-of-stock process is a process that outputs out-of-stock instructions and picking instructions to out-stock the target item. The first item image acquisition process outputs out-of-stock instructions and picking instructions, but is a dummy out-of-stock process that does not actually out-stock the target item but acquires an image of the target item. The first item image acquisition process can reuse the API (Application Interface) of the item out-of-stock process by utilizing the instructions of the item out-of-stock process. The second item image acquisition process is a process for photographing the item, which outputs out-of-stock instructions and photographing instructions to acquire an image of the target item.

[0035] [Item issue processing] FIG. 5 is a sequence diagram showing an example of an item retrieval process according to the embodiment. In the WES 30, the information acquisition interface 303 receives the goods management information such as a shipping list transmitted from the WMS 1 (ST101). The input unit 3011 inputs the goods management information received by the information acquisition interface 303.

[0036] The selection unit 3012 selects the item retrieval process or the item image acquisition process according to the time period. For example, when the current time is included in a first time period (e.g., 6:00-22:00), the selection unit 3012 selects the item retrieval process, and when the current time is included in a second time period (e.g., 23:00-5:00) different from the first time period, the selection unit 3012 selects the first or second item image acquisition process.

[0037] When an item outgoing process is selected, the selection unit 3012 selects item information specifying one or more items to be outgoing based on the item management information, and sets, for example, flag 0 (ST102). Flag 0 indicates that an item outgoing process has been selected. The generation unit 3013 generates an outgoing instruction for one or more items specified by the item information selected as the item to be outgoing, or an outgoing instruction for a case containing the specified one or more items, and generates a picking instruction corresponding to the outgoing instruction (ST103). The generation unit 3013 generates an outgoing quantity according to the outgoing instruction. The memory 302 stores the outgoing quantity according to the outgoing instruction.

[0038] Here, the retrieval of an item contained in a case will be described, but an item itself that is not contained in a case may be retrieved. The memory 302 stores an inventory management database indicating the storage location of each item. The generation unit 3013 detects the storage location of one or more items specified by the item information selected as the retrieval target based on the inventory management database. The generation unit 3013 generates a retrieval instruction for retrieving the case containing the one or more items specified by the item information from the detected storage location.

[0039] The output unit 3014 outputs the generated leaving instruction. The communication interface 304 outputs the leaving instruction to the transport control device 40 (ST104).

[0040] In the transport control device 40, the communication interface 403 receives the delivery instruction, and the processor 401 generates a transport control signal for controlling the transport of the transport device 41 based on the delivery instruction. The communication interface 403 transmits the generated transport control signal to the transport device 41 (ST105).

[0041] The transport device 41 receives the transport control signal. Based on the transport control signal, the transport device 41 moves to a storage position of one or more items designated by the item information selected as the item to be removed from the warehouse, and transports, for example, a case or the like containing the designated one or more items from the storage position to a gripping position (ST106).

[0042] The processor 401 of the transport control device 40 communicates with the transport device 41 via the communication interface 403 and monitors the traveling position of the transport device 41 in real time. The processor 401 detects that the transport device 41 has arrived at the gripping position, and the communication interface 403 outputs an arrival notification to the WES 30 (ST107). Note that the arrival notification is not essential. The difference between the case with and without the arrival notification will be explained in detail later.

[0043] In the WES 30, the communication interface 304 receives the arrival notification, the processor 301 outputs a picking instruction based on the reception of the arrival notification, and the communication interface 304 outputs the picking instruction to the picking control device 50 that recognizes and picks the captured image acquired in response to the picking instruction (ST108). Note that instead of the WES 30, the transport control device 40 may generate a picking instruction corresponding to the outgoing instruction transmitted from the WES 30 and output the picking instruction to the picking control device 50.

[0044] In the picking control device 50, the communication interface 503 receives the picking instruction, and the processor 501 outputs a photographing instruction to the recognition camera 60 based on the received picking instruction (ST109).

[0045] The recognition camera 60 photographs the case transported to the gripping position based on the photographing instruction (ST110), and outputs the photographed image data to the picking control device 50 (ST111).

[0046] In the picking control device 50, when captured image data is sent from the recognition camera 60, the communication interface 502 transmits an image acquisition notification to the WES 30 (ST112).

[0047] The WES 30 does not execute the delivery cancellation described below based on the set flag 0. Delivery cancellation will be described in detail later.

[0048] In the picking control device 50, the recognition processing unit 5011 of the processor 501 recognizes an article included in the captured image data sent from the recognition camera 60 based on the recognition dictionary in the memory 502 (storage area 5021) (ST113).

[0049] The recognition camera 60 may be a camera that captures video, and may output captured video data for a certain period of time. In this case, the recognition processing unit 5011 detects one or more frames from the captured video data output from the recognition camera 60 based on the timing of receiving a picking instruction, and recognizes the article image included in the detected frame based on the recognition dictionary in the memory 502 (storage area 5021).

[0050] The processor 501 generates a picking control signal based on the recognition result of the item, and the communication interface 503 outputs the picking control signal to the picking device 51 (ST114). For example, the picking control signal includes a signal for gripping (adsorbing or clamping) the item in a method designated for each item.

[0051] The picking device 51 picks up the article at the picking position based on the picking control signal and moves it to the target position (ST115).

[0052] The processor 501 communicates with the picking device 51 via the communication interface 503 and monitors the operation of the picking device 51 in real time. For example, the processor 501 detects whether the picking is successful or not based on the information output from the picking device 51, and further detects the time and number of times required for the successful picking, and stores the result of the picking in the memory 502 as a picking history. The picking history includes the date and time, the item identification information of the target item, and the picking result (successful or failed). When the processor 501 recognizes the completion of the outgoing based on the information output from the picking device 51, it outputs an outgoing completion notification to the WES 30 via the communication interface 503 (ST116). In addition, the processor 501 periodically outputs the picking history to the WES 30 via the communication interface 503.

[0053] In the WES 30, the communication interface 304 receives the outgoing completion notice, and the processor 301 updates the inventory management database stored in the memory 302 (ST117). For example, the processor 301 compares the outgoing quantity of the specified item stored in the memory 302 with the outgoing quantity of the specified item indicated in the outgoing completion notice, and if they match, subtracts the outgoing completion quantity of the specified item indicated in the outgoing completion notice from the inventory quantity of the specified item.

[0054] After an article is gripped from the case that has been transported to the gripping position based on the transport control signal, the transport device 41 transports the case that has been transported to the gripping position to its original position and returns the case to its original position. For example, the WES 30 may be triggered by an image acquisition notification from the picking control device 50 to output a transport control signal to return the case to its original position, or may be triggered by a picking completion notification from the picking control device 50 to output a transport control signal to return the case to its original position.

[0055] Here, we will provide additional information about the arrival notification in ST 107. As described above, the arrival notification is not essential.

[0056] The case where there is an arrival notification will be described. This is effective when the WES 30 outputs multiple outgoing instructions to the transport control device 40, and the WES 30, which has received an arrival notification corresponding to each outgoing instruction, outputs a picking instruction to the picking control device 50, triggered by the arrival notification corresponding to each outgoing instruction. This allows for consistency between the item that is the subject of the outgoing instruction from the WES 30 and the item to be picked by the picking device 51. Note that when the transport control device 40 outputs a picking instruction instead of the WES 30, it detects the arrival of the transport device 41 at the gripping position and outputs a picking instruction based on this arrival detection.

[0057] A case where there is no arrival notification will be described. In order to ensure consistency between the items that are the subject of an outgoing instruction from WES 30 and the items picked by the picking device 51, WES 30 outputs outgoing instructions one by one to the transport control device 40, and the picking control device 50 controls the picking of the items based on the outgoing instructions. This allows consistency between the items that are the subject of an outgoing instruction from WES 30 and the items picked by the picking device 51 even without an arrival notification. Note that when the transport control device 40 outputs a picking instruction instead of WES 30, as described above, it detects the arrival of the transport device 41 at the gripping position and outputs a picking instruction based on this arrival detection.

[0058] As mentioned above, if there is an arrival notification, multiple departure instructions can be output. If there is no arrival notification, sending and receiving the arrival notification can be omitted.

[0059] [First item image acquisition process] FIG. 6 is a sequence diagram illustrating an example of the first article image acquisition process according to the embodiment. In the WES 30, the information acquisition interface 303 receives the goods management information such as a shipping list transmitted from the WMS 1 (ST201). The input unit 3011 inputs the goods management information received by the information acquisition interface 303.

[0060] The selection unit 3012 selects the article retrieval process or the article image acquisition process according to the time period. For example, when the current time is included in the second time period (e.g., 23:00-5:00), the selection unit 3012 selects the first article image acquisition process.

[0061] When the item image acquisition process is selected, the selection unit 3012 selects item information specifying one or more items for which images are to be acquired based on the item management information, and sets, for example, flag 1 (ST202). Flag 1 indicates that the item image acquisition process using the dummy outbound process has been selected. The generation unit 3013 generates an outbound instruction for one or more items specified by the item information selected as the image acquisition target, or an outbound instruction for a case containing the specified one or more items, and generates a picking instruction and a cancellation instruction corresponding to the outbound instruction (ST203A). The generation unit 3013 generates an outbound quantity according to the outbound instruction. The memory 302 stores the outbound quantity according to the outbound instruction.

[0062] The memory 302 stores an inventory management database indicating the storage location of each item. The generation unit 3013 detects the storage location of one or more items specified by the item information selected as the image acquisition target based on the inventory management database. The generation unit 3013 generates a shipping instruction for shipping a case containing one or more items specified by the item information from the detected storage location.

[0063] The output unit 3014 outputs the generated leaving instruction. The communication interface 304 outputs the leaving instruction to the transport control device 40 (ST204).

[0064] In the transport control device 40, the communication interface 403 receives the delivery instruction, and the processor 401 generates a transport control signal for controlling the transport of the transport device 41 based on the delivery instruction. The communication interface 403 transmits the generated transport control signal to the transport device 41 (ST205).

[0065] The transport device 41 receives the transport control signal. Based on the transport control signal, the transport device 41 moves to a storage position of one or more items designated by the item information selected as the item to be removed from the warehouse, and transports, for example, a case or the like containing the designated one or more items from the storage position to a gripping position (ST206).

[0066] The processor 401 detects that the transport device 41 has arrived at the gripping position, and the communication interface 403 outputs an arrival notification to the WES 30 (ST207). Note that the arrival notification is not essential. The difference between the case with and without the arrival notification is as described above.

[0067] In the WES 30, the communication interface 304 receives the arrival notification, the processor 301 outputs a picking instruction based on the reception of the arrival notification, and the communication interface 304 outputs the picking instruction to the picking control device 50 that recognizes and picks the captured image acquired in response to the picking instruction (ST208A). Note that instead of the WES 30, the transport control device 40 may generate a picking instruction corresponding to the outgoing instruction transmitted from the WES 30 and output the picking instruction to the picking control device 50.

[0068] In the picking control device 50, the communication interface 503 receives the picking instruction, and the processor 501 outputs a photographing instruction to the recognition camera 60 based on the received picking instruction (ST209).

[0069] The recognition camera 60 photographs the case transported to the gripping position based on the photographing instruction (ST210), and outputs the photographed image data to the picking control device 50 (ST211).

[0070] In the picking control device 50, when captured image data is sent from the recognition camera 60, the communication interface 503 transmits an image acquisition notification to the WES 30 (ST212).

[0071] WES30 executes a shipping cancellation based on the set flag 1. For example, the generation unit 3013 generates a shipping cancellation. The processor 301 deletes the shipping quantity of the specified item stored in the memory 302 based on the shipping cancellation. Alternatively, the processor 301 may set the shipping completion quantity to 0 based on the shipping cancellation and update the inventory management database stored in the memory 302. That is, the quantity in the inventory management database does not change due to the dummy shipping process.

[0072] In addition, the output unit 3014 outputs a release cancel based on the acquisition of the photographed image. That is, the output unit 3014 outputs a release cancel based on the reception of the image acquisition notification. The communication interface 304 transmits the release cancel to the picking control device 50 (ST216).

[0073] In the picking control device 50, the communication interface 503 receives the shipping cancellation, cancels the item recognition and picking process based on the picking instruction, and uploads the captured image data sent from the recognition camera 60 to the WES 30 (ST217).

[0074] The recognition camera 60 may be a camera that captures moving images, and may output captured moving image data for a certain period of time. In this case, the recognition processing unit 5011 detects one or more frames from the captured moving image data output from the recognition camera 60 based on the timing of receiving a picking instruction, and uploads the captured image data of the detected frames to the WES 30.

[0075] In the WES 30, the communication interface 304 receives the captured image data, and the processor 301 updates the recognition dictionary stored in the memory 302 (ST218). Note that a person in charge may be in charge of some of the processing, such as area designation of the captured image data. The communication interface 304 also transmits the updated recognition dictionary or dictionary update data based on the captured image to the picking control device 50 (ST219).

[0076] In the picking control device 50, the communication interface 503 receives the updated recognition dictionary, and the processor 501 replaces the recognition dictionary in the memory area 5021 with the updated recognition dictionary. Alternatively, the processor 501 updates the recognition dictionary in the memory area 5021 based on the dictionary update data (ST220).

[0077] The transport device 41 transports the case transported to the gripping position to its original position based on the transport control signal, and returns the case to its original position. For example, the WES 30 is triggered by an image acquisition notification from the picking control device 50 to output a transport control signal and return the case to its original position.

[0078] Here, a description will be given of the selection of article information of the image acquisition target in ST 202. The selection unit 3012 selects article information of the image acquisition target based on the following conditions.

[0079] For example, the selection unit 3012 selects item information specifying an item that has been stocked for the first time as item information specifying one or more items for which an image is to be acquired based on the item management information. The selection unit 3012 detects item information specifying an item that has been stocked for the first time based on a flag indicating first stocking included in the item management information, and selects the detected item information. Alternatively, the selection unit 3012 detects item information specifying an item that has been stocked for the first time based on the stocking history (item identification information with a stocking record) stored in the memory 302, and selects the detected item information.

[0080] Furthermore, the selection unit 3012 selects, based on the item management information, item information specifying an item with no image registered or with no recognition dictionary support as item information specifying one or more items for which images are to be acquired. The selection unit 3012 detects item information specifying an item with no image registered or with no recognition dictionary support based on a flag or the like indicating that an image is not registered or that a recognition dictionary is not supported, which is included in the item management information, and selects the detected item information.

[0081] Furthermore, based on the item management information, the selection unit 3012 selects item information that specifies an item with a gripping success rate lower than a reference value as item information that specifies one or more items for which images are to be acquired. The selection unit 3012 compares the reference value with the gripping success rate included in the item management information, and selects item information that specifies an item with a gripping success rate lower than the reference value. The gripping success rate is analyzed based on the gripping history that is periodically collected by the WES30. For example, the WMS1 periodically collects the gripping history that is periodically collected by the WES30, analyzes the gripping success rate based on the gripping location history, and reflects it in the item management information. Alternatively, the WES30 may analyze the gripping success rate based on the gripping location history.

[0082] [Second item image acquisition process] FIG. 7 is a sequence diagram illustrating an example of the second article image acquisition process according to the embodiment. The second article image acquisition process will be described mainly with respect to the differences from the first article image acquisition process, and a description of the common parts will be omitted as appropriate.

[0083] In the WES 30, the information acquisition interface 303 receives the goods management information such as a shipping list transmitted from the WMS 1 (ST201). The input unit 3011 inputs the goods management information received by the information acquisition interface 303.

[0084] The selection unit 3012 selects the item retrieval process or the item image acquisition process according to the time period. For example, when the current time is included in the second time period (e.g., 23:00-5:00), the selection unit 3012 selects the second item image acquisition process.

[0085] When the item image acquisition process is selected, the selection unit 3012 selects item information that specifies one or more items for which images are to be acquired based on the item management information, and sets, for example, flag 2 (ST202). Details of the item information selection process are as described above. Flag 2 indicates that the item image acquisition process for acquiring item images has been selected. The generation unit 3013 generates an outgoing instruction for one or more items specified by the item information selected as the image acquisition target, or an outgoing instruction for a case containing the specified one or more items, and generates a photographing instruction corresponding to the outgoing instruction (ST203B). The generation unit 3013 does not generate an outgoing quantity because there is no actual outgoing.

[0086] The communication interface 304 outputs a delivery instruction to the transport control device 40 (ST204).

[0087] In the transport control device 40, the processor 401 generates a transport control signal for controlling the transport of the transport device 41 based on the leaving instruction. The communication interface 403 transmits the generated transport control signal to the transport device 41 (ST205).

[0088] Based on the transport control signal, the transport device 41 moves to a storage location of one or more items specified by the item information selected as items to be shipped out, and transports, for example, a case or the like containing the specified one or more items from the storage location to a gripping position (ST206).

[0089] The processor 401 detects that the transport device 41 has arrived at the gripping position, and the communication interface 403 outputs an arrival notification to the WES 30 (ST207).

[0090] In the WES 30, the communication interface 304 receives the arrival notification, the processor 301 outputs an image capture instruction based on the reception of the arrival notification, and the communication interface 304 outputs the image capture instruction to the picking control device 50 (ST208B).

[0091] In the picking control device 50, the communication interface 503 receives the image capture instruction, and the processor 501 outputs the image capture instruction to the recognition camera 60 based on the received image capture instruction (ST209).

[0092] The recognition camera 60 photographs the case transported to the gripping position based on the photographing instruction (ST210), and outputs the photographed image data to the picking control device 50 (ST211).

[0093] In the picking control device 50, when the captured image data is sent from the recognition camera 60, the communication interface 503 transmits an image acquisition notification to the WES 30 (ST212). Note that the image acquisition notification is not essential. Furthermore, the communication interface 503 uploads the captured image data sent from the recognition camera 60 to the WES 30 (ST217).

[0094] In the WES 30, the communication interface 304 receives the captured image data, and the processor 301 updates the recognition dictionary stored in the memory 302 (ST218). In addition, the communication interface 304 transmits the updated recognition dictionary or dictionary update data based on the captured image to the picking control device 50 (ST219).

[0095] In the picking control device 50, the communication interface 503 receives the updated recognition dictionary, and the processor 501 replaces the recognition dictionary in the memory area 5021 with the updated recognition dictionary. Alternatively, the processor 501 updates the recognition dictionary in the memory area 5021 based on the dictionary update data (ST220).

[0096] The transport device 41 transports the case transported to the gripping position to its original position based on the transport control signal, and returns the case to its original position. For example, the WES 30 is triggered by an image acquisition notification from the picking control device 50 to output a transport control signal and return the case to its original position.

[0097] In the above-described article retrieval process, it has not been described that image data captured by the recognition camera 60 of an article to be retrieved is used to update the recognition dictionary, but this captured image data may be used to update the recognition dictionary. For example, in the picking control device 50, when captured image data is sent from the recognition camera 60, the communication interface 503 transmits an image acquisition notification to the WES 30, and further uploads the captured image data to the WES 30. In addition, such an article retrieval process for uploading captured image data may be used in combination with the first or second article image acquisition process described above.

[0098] According to the above-described embodiments, it is possible to provide an article management device, an article management method, an article management program, and an article management system that are excellent in collecting learning data for article recognition.

[0099] The WES30 according to the embodiment executes a first article image acquisition process to acquire article images for a recognition dictionary in the operating environment of the picking device 51. The difference between the illumination conditions under which the article images for the recognition dictionary were acquired and the illumination conditions under which the article images were acquired in the recognition process is reduced, thereby improving the recognition accuracy.

[0100] The WES30 according to the embodiment executes the item warehousing process and the item image acquisition process (first or second item image acquisition process) at different times. For example, the item warehousing process is executed during the day, and the item image acquisition process is executed at night. It is possible to acquire item images for the recognition dictionary without reducing the efficiency of the item warehousing process. It is also easy to periodically add images of new items that are added each season.

[0101] In the first article image acquisition process, the API of the article retrieval process can be reused, which is advantageous in various customizations of the system. In the second article image acquisition process, retrieval cancellation is unnecessary, which can reduce the actual processing load.

[0102] In addition, the program according to the present embodiment may be transferred in a state stored in an electronic device such as the WES30, or may be transferred in a state not stored in the electronic device. In the latter case, the program may be transferred via a network, or may be transferred in a state stored in a storage medium. The storage medium is a non-transitory tangible medium. The storage medium is a computer-readable medium. The storage medium may be an optical disk, a memory card, or any other medium capable of storing a program and being readable by a computer, and the form of the storage medium is not important. The electronic device downloads the program transferred (provided) via a network and installs it in memory, or reads the program from the storage medium and installs it in memory.

[0103] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0104] 1. WMS 2. Warehouse processing system 30…W.E.S. 40...Transport control device 41...Transportation device 50...Picking control device 51...Picking device 60…Recognition camera 301...Processor 302…Memory 303: Information acquisition interface 304…Communication interface 401...Processor 402…Memory 403…Communication interface 501...Processor 502…Memory 503…Communication interface 3011... Input section 3012…Selection section 3013...Generation section 3014...Output section 5011...Recognition processing unit 5021...Storage area

Claims

1. an information acquisition interface for acquiring item management information; A processor that selects, based on the item management information, item information that specifies an item to be imaged, and generates a warehousing instruction for the item specified by the item information selected as the image acquisition target; a communication interface that communicates with a transport control device that controls a transport device that transports an item based on a delivery instruction and a picking control device that controls a picking device that recognizes and picks an item based on a photographed image acquired from the item transported by the transport device, and outputs at least a delivery instruction to the transport control device and receives the photographed image acquired in response to the delivery instruction; An article management device comprising:

2. The article management device according to claim 1 , wherein the communication interface transmits dictionary update data based on the captured image in order to update an article recognition dictionary in the picking control device.

3. The article management device according to claim 1 , wherein the processor selects article information specifying an article that is being stocked for the first time as the article information specifying the article to be imaged.

4. The article management device according to claim 1 , wherein the processor selects article information specifying an article whose image has not been registered as the article information specifying the article to be imaged.

5. The article management device according to claim 1 , wherein the processor selects article information specifying an article having a picking success rate lower than a reference value as the article information specifying the article to be imaged.

6. The processor selects first item information that specifies a first item to be imaged based on the item management information, and generates a shipping instruction and a shipping cancellation for the first item; The item management device of claim 1, wherein the communication interface outputs an instruction to release the first item in response to the selection of the first item information, and outputs a cancellation of release of the first item to the picking control device based on the acquisition of a first photographed image of the first item.

7. The processor generates a picking instruction for the first item; The item management device of claim 6, wherein the communication interface outputs an instruction to unload the first item to the transport control device, outputs an instruction to pick the first item to the picking device which recognizes and picks an image acquired in response to the picking instruction, and outputs a cancellation of unloading the first item to the picking device based on the acquisition of a first image of the first item.

8. The processor selects, during a predetermined time period, first item information that specifies a first item to be imaged based on the item management information, and generates an outbound instruction and an outbound cancellation for the first item, and during a time period different from the predetermined time period, selects, during a time period different from the predetermined time period, second item information that specifies a second item to be outbound based on the item management information, and generates an outbound instruction for the second item; 2. The item management device of claim 1, wherein the communication interface outputs an out-of-stock instruction and an out-of-stock cancellation for the first item in response to the selection of the first item information, and outputs an out-of-stock instruction for the second item in response to the selection of the second item information.

9. Acquire inventory management information, Selecting article information that specifies an article to be imaged based on the article management information; Generate a warehousing instruction for an item designated by the item information selected as the image acquisition target; An item management method that communicates with a transport control device that controls a transport device that transports items based on an outbound instruction, and a picking control device that controls a picking device that recognizes and picks items based on photographed images acquired from items transported by the transport device, outputs an outbound instruction to at least the transport control device, and receives the photographed images acquired in response to the outbound instruction.

10. On the computer, A procedure for obtaining item management information; A step of selecting item information that specifies an item to be imaged based on the item management information; A step of generating a warehousing instruction for an item specified by the item information selected as the image acquisition target; An item management program for communicating with a transport control device that controls a transport device that transports items based on an outbound instruction, and a picking control device that controls a picking device that recognizes and picks items based on photographed images acquired from items transported by the transport device, and for outputting at least an outbound instruction to the transport control device and receiving the photographed images acquired in response to the outbound instruction.

11. An item management system including an item management device that outputs a delivery instruction, a transport control device that controls a transport device that transports items based on the delivery instruction, and a picking control device that controls a picking device that picks up items transported by the transport device, The article management device includes: an information acquisition interface for acquiring item management information; A first processor selects item information that specifies an item to be imaged based on the item management information, and generates a shipping instruction for the item specified by the item information selected as the image acquisition target; a first communication interface that communicates with a transport control device that controls the transport device and a picking control device that controls a picking device that recognizes and picks up an item based on a photographed image acquired from an item transported by the transport device, outputs a delivery instruction to at least the transport control device, and receives the photographed image acquired in response to the delivery instruction; Equipped with The transport control device includes: A second processor that generates an item transport control signal based on an item retrieval instruction output from the item management device; a second communication interface that receives an instruction to take out an item from the item management device and transmits the transport control signal to the transport device; The picking control device includes: a third processor that generates an image capturing instruction for capturing an image of an item transported by the transport device based on an item retrieval instruction output from the item management device, and generates an item picking control signal for picking the item based on an item image acquired from the item by the image capturing; and a third communication interface that receives an item removal instruction output from the item management device and transmits the photography instruction and the picking control signal to the picking device.

Images