Article management device, article management method, article management program, computer-readable storage medium, and article management system
The article management system improves article recognition accuracy and data collection efficiency by integrating a warehouse management system with conveyance and picking devices to adapt to varying illumination conditions and efficiently manage new items, addressing the challenges of recognition accuracy and data collection in logistics automation.
Patent Information
- Application Number
- PCT/JP2024/020378
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-06-09
- Filing Date
- 2024-06-04
- Publication Date
- 2024-12-12
AI Technical Summary
In logistics automation, machine learning-based article recognition systems face decreased recognition accuracy due to varying illumination conditions and the need for extensive learning data collection, especially when new items are introduced, which increases the burden on data collection processes.
An article management system that includes a warehouse management system, a warehouse execution system, and devices for conveyance and picking, which acquires article management information, generates delivery instructions, and updates recognition dictionaries using images from different illumination conditions, allowing for improved recognition accuracy and efficient data collection across varying time zones.
The system enhances article recognition accuracy by adapting to different illumination conditions and simplifies the collection of learning data, reducing the burden of data collection and enabling efficient handling of new items, while maintaining logistics processing efficiency.
Smart Images

Figure JP2024020378_12122024_PF_FP_ABST
Abstract
Description
Item management device, item management method, item management program, computer-readable storage medium, and item management system
[0001] An embodiment of the present invention relates to an article management device, an article management method, an article management program, a computer-readable storage medium, and an article management system.
[0002] In recent years, the volume of various products (hereinafter referred to as "goods") handled at distribution centers and other locations in logistics has increased, and automation of goods processing has been promoted. For example, in a warehouse, various automated devices work together, and each device delivers goods or cases or equipment containing goods according to instructions from a higher-level device. Known automated devices include conveying devices and picking devices, and the retrieval work of goods stored in a warehouse is automated by these devices.
[0003] For example, the host device issues an instruction to retrieve an item based on a retrieval list. The transport device transports the item based on the retrieval instruction. The picking device photographs the transported item, recognizes the item from the photographed image using machine learning, and picks the recognized item.
[0004] Japanese Patent Application Publication No. 2022-181623
[0005] As described above, when machine learning is used for item recognition processing, it is necessary to create a recognition dictionary based on the collection of training data, etc. For example, the recognition dictionary is created in advance by capturing images of items to be picked in a training data collection area to collect training data. In this case, if the lighting conditions of the picking area where the picking device is actually installed differ from the lighting conditions of the training data collection area, recognition accuracy may decrease, and improvements are desired. Furthermore, in the logistics industry, new items are sometimes added each season, and there is a demand for reducing the burden of collecting training data, etc.
[0006] An object of the present invention is to provide an article management device, an article management method, an article management program, a computer-readable storage medium, and an article management system that are excellent at collecting learning data for article recognition.
[0007] An item management device according to an embodiment includes an information acquisition interface, a processor, and a communication interface. The information acquisition interface acquires item management information. The processor selects item information specifying items for which images are to be acquired based on the item management information, and generates a shipping instruction for the items specified by the item information selected as the items for which images are to be acquired. The communication interface communicates with a transport control device that controls a transport device that transports items based on the shipping instruction, and a picking control device that controls a picking device that recognizes and picks the items based on photographed images acquired from the items transported by the transport device, and outputs a shipping instruction to at least the transport control device and receives the photographed images acquired in response to the shipping instruction.
[0008] FIG. 1 is a conceptual diagram showing a logistics system as an example of an item processing system according to an embodiment. FIG. 2 is a block diagram showing an example of a WES according to an embodiment. FIG. 3 is a block diagram showing an example of a transport control device according to an embodiment. FIG. 4 is a block diagram showing an example of a picking control device according to an embodiment. FIG. 5 is a sequence diagram showing an example of an item retrieval process according to an embodiment. FIG. 6 is a sequence diagram showing an example of a first item image acquisition process according to an embodiment. FIG. 7 is a sequence diagram showing an example of a second item image acquisition process according to an embodiment. Embodiment
[0009] Each embodiment will be described below with reference to the drawings. [Configuration] Fig. 1 is a conceptual diagram showing a logistics system, which is an example of an item processing system according to an embodiment. As shown in Fig. 1, the logistics system S includes a warehouse management system (WMS) 1 and a warehouse processing system 2.
[0010] The warehouse processing system 2 includes a warehouse execution system (WES) 30, a transport control device 40, a transport device (transport robot) 41, a picking control device 50, a picking device (picking robot) 51, and a recognition camera 60. For example, one transport control device 40 controls one or more transport devices 41, and one picking control device 50 controls one or more picking devices 51.
[0011] The WMS 1 can be configured with one or more computers, i.e., a processor, memory, interface, etc. The processor may be a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), etc. The WMS 1 receives item management information such as an order list from a higher-level server and transmits it to the WES 30.
[0012] An order list is a list of items stored on warehouse shelves or the like, and specifies one or more items. For example, an order list includes item information, ordering information, delivery information, etc. Item information includes the number of items, item identification information, item names, etc. Order information includes the order date and time and the orderer, etc. Delivery information includes the delivery destination, delivery date and time, recipient, etc. Furthermore, the order list includes a flag indicating first inventory, a flag indicating an image not registered or not compatible with the recognition dictionary, and the item grasping success rate.
[0013] The WES 30 (server) is an example of an item management device and is composed of one or more general-purpose computers. The WES 30 communicates with the WMS 1, as well as with the transport control device 40 and the picking control device 50. The WES 30 receives item management information, such as order lists, sent from the WMS 1 and generates and outputs various instructions based on the item management information. The WES 30 generates an outbound instruction to have the transport device 41 transport items and outputs the outbound instruction to the transport control device 40. The WES 30 also generates a picking instruction to have the picking device 51 pick items transported by the transport device 41 and outputs the picking instruction to the picking control device 50.
[0014] The transport control device 40 is composed of one or more general-purpose computers. The transport control device 40 communicates with the WES 30 and also with the transport device 41. The transport control device 40 receives a shipping instruction transmitted from the WES 30, generates a transport control signal for controlling the transport device 41 based on the shipping instruction, transmits the transport control signal to the transport device 41, and controls the transport of items by the transport device 41. Note that instead of a picking instruction from the WES 30, the transport control device 40 may generate a picking instruction for causing a picking device 51 to pick an item transported by the transport device 41 based on the shipping instruction transmitted from the WES 30, and output the picking instruction to the picking control device 50.
[0015] The transport device 41 communicates with the transport control device 40, receives a transport control signal transmitted from the transport control device 40, and transports an item or a case containing the item from a storage position to a destination position based on the transport control signal. The transport device 41 is a transport robot of any type, such as an Auto Guided Forklift (AGF) that retrieves a case containing an item from a high-rise location and transports it, an Automated Guided Vehicle (AGV) that transports shelves or the like that store cases containing items, or an Autonomous Case-handling Robot (ACR) that transports cases containing items.
[0016] The picking control device 50 is composed of one or more general-purpose computers. The picking control device 50 communicates with the WES 30 and also with the picking device 51. The picking control device 50 receives picking instructions transmitted from the WES 30 or the transport control device 40, transmits picking control signals to the picking device 51, and controls the picking of items by the picking device 51. For example, the picking control device 50 recognizes item images contained in image data captured by the recognition camera 60 based on an accumulated recognition dictionary, and generates a picking control signal that controls the picking device 51 based on the recognition result.
[0017] The picking device 51 communicates with the picking control device 50, receives a picking control signal transmitted from the picking control device 50, and grips an item and moves it from the gripping position to the destination position based on the picking control signal. The picking control signal may be a signal that simply instructs the start of picking, in which case the picking device 51 grips an item that arrives at the gripping position or an item contained in a case based on a detection signal of an item or case arriving at the gripping position and moves it from the gripping position to the destination position.
[0018] The recognition camera 60 is an RGB camera, a 3D camera, or a combination thereof that captures images of the gripping position and outputs the captured image data to the picking control device 50. The picking control device 50 uploads the captured image data to the WES 30. The picking control device 50 also recognizes the item images included in the images captured by the recognition camera 60 based on an accumulated recognition dictionary, and generates a picking control signal based on the recognition result. The picking control device 50 also periodically receives dictionary update data generated based on the uploaded captured image data from the WES 30 and updates the recognition dictionary.
[0019] 2 is a block diagram showing an example of a WES according to an embodiment. As shown in FIG. 2, the WES 30 includes a processor 301, a memory 302, an information acquisition interface 303, and a communication interface 304. The information acquisition interface 303 and the communication interface 304 may be a common interface or may be independent interfaces.
[0020] The processor 301 is a CPU, an MPU, a DSP, or the like. The processor 301 operates based on a program stored in the memory 302. The processor 301 includes an input unit 3011, a selection unit 3012, a generation unit 3013, and an output unit 3014. The processor 301 executes the program stored in the memory 302 to realize the functions of each unit.
[0021] The input unit 3011 inputs an order list such as item management information. The selection unit 3012 selects item information specifying items for which images are to be acquired based on the item management information. The generation unit 3013 generates a shipping instruction for the items specified by the item information selected as items for which images are to be acquired. The output unit 3014 outputs the generated shipping instruction.
[0022] The memory 302 is a non-volatile memory and stores an operating program for the processor 301. The memory 302 also stores an inventory management database, a stock history, a shipping history, etc. The inventory management database is data indicating the storage location of each item, and includes warehouse map data (coordinate data), location identification information associated with the map data, storage location information associated with the location identification information, and item information associated with the storage location information. The stock history includes the stocking date and time and item information of the stocked items. The shipping history includes the shipping date and time and item information of the shipped items.
[0023] The information acquisition interface 303 acquires an order list such as product management information sent from the WMS 1 .
[0024] The communication interface 304 communicates with a transport control device 40 that controls a transport device 41 that transports items, and a picking control device 50 that controls a picking device 51 that recognizes and picks items based on captured images of the items transported by the transport device 41. The communication interface 304 outputs a shipping instruction to the transport control device 40.
[0025] 3 is a block diagram illustrating an example of a transport control device according to an embodiment. As shown in FIG. 3, the transport control device 40 includes a processor 401, a memory 402, a communication interface 403, and the like.
[0026] The processor 401 is a CPU, an MPU, a DSP, or the like. The processor 401 operates based on a program stored in the memory 402. The processor 401 executes the program stored in the memory 402 to generate a control signal for controlling the conveying device 41 based on a delivery instruction from the WES 30.
[0027] The memory 402 is a non-volatile memory and stores the operating program of the processor 401 and the like.
[0028] The communication interface 403 receives a shipping instruction from the WES 30 and transmits a control signal to the transport device 41 .
[0029] 4 is a block diagram illustrating an example of a picking control device according to an embodiment. As shown in FIG. 4, the picking control device 50 includes a processor 501, a memory 502, a communication interface 503, and the like.
[0030] The processor 501 is a CPU, an MPU, a DSP, or the like. The processor 501 operates based on a program stored in the memory 502. The processor 501 executes the program stored in the memory 502 to generate a control signal for controlling the picking device 51 based on a picking instruction from the WES 30 or the transport control device 40. The processor 501 also includes a recognition processing unit 5011. The recognition processing unit 5011 recognizes an item image included in an image captured by the recognition camera 60 based on a recognition dictionary stored in the memory 502.
[0031] The memory 502 is a non-volatile memory and stores the operation program of the processor 501. The memory 502 also includes a storage area 5021 that stores a recognition dictionary.
[0032] The communication interface 503 receives a picking instruction from the WES 30 or the transport control device 40 and transmits a control signal to the picking device 51 .
[0033] Below, the item retrieval processing will be explained with reference to the sequence diagram shown in Figure 5, the first item image acquisition processing will be explained with reference to the sequence diagram shown in Figure 6, and the second item image acquisition processing will be explained with reference to the sequence diagram shown in Figure 7.
[0034] The item retrieval process is a process of outputting a retrieval instruction and a picking instruction and retrieving the target item. The first item image acquisition process outputs a retrieval instruction and a picking instruction, but is a dummy retrieval process in which the target item is not actually retrieved but an image of the target item is acquired. In the first item image acquisition process, by utilizing the instructions of the item retrieval process, it is possible to reuse the API (Application Interface) of the item retrieval process. The second item image acquisition process is a process for photographing the item, outputting a retrieval instruction and a photographing instruction for photographing and acquiring an image of the target item.
[0035] 5 is a sequence diagram showing an example of an item retrieval process according to an embodiment. In the WES 30, the information acquisition interface 303 receives item management information such as an retrieval list transmitted from the WMS 1 (ST101). The input unit 3011 inputs the item management information received by the information acquisition interface 303.
[0036] The selection unit 3012 selects the item retrieval process or the item image acquisition process depending on the time period. For example, the selection unit 3012 selects the item retrieval process when the current time is within a first time period (e.g., 6:00-22:00), and selects the first or second item image acquisition process when the current time is within a second time period (e.g., 23:00-5:00) different from the first time period.
[0037] When the item outgoing process is selected, the selection unit 3012 selects item information specifying one or more items to be outgoing based on the item management information, and sets, for example, flag 0 (ST102). Flag 0 indicates that the item outgoing process has been selected. The generation unit 3013 generates an outgoing instruction for one or more items specified by the item information selected as the outgoing target, or an outgoing instruction for a case containing the specified one or more items, and generates a picking instruction corresponding to the outgoing instruction (ST103). The generation unit 3013 generates an outgoing quantity according to the outgoing instruction. The memory 302 stores the outgoing quantity according to the outgoing instruction.
[0038] Here, the retrieval of items contained in a case will be described, but items not contained in a case may also be retrieved. The memory 302 stores an inventory management database indicating the storage location of each item. The generation unit 3013 detects the storage location of one or more items specified by the item information selected as the retrieval target based on the inventory management database. The generation unit 3013 generates a retrieval instruction for retrieving the case containing the one or more items specified by the item information from the detected storage location.
[0039] The output unit 3014 outputs the generated shipping instruction. The communication interface 304 outputs the shipping instruction to the transport control device 40 (ST104).
[0040] In the transport control device 40, the communication interface 403 receives the delivery instruction, and the processor 401 generates a transport control signal for controlling the transport of the transport device 41 based on the delivery instruction. The communication interface 403 transmits the generated transport control signal to the transport device 41 (ST105).
[0041] The conveying device 41 receives the conveying control signal. Based on the conveying control signal, the conveying device 41 moves to a storage position of one or more items specified by the item information selected as items to be removed from the warehouse, and conveys, for example, a case or the like containing the one or more specified items from the storage position to a gripping position (ST106).
[0042] The processor 401 of the transport control device 40 communicates with the transport device 41 via the communication interface 403 and monitors the traveling position of the transport device 41 in real time. The processor 401 detects that the transport device 41 has arrived at the gripping position, and the communication interface 403 outputs an arrival notification to the WES 30 (ST107). Note that the arrival notification is not required. The difference between the case where an arrival notification is given and the case where it is not given will be explained in detail later.
[0043] In the WES 30, the communication interface 304 receives the arrival notification, the processor 301 outputs a picking instruction based on the received arrival notification, and the communication interface 304 outputs the picking instruction to the picking control device 50, which recognizes the photographed image acquired in response to the picking instruction and performs picking (ST108). Note that instead of the WES 30, the transport control device 40 may generate a picking instruction corresponding to the shipping instruction transmitted from the WES 30 and output the picking instruction to the picking control device 50.
[0044] In the picking control device 50, the communication interface 503 receives the picking instruction, and the processor 501 outputs a photographing instruction to the recognition camera 60 based on the received picking instruction (ST109).
[0045] The recognition camera 60 photographs the case transported to the gripping position based on the photographing instruction (ST110), and outputs the photographed image data to the picking control device 50 (ST111).
[0046] In the picking control device 50, when the captured image data is sent from the recognition camera 60, the communication interface 502 transmits an image acquisition notification to the WES 30 (ST112).
[0047] The WES 30 does not execute the delivery cancellation described below based on the set flag 0. Delivery cancellation will be described in detail later.
[0048] In the picking control device 50, the recognition processing unit 5011 of the processor 501 recognizes the item included in the photographed image data sent from the recognition camera 60 based on the recognition dictionary in the memory 502 (storage area 5021) (ST113).
[0049] The recognition camera 60 may be a camera that captures video and outputs captured video data for a certain period of time. In this case, the recognition processing unit 5011 detects one or more frames from the captured video data output from the recognition camera 60 based on the timing of receiving a picking instruction, and recognizes the item images included in the detected frames based on the recognition dictionary in the memory 502 (storage area 5021).
[0050] The processor 501 generates a picking control signal based on the result of the recognition of the item, and the communication interface 503 outputs the picking control signal to the picking device 51 (ST114). For example, the picking control signal includes a signal for grasping (by suction or clamping) the item in a method specified for each item.
[0051] The picking device 51 picks up the item at the picking position based on the picking control signal and moves it to the target position (ST115).
[0052] The processor 501 communicates with the picking device 51 via the communication interface 503 and monitors the operation of the picking device 51 in real time. For example, the processor 501 detects whether a picking attempt was successful or unsuccessful based on information output from the picking device 51, and further detects the time and number of attempts required for successful picking, and stores these in the memory 502 as a picking history. The picking history includes the date and time, the item identification information of the target item, and the picking result (pickup success or picking failure). When the processor 501 recognizes that an item has been unloaded based on the information output from the picking device 51, it outputs a unloading completion notification to the WES 30 via the communication interface 503 (ST116). Furthermore, the processor 501 periodically outputs the picking history to the WES 30 via the communication interface 503.
[0053] In the WES 30, the communication interface 304 receives the shipping completion notice, and the processor 301 updates the inventory management database stored in the memory 302 (ST117). For example, the processor 301 compares the shipping quantity of the specified item stored in the memory 302 with the shipping quantity of the specified item indicated in the shipping completion notice, and if they match, subtracts the shipping completed quantity of the specified item indicated in the shipping completion notice from the inventory quantity of the specified item.
[0054] After the items are grasped from the case that has been transported to the gripping position based on the transport control signal, the transport device 41 transports the case that has been transported to the gripping position back to its original position and returns the case to its original position. For example, the WES 30 may be triggered by an image acquisition notification from the picking control device 50 to output a transport control signal and return the case to its original position, or may be triggered by a picking completion notification from the picking control device 50 to output a transport control signal and return the case to its original position.
[0055] Here, we will provide additional information about the arrival notification in ST107. As mentioned above, the arrival notification is not essential.
[0056] The case where an arrival notification is received will be described. This is effective when the WES 30 outputs multiple outbound instructions to the transport control device 40. Upon receiving the arrival notification corresponding to each outbound instruction, the WES 30 outputs a picking instruction to the picking control device 50, triggered by the arrival notification corresponding to each outbound instruction. This ensures consistency between the items for which the outbound instruction is issued by the WES 30 and the items to be picked by the picking device 51. Note that when the transport control device 40 outputs a picking instruction instead of the WES 30, it detects the arrival of the transport device 41 at the gripping position and outputs a picking instruction based on this arrival detection.
[0057] The following describes the case where there is no arrival notification. To ensure that the items for which the WES 30 issues an outbound instruction are consistent with the items to be picked by the picking device 51, the WES 30 outputs outbound instructions to the transport control device 40 one by one, and the picking control device 50 controls the picking of the items based on the outbound instructions. This ensures that the items for which the WES 30 issues an outbound instruction are consistent with the items to be picked by the picking device 51, even without an arrival notification. Note that when the transport control device 40 outputs a picking instruction instead of the WES 30, it detects the arrival of the transport device 41 at the gripping position and outputs a picking instruction based on this arrival detection, as described above.
[0058] As mentioned above, if there is an arrival notification, multiple departure instructions can be output. If there is no arrival notification, sending and receiving of arrival notification can be omitted.
[0059] 6 is a sequence diagram showing an example of the first item image acquisition process according to the embodiment. In the WES 30, the information acquisition interface 303 receives item management information such as an inventory list transmitted from the WMS 1 (ST201). The input unit 3011 inputs the item management information received by the information acquisition interface 303.
[0060] The selection unit 3012 selects the item retrieval process or the item image acquisition process depending on the time period. For example, if the current time is included in the second time period (e.g., 23:00-5:00), the selection unit 3012 selects the first item image acquisition process.
[0061] When the item image acquisition process is selected, the selection unit 3012 selects item information specifying one or more items for which images are to be acquired based on the item management information, and sets, for example, flag 1 (ST202). Flag 1 indicates that the item image acquisition process using the dummy outbound process has been selected. The generation unit 3013 generates an outbound instruction for one or more items specified by the item information selected as the image acquisition target, or an outbound instruction for a case containing the specified one or more items, and generates a picking instruction and a cancellation instruction corresponding to the outbound instruction (ST203A). The generation unit 3013 generates an outbound quantity corresponding to the outbound instruction. The memory 302 stores the outbound quantity corresponding to the outbound instruction.
[0062] The memory 302 stores an inventory management database indicating the storage location of each item. The generation unit 3013 detects the storage location of one or more items specified by the item information selected as the image acquisition target based on the inventory management database. The generation unit 3013 generates a shipping instruction for shipping a case containing one or more items specified by the item information from the detected storage location.
[0063] The output unit 3014 outputs the generated shipping instruction. The communication interface 304 outputs the shipping instruction to the transport control device 40 (ST204).
[0064] In the transport control device 40, the communication interface 403 receives the delivery instruction, and the processor 401 generates a transport control signal for controlling the transport of the transport device 41 based on the delivery instruction. The communication interface 403 transmits the generated transport control signal to the transport device 41 (ST205).
[0065] The conveying device 41 receives the conveying control signal and moves to a storage location of one or more items designated by the item information selected as items to be removed from the warehouse based on the conveying control signal, and conveys, for example, a case or the like containing the designated one or more items from the storage location to a gripping location (ST206).
[0066] The processor 401 detects that the transport device 41 has arrived at the gripping position, and the communication interface 403 outputs an arrival notification to the WES 30 (ST207). Note that the arrival notification is not required. The difference between the case where an arrival notification is given and the case where it is not given is as explained above.
[0067] In the WES 30, the communication interface 304 receives the arrival notification, the processor 301 outputs a picking instruction based on the received arrival notification, and the communication interface 304 outputs the picking instruction to the picking control device 50, which recognizes the captured image acquired in response to the picking instruction and performs picking (ST208A). Note that instead of the WES 30, the transport control device 40 may generate a picking instruction corresponding to the shipping instruction transmitted from the WES 30 and output the picking instruction to the picking control device 50.
[0068] In the picking control device 50, the communication interface 503 receives the picking instruction, and the processor 501 outputs a photographing instruction to the recognition camera 60 based on the received picking instruction (ST209).
[0069] The recognition camera 60 photographs the case transported to the gripping position based on the photographing instruction (ST210), and outputs the photographed image data to the picking control device 50 (ST211).
[0070] In the picking control device 50, when the captured image data is sent from the recognition camera 60, the communication interface 503 transmits an image acquisition notification to the WES 30 (ST212).
[0071] The WES 30 executes a shipping cancellation based on the set flag 1. For example, the generation unit 3013 generates a shipping cancellation. The processor 301 deletes the shipping quantity of the specified item stored in the memory 302 based on the shipping cancellation. Alternatively, the processor 301 may set the completed shipping quantity to 0 based on the shipping cancellation and update the inventory management database stored in the memory 302. In other words, the quantity in the inventory management database does not change due to the dummy shipping process.
[0072] The output unit 3014 also outputs a release cancellation based on the acquisition of the photographed image. That is, the output unit 3014 outputs a release cancellation based on the reception of the image acquisition notification. The communication interface 304 transmits the release cancellation to the picking control device 50 (ST216).
[0073] In the picking control device 50, the communication interface 503 receives the shipping cancellation, cancels the item recognition and picking process based on the picking instruction, and uploads the captured image data sent from the recognition camera 60 to the WES 30 (ST217).
[0074] The recognition camera 60 may be a camera that captures video and outputs captured video data for a certain period of time. In this case, the recognition processing unit 5011 detects one or more frames from the captured video data output from the recognition camera 60 based on the timing at which a picking instruction is received, and uploads the captured image data of the detected frames to the WES 30.
[0075] In the WES 30, the communication interface 304 receives the captured image data, and the processor 301 updates the recognition dictionary stored in the memory 302 (ST218). Note that a person in charge may also be responsible for some processing, such as area designation of the captured image data. The communication interface 304 also transmits the updated recognition dictionary or dictionary update data based on the captured image to the picking control device 50 (ST219).
[0076] In the picking control device 50, the communication interface 503 receives the updated recognition dictionary, and the processor 501 replaces the recognition dictionary in the memory area 5021 with the updated recognition dictionary. Alternatively, the processor 501 updates the recognition dictionary in the memory area 5021 based on the dictionary update data (ST220).
[0077] The transport device 41 transports the case that has been transported to the gripping position back to its original position based on the transport control signal, and returns the case to its original position. For example, the WES 30 is triggered by an image acquisition notification from the picking control device 50 to output a transport control signal and return the case to its original position.
[0078] Here, the selection of the article information of the image acquisition target in ST202 will be described. The selection unit 3012 selects the article information of the image acquisition target based on the following conditions.
[0079] For example, the selection unit 3012 selects item information specifying an item that has been initially stocked as item information specifying one or more items for which images are to be acquired based on the item management information. The selection unit 3012 detects item information specifying an item that has been initially stocked based on a flag indicating initial stocking included in the item management information, and selects the detected item information. Alternatively, the selection unit 3012 detects item information specifying an item that has been initially stocked based on the stocking history (item identification information with a stocking record) stored in the memory 302, and selects the detected item information.
[0080] Furthermore, the selection unit 3012 selects, based on the item management information, item information specifying one or more items for which images are to be acquired, item information specifying an item for which an image is not registered or for which a recognition dictionary is not supported. The selection unit 3012 detects item information specifying an item for which an image is not registered or for which a recognition dictionary is not supported, based on a flag or the like included in the item management information indicating that an image is not registered or that a recognition dictionary is not supported, and selects the detected item information.
[0081] Furthermore, based on the item management information, the selection unit 3012 selects item information specifying one or more items for which images are to be acquired, which specifies items with a gripping success rate lower than a reference value. The selection unit 3012 compares the reference value with the gripping success rate included in the item management information, and selects item information specifying items with a gripping success rate lower than the reference value. The gripping success rate is analyzed based on the gripping history periodically collected by the WES 30. For example, the WMS 1 periodically collects the gripping history collected by the WES 30, analyzes the gripping success rate based on the gripping location history, and reflects the analysis result in the item management information. Alternatively, the WES 30 may analyze the gripping success rate based on the gripping location history.
[0082] 7 is a sequence diagram illustrating an example of a second article image acquisition process according to an embodiment. The second article image acquisition process will be described mainly with respect to differences from the first article image acquisition process, and descriptions of common parts will be omitted as appropriate.
[0083] In the WES 30, the information acquisition interface 303 receives the goods management information such as the shipping list transmitted from the WMS 1 (ST201). The input unit 3011 inputs the goods management information received by the information acquisition interface 303.
[0084] The selection unit 3012 selects the item retrieval process or the item image acquisition process depending on the time period. For example, if the current time is included in the second time period (e.g., 23:00-5:00), the selection unit 3012 selects the second item image acquisition process.
[0085] When the item image acquisition process is selected, the selection unit 3012 selects item information that specifies one or more items to be imaged based on the item management information, and sets, for example, flag 2 (ST202). Details of the item information selection process are as described above. Flag 2 indicates that the item image acquisition process for acquiring item images has been selected. The generation unit 3013 generates an outgoing instruction for one or more items specified by the item information selected as the image acquisition target, or an outgoing instruction for a case containing the specified one or more items, and generates a photographing instruction corresponding to the outgoing instruction (ST203B). The generation unit 3013 does not generate an outgoing quantity because there is no actual outgoing.
[0086] The communication interface 304 outputs a shipping instruction to the transport control device 40 (ST204).
[0087] In the transport control device 40, the processor 401 generates a transport control signal for controlling transport of the transport device 41 based on the retrieval instruction. The communication interface 403 transmits the generated transport control signal to the transport device 41 (ST205).
[0088] Based on the transport control signal, the transport device 41 moves to the storage location of one or more items specified by the item information selected as the items to be shipped, and transports, for example, a case containing the specified one or more items from the storage location to a gripping position (ST206).
[0089] The processor 401 detects that the transport device 41 has arrived at the gripping position, and the communication interface 403 outputs an arrival notification to the WES 30 (ST207).
[0090] In the WES 30, the communication interface 304 receives the arrival notification, the processor 301 outputs a photography instruction based on the reception of the arrival notification, and the communication interface 304 outputs the photography instruction to the picking control device 50 (ST208B).
[0091] In the picking control device 50, the communication interface 503 receives the image capture instruction, and the processor 501 outputs the image capture instruction to the recognition camera 60 based on the received image capture instruction (ST209).
[0092] The recognition camera 60 photographs the case transported to the gripping position based on the photographing instruction (ST210), and outputs the photographed image data to the picking control device 50 (ST211).
[0093] In the picking control device 50, when the captured image data is sent from the recognition camera 60, the communication interface 503 sends an image acquisition notification to the WES 30 (ST212). Note that the image acquisition notification is not required. Furthermore, the communication interface 503 uploads the captured image data sent from the recognition camera 60 to the WES 30 (ST217).
[0094] In the WES 30, the communication interface 304 receives the captured image data, and the processor 301 updates the recognition dictionary stored in the memory 302 (ST218). The communication interface 304 also transmits the updated recognition dictionary or dictionary update data based on the captured image to the picking control device 50 (ST219).
[0095] In the picking control device 50, the communication interface 503 receives the updated recognition dictionary, and the processor 501 replaces the recognition dictionary in the memory area 5021 with the updated recognition dictionary. Alternatively, the processor 501 updates the recognition dictionary in the memory area 5021 based on the dictionary update data (ST220).
[0096] The transport device 41 transports the case that has been transported to the gripping position back to its original position based on the transport control signal, and returns the case to its original position. For example, the WES 30 is triggered by an image acquisition notification from the picking control device 50 to output a transport control signal and return the case to its original position.
[0097] In the above-described item retrieval process, although it has not been described that image data of an item to be retrieved taken by the recognition camera 60 is used to update the recognition dictionary, this captured image data may be used to update the recognition dictionary. For example, in the picking control device 50, when captured image data is sent from the recognition camera 60, the communication interface 503 transmits an image acquisition notification to the WES 30 and further uploads the captured image data to the WES 30. Furthermore, this item retrieval process for uploading captured image data may be used in combination with the first or second item image acquisition process described above.
[0098] According to the above-described embodiments, it is possible to provide an article management device, an article management method, an article management program, and an article management system that are excellent in collecting learning data for article recognition.
[0099] The WES 30 according to the embodiment executes a first item image acquisition process to acquire images of items for the recognition dictionary in the operating environment of the picking device 51. The difference between the lighting conditions under which the item images for the recognition dictionary were acquired and the lighting conditions under which the item images were acquired in the recognition process is reduced, thereby improving recognition accuracy.
[0100] The WES 30 according to the embodiment executes the item retrieval process and the item image acquisition process (first or second item image acquisition process) at different times. For example, the item retrieval process is executed during the day and the item image acquisition process is executed at night. Item images for the recognition dictionary can be acquired without reducing the efficiency of the item retrieval process. It is also easy to periodically add new item images that are added seasonally.
[0101] The first item image acquisition process allows for the reuse of the API of the item retrieval process, which is advantageous for various customizations of the system. The second item image acquisition process eliminates the need for retrieval cancellation, which reduces the actual processing load.
[0102] Furthermore, the program according to this embodiment may be transferred in a state where it is stored in an electronic device such as the WES 30, or may be transferred in a state where it is not stored in an electronic device. In the latter case, the program may be transferred via a network or in a state where it is stored in a storage medium. The storage medium is a non-transitory tangible medium. The storage medium is a computer-readable medium. The storage medium may be any medium, such as an optical disk or a memory card, that can store a program and is readable by a computer, and its form is not important. The electronic device downloads the program transferred (provided) via a network and installs it in its memory, or reads the program from the storage medium and installs it in its memory.
[0103] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims.
Claims
1. An item management device comprising: an information acquisition interface for acquiring item management information; a processor for selecting item information specifying items for which images are to be acquired based on the item management information, and generating an outbound instruction for the items specified by the item information selected as the items for which images are to be acquired; and a communication interface for communicating with a transport control device for controlling a transport device that transports items based on the outbound instruction, and a picking control device for controlling a picking device that recognizes and picks the items based on photographed images acquired from the items transported by the transport device, and for outputting at least an outbound instruction to the transport control device and receiving the photographed images acquired in response to the outbound instruction.
2. The article management device according to claim 1, wherein the communication interface transmits dictionary update data based on the captured image in order to update an article recognition dictionary in the picking control device.
3. The article management device according to claim 1, wherein the processor selects article information specifying an article that is being stocked for the first time as the article information specifying the article for which an image is to be acquired.
4. The article management device according to claim 1, wherein the processor selects article information specifying an article whose image has not been registered as the article information specifying the article whose image is to be acquired.
5. The item management device of claim 1, wherein the processor selects item information specifying an item having a picking success rate lower than a reference value as the item information specifying the item to be imaged.
6. The item management device of claim 1, wherein the processor selects first item information specifying the first item for which image acquisition is to be performed based on the item management information, and generates an out-of-stock instruction and an out-of-stock cancellation for the first item, and the communication interface outputs an out-of-stock instruction for the first item in response to the selection of the first item information, and outputs an out-of-stock cancellation for the first item to the picking control device based on the acquisition of a first photographed image of the first item.
7. The item management device of claim 6, wherein the processor generates a picking instruction for the first item, the communication interface outputs an out-of-stock instruction for the first item to the transport control device, outputs a picking instruction for the first item to the picking device which recognizes and picks the image acquired in response to the picking instruction, and outputs a cancellation of out-of-stock of the first item to the picking device based on the acquisition of a first image of the first item.
8. The item management device of claim 1, wherein the processor selects, during a specified time period, first item information specifying a first item to be imaged based on the item management information, and generates an outbound shipping instruction and an outbound cancellation for the first item, and during a time period different from the specified time period, selects, during a specified time period, second item information specifying a second item to be shipped based on the item management information, and generates an outbound shipping instruction for the second item, and the communication interface outputs an outbound shipping instruction and an outbound shipping cancellation for the first item in response to the selection of the first item information, and outputs an outbound shipping instruction for the second item in response to the selection of the second item information.
9. An item management method comprising: acquiring item management information; selecting item information specifying an item for which an image is to be acquired based on the item management information; generating an outbound instruction for an item specified by the item information selected as the item for which an image is to be acquired; communicating with a transport control device that controls a transport device that transports the item based on the outbound instruction and a picking control device that controls a picking device that recognizes and picks the item based on a captured image acquired from the item transported by the transport device; outputting an outbound instruction to at least the transport control device; and receiving the captured image acquired in response to the outbound instruction.
10. An item management program for causing a computer to execute the following steps: acquiring item management information; selecting item information specifying an item for which an image is to be acquired based on the item management information; generating an outbound instruction for an item specified by the item information selected as the item for which an image is to be acquired; and communicating with a transport control device that controls a transport device that transports an item based on the outbound instruction, and a picking control device that controls a picking device that recognizes and picks the item based on a captured image acquired from the item transported by the transport device, outputting an outbound instruction to at least the transport control device, and receiving the captured image acquired in response to the outbound instruction.
11. A non-transitory computer-readable storage medium storing an item management program executed by a processor, wherein the item management program causes a computer to execute the following steps: acquiring item management information; selecting item information that specifies an item for which an image is to be acquired based on the item management information; generating an outbound instruction for an item specified by the item information selected as the item for which an image is to be acquired; and communicating with a transport control device that controls a transport device that transports items based on the outbound instruction and a picking control device that controls a picking device that recognizes and picks the items based on photographed images acquired from the items transported by the transport device, outputting an outbound instruction to at least the transport control device, and receiving the photographed images acquired in response to the outbound instruction.
12. An item management system including an item management device that outputs an outgoing instruction, a transport control device that controls a transport device that transports items based on the outgoing instruction, and a picking control device that controls a picking device that picks items transported by the transport device, wherein the item management device comprises: an information acquisition interface that acquires item management information; a first processor that selects item information that specifies an item for which an image is to be acquired based on the item management information, and generates an outgoing instruction for an item specified by the item information selected as the image acquisition target; a first communication interface that communicates with the transport control device that controls the transport device and the picking control device that controls the picking device that recognizes and picks the item based on a photographed image acquired from the item transported by the transport device, and outputs an outgoing instruction to at least the transport control device and receives the photographed image acquired in response to the outgoing instruction; and the transport control device comprises: a second processor that generates an item transport control signal based on an item outgoing instruction output from the item management device; and a second communication interface that receives an item outgoing instruction output from the item management device and transmits the transport control signal to the transport device; and the picking control device An item management system comprising: a third processor that generates a photographing instruction to photograph an item transported by the transport device based on an item release instruction output from the item management device, and generates an item picking control signal to pick the item based on an item image obtained from the item by photographing; and a third communication interface that receives an item release instruction output from the item management device and transmits the photographing instruction and the picking control signal to the picking device.
Citation Information
Patent Citations
Picking robot and picking system
JP2019025566A
Data collection method for machine learning and camera system
JP2020197988A
Article inspection device, article inspection method, and program
JP2021155183A
Information processing device, information processing method, and program
JP2022033149A
Pick-up device, pick-up system, and conveyance shelf
JP2022181623A
Cited By
Detection system and sorting method
CN119387187A
