Information processing apparatus, information processing system, information processing method, and program

Abstract

To guide a route from a current position of a worker to a storage position of an article in a storage shelf in real time.SOLUTION: An information processing device according to the present disclosure includes an acquisition unit configured to temporally continuously acquire images captured by a work terminal and acquire a plurality of article identification codes attached to articles and a plurality of storage position identification codes attached to respective storage positions in a warehouse facility, a spatial position information generation unit configured to estimate three dimensional relative positions of the articles and the work terminal in real time based on the article identification codes and the storage position identification codes to generate three dimensional spatial position information, a spatial map creation unit configured to integrate the spatial position information to create a spatial map, and a route information generation unit configured to generate route information for guiding a route from a position of the work terminal to a storage position of an article specified based on the article identification code using the spatial map.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses an article storage assistance device that uses AR / VR. In Patent Document 1, individual product identification information is attached to products stored in a warehouse. In addition, shelf address identification information, which is information about the location within the warehouse, is attached to storage shelves in the warehouse.

[0003] When the individual product identification information of a product is acquired using the HMC (head-mounted camera) of the HMD (head-mounted display) worn by the worker, the information processing device searches a database using the individual product identification information as a key to identify storage location information where the product should be stored. Upon receiving the storage location information, the HMD displays information related to the location where the product should be stored so that it can be visually confirmed by the worker. The worker can then start work after confirming the instructions for the storage location.

[0004] When a worker who has finished storing work visually checks the shelf address identification information attached to the storage shelf, the HMC mounted on the HMD acquires the shelf address identification information of the product in question. The information processing device compares and verifies the shelf address identification information with the storage location information obtained by accessing the database (DB).

[0005] When changing a storage location, the worker actually moves the storage location of the product while giving instructions to the HMD to change the storage location of the product. When the worker visually checks the individual product identification information of the product and then looks toward the shelf address identification information related to the changed storage location, the HMC mounted on the HMD acquires the individual product identification information of the product and the changed storage location information. The information processing device receives the individual product identification information and the changed storage location information, and accesses the DB to update the data. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-124023 Summary of the Invention [Problem to be solved by the invention]

[0007] In many warehouse product management systems, identification information such as barcodes is attached to storage shelves and products. By having workers read each identification information in turn, the location where the worker stored the product and the product stored in that location can be ascertained.

[0008] Patent Document 1 proposes a technology that links individual product identification information with shelf address identification information when a worker handles a product, but it is not possible to obtain the worker's location in the warehouse in real time.There is a demand for a technology that can provide real-time guidance on the route from the worker's current location to the product's storage location on the storage shelf.

[0009] In view of the above-mentioned problems, the object of the present disclosure is to provide an information processing device, an information processing system, an information processing method, and a program that can provide real-time guidance on the route from a worker's current location to the storage location of an item on a storage shelf. [Means for solving the problem]

[0010] The information processing device according to the present disclosure includes an acquisition unit that acquires images taken by a work terminal in succession over time and acquires a plurality of item identification codes attached to items and storage location identification codes attached to each storage location in warehouse equipment; a spatial location information generation unit that estimates the three-dimensional relative positions of the items and the work terminal in real time based on the item identification codes and the storage location identification codes and generates three-dimensional spatial location information; a spatial map creation unit that integrates the spatial location information to create a spatial map; and a route information generation unit that generates route information that uses the spatial map to guide users along a route from the position of the work terminal to the storage location of the item identified based on the item identification code.

[0011] The information processing system according to the present disclosure includes a work terminal that generates images including item identification codes attached to items and storage location identification codes attached to each storage location in warehouse equipment by capturing images continuously over time, and an information processing device to which the images are input. The information processing device includes an acquisition unit that acquires the images continuously over time and acquires a plurality of the item identification codes and the storage location identification codes, a spatial location information generation unit that estimates the three-dimensional relative positions of the items and the work terminal in real time based on the item identification codes and the storage location identification codes and generates three-dimensional spatial location information, a spatial map creation unit that integrates the spatial location information to create a spatial map, and a route information generation unit that uses the spatial map to generate route information that provides guidance along a route from the position of the work terminal to the storage location of the item identified based on the item identification code.

[0012] The information processing method according to the present disclosure includes a process in which a computer continuously acquires images taken by a work terminal over time, and acquires multiple item identification codes attached to items and storage location identification codes attached to each storage location in warehouse equipment; a process in which the computer estimates the relative three-dimensional positions of the items and the work terminal in real time based on the item identification codes and the storage location identification codes, and generates three-dimensional spatial location information; a process in which the computer integrates the spatial location information to create a spatial map; and a process in which the computer generates route information that uses the spatial map to guide users along a route from the position of the work terminal to the storage location of the item identified based on the item identification code.

[0013] The program according to the present disclosure causes a computer to execute the following processes: a process of acquiring images taken by a work terminal in succession over time, and acquiring multiple item identification codes attached to items and storage location identification codes attached to each storage location in warehouse equipment; a process of estimating the three-dimensional relative positions of the items and the work terminal in real time based on the item identification codes and the storage location identification codes, and generating three-dimensional spatial location information; a process of integrating the spatial location information to create a spatial map; and a process of generating route information using the spatial map to guide the route from the position of the work terminal to the storage location of the item identified based on the item identification code. [Effects of the Invention]

[0014] According to the present disclosure, it is possible to provide real-time guidance on the route from the worker's current location to the storage location of the item on the storage shelf. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 2] 1 is a flowchart of an information processing method according to the present disclosure. [Figure 3] 1 is a block diagram illustrating a configuration of an information processing system according to the present disclosure. [Figure 4]1 is a block diagram showing the overall configuration of an information processing system according to the present disclosure. [Figure 5] FIG. 1 is a diagram illustrating an example of a warehouse to which an information processing system is applied. [Figure 6] FIG. 5 is a block diagram showing the configuration of the work terminal of FIG. 4. [Figure 7] FIG. 5 is a block diagram showing the configuration of the management terminal of FIG. 4. [Figure 8] FIG. 5 is a block diagram showing the configuration of the server in FIG. 4. [Figure 9] FIG. 10 is a diagram illustrating an example of layout information. [Figure 10] FIG. 10 is a diagram illustrating an example of product identification information. [Figure 11] FIG. 10 is a diagram illustrating an example of spatial position information. [Figure 12] FIG. 10 is a diagram showing a flow of preparation for product management operations in the information processing system. [Figure 13] FIG. 10 is a diagram showing a flow in which a manager issues a picking work instruction to a worker. [Figure 14] 10 shows a workflow in which a worker operates a work terminal and views work information. [Figure 15] This shows the internal processing flow of the system that is carried out while the worker is working. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and for clarity of explanation, duplicate explanations will be omitted as necessary.

[0017] The present disclosure relates to a technology for managing items and supporting work by workers in a warehouse, etc. In the following, an example will be described as an embodiment in which a worker patrols a warehouse and performs work of storing products on storage shelves and retrieving (picking) products.

[0018] Generally, a warehouse management system (WMS (Warehouse Management System)) is used to manage products in a warehouse. In many cases, storage shelves are attached with shelf position identification codes such as barcodes that indicate the location of the storage shelf within the warehouse. In addition, products are attached with product identification codes such as barcodes (JAN (Japanese Article Number) codes).

[0019] In the comparative example, an operator sequentially reads the shelf location identification code and the product identification code using a handheld terminal equipped with a barcode reader, and the management system associates the product to be worked on with the location of the storage shelf where the product is stored. However, this method requires a lot of work from the operator, and has the drawback that the information on the product's storage location is not pinpointed. When there are multiple products around a single storage shelf, the operator is tasked with selecting a specific product from among them.

[0020] Furthermore, the location of the worker can be determined from the location within the warehouse of the shelf location identification code read by the worker, but this merely indicates the location where the worker was when handling the product. If it is desired to use augmented reality (AR) or mixed reality (MR) technology to pinpoint the location from the worker's current location to the storage location of the product, it is not appropriate to use the location within the warehouse of the shelf location identification code read by the worker as the location of the worker.

[0021] Furthermore, many handheld terminals lack real-time capabilities, such as only being able to download data when connected to a dock attached to a management device, which makes it difficult for warehouse managers to give timely instructions to workers.

[0022] In comparison, there are products that use the camera function of a smartphone to read barcodes and QR codes (registered trademark) to manage products, but these are merely a substitute for handheld terminals. In other words, the smartphone is only used to link "storage shelves" with "products," whether it is reading one barcode at a time at any time, or multiple barcodes at the same time.

[0023] A first object of the embodiment is to enable real-time guidance of a route from the current location of a worker to the storage location of an item on a storage shelf.

[0024] Furthermore, in the comparative example, the storage location of a product is determined by the shelf location identification code, which means that the actual location of the product cannot be pinpointed. For example, a widely used method is to attach RFID (radio frequency identification) tags or BLE (Bluetooth Low Energy) tags to products to identify their locations. However, such tags are expensive, and some products require battery replacement, which increases the cost of simple devices. Furthermore, the need to attach tags to products correctly and to constantly manage tags to check for dead batteries, etc., limits the cost-saving benefits.

[0025] A second object of the embodiment is to pinpoint the location where a product is stored without using a device such as a tag.

[0026] Embodiment 1. An example configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a diagram showing the configuration of the information processing device 1 according to the present disclosure. As shown in Fig. 1, the information processing device 1 includes an acquisition unit 2, a spatial position information generation unit 3, a spatial map creation unit 4, and a route information generation unit 5. The information processing device 1 identifies a storage location in a warehouse where a product is to be stored.

[0027] Here, a storage location refers to a location within a warehouse where a product is stored. For example, if a warehouse is provided with multiple shelves as warehouse equipment, a storage location can refer to a specific storage space on the shelf. A storage space on the shelf can refer to a space for storing products separated by shelves or the like. Each storage location is assigned shelf location identification information (storage location identification information) using an identification code such as a barcode or QR code (registered trademark) that is necessary to identify the location of the storage location.

[0028] Furthermore, each product handled by a worker is assigned product identification information (item identification information) using an identification code such as a barcode or QR code (registered trademark). The product identification information may be a JAN code. The product identification information includes information necessary to identify the storage location of the product. Note that the product identification information may also include information for uniquely identifying the product.

[0029] The product identification information may include, by way of example, at least some of the following information: ·Product name Product height / width / depth dimensions Product weight Product shape ·Shipper information - Information indicating the storage location of the product specified by the shipper - Item arrival date - Information about the planned storage period of the product · Expiration date management information for managing product expiration dates, best-before dates, use-by dates, etc. - Invoice number, tracking number, etc. Product quantity information -Information on the number of items packed in one pallet

[0030] The information processing device 1 can perform various types of communication with external devices via a network. Specifically, the information processing device 1 can be connected via a communication line such as the Internet to a work terminal carried by a worker who performs work such as storing and picking products in a warehouse. However, the communication line is not limited to the Internet, and may be a combination of the Internet and another communication line, or may be a communication line other than the Internet. As a communication method, for example, wireless communication or wired communication such as 4G (Generation), 5G, local 5G, Wi-Fi (registered trademark), and LTE (Long Term Evolution) can be used. However, the communication method is not limited to these examples.

[0031] By using a work terminal to take continuous photographs over time, a worker can generate an image that includes product identification information attached to the product and shelf position identification information attached to each storage position on a shelf in the warehouse. For example, a worker can move around the warehouse while taking photographs of the surrounding environment using the work terminal. Note that an "image that includes product identification information and shelf position identification information" can be an image in which both product identification information and shelf position identification information are included in a single image, or an image in which multiple images taken consecutively include product identification information and shelf position identification information.

[0032] The acquisition unit 2 acquires images captured by the work terminal in a time-series manner. As a result, the acquisition unit 2 acquires multiple pieces of product identification information and shelf position identification information. In other words, the acquisition unit 2 may acquire an image in which both product identification information and shelf position identification information are included in a single image, or may acquire product identification information and shelf position identification information by sequentially acquiring multiple images that are time-series.

[0033] The spatial position information generating unit 3 estimates the relative three-dimensional positions of the product and the work terminal in real time based on the product identification information and shelf position identification information contained in the acquired image, and generates three-dimensional spatial position information. Note that, since the position of each storage shelf in the warehouse is determined in advance, the three-dimensional spatial position information indicating the shelf position can be stored in advance in a storage unit (not shown).

[0034] The spatial position information generation unit 3 can estimate the three-dimensional position of the work terminal that read the identification codes from the positions of the identification codes indicating product identification information and the identification codes indicating shelf position identification information in the image. The spatial position information generation unit 3 can also identify products based on the product identification information included in the image, and estimate the position of the identified product relatively from the positions of the identification codes indicating product identification information and the identification codes indicating shelf position identification information in the image, and the estimated position of the work terminal. In other words, the spatial position information generation unit 3 can pinpoint the actual position of a product.

[0035] The method used to estimate the position of the work terminal and the position of the product is not particularly limited. For example, a method can be used in which information about the size of the identification code and the attachment position of the identification code is registered in advance, and coordinates are estimated based on the position and distortion of the arrangement of pixels representing the identification code or storage shelf in the image. Alternatively, a method can be used in which photogrammetry is performed from the difference between frames of consecutive acquired images. SLAM (Simultaneous Localization and Mapping) can be used, which uses a 3D sensor to estimate the self-position and create a map of the space.

[0036] The spatial map creation unit 4 creates a spatial map by integrating the generated spatial position information. The spatial map creation unit 4 creates a spatial map by, for example, integrating and overlaying the identification codes indicating product identification information and the positions of the identification codes indicating shelf position identification information contained in the image, the estimated position of the work terminal, and the estimated position of the identified product. For example, the spatial map creation unit 4 can make adjustments to unify the coordinate system and align the positions of each identification code, work terminal, and product to create a three-dimensional spatial map.

[0037] The route information generating unit 5 generates route information using a spatial map to guide a route from the location of the work terminal to the storage location of the product identified based on the product identification information. The storage location of the product to be worked on can be identified based on the above-mentioned product identification information. For example, if the work to be performed by the worker is product storage work, the route information generating unit 5 displays on the spatial map the storage location for storing the product to be worked on.

[0038] The generated route information is output to the work terminal, allowing the worker to carry out work while checking a spatial map that overlays multiple pieces of information, including the estimated positions of the product and work terminal, and information on the storage location of the product to be worked on.

[0039] Next, the processing performed by the information processing device 1 will be described with reference to Fig. 2. Fig. 2 is a flowchart of an information processing method according to the present disclosure. First, the information processing device 1 acquires images captured by a work terminal in succession over time, and acquires multiple items of item identification information attached to the items and shelf position identification information attached to each storage position in the warehouse equipment (S11). Then, the information processing device 1 estimates the three-dimensional relative positions of the items and the work terminal in real time based on the item identification information and shelf position identification information, and generates three-dimensional spatial position information (S12).

[0040] The information processing device 1 then integrates the spatial position information to create a spatial map (S13). The information processing device 1 then generates route information using the spatial map to guide the route from the position of the work terminal to the storage location of the item to be worked on, which was identified based on the product identification information (S14). In this way, the information processing device 1 makes it possible to obtain route information that guides the route from the current position of the worker to the location of the item to be worked on in real time, without the need for the complicated task of reading the identification code, simply by capturing an image containing an identification code indicating the product identification information and shelf position identification information while the worker is moving.

[0041] The information processing device 1 includes a processor, a memory, and a storage device (not shown). The storage device stores a program that causes a computer to execute each process of the management method according to the first embodiment. The processor loads the program from the storage device into the memory and executes the program. As a result, the processor realizes the functions of an acquisition unit 2, a spatial position information generation unit 3, a spatial map creation unit 4, and a route information generation unit 5.

[0042] Each component of the information processing device 1 may be realized by dedicated hardware. Furthermore, some or all of the components of each device may be realized by general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and programs. Furthermore, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), quantum processor (quantum computer control chip), etc. may be used as the processor.

[0043] Furthermore, when some or all of the components of the information processing device 1 are realized by multiple devices, circuits, etc., the multiple devices, circuits, etc. may be centrally or decentralized. FIG. 3 is a block diagram showing the configuration of an information processing system 101 according to the present disclosure. As shown in FIG. 3, the information processing system 101 includes an acquisition unit 102, a spatial position information generation unit 103, a spatial map creation unit 104, and a route information generation unit 105. As described above, the detailed operation of each component of the information processing system 101 corresponds to the operation of each component of the information processing device 1. The devices, circuits, etc. realizing each component of FIG. 3 may be realized as a client-server system, a cloud computing system, or the like, connected via a communication network. Furthermore, the functions of the information processing device 1 may be provided in a SaaS (Software as a Service) format.

[0044] Embodiment 2. 4 is a block diagram showing the overall configuration of an information processing system 100 according to the present disclosure. The information processing system 100 includes a server 10, a work terminal 20, and a management terminal 30. The server 10, the work terminal 20, and the management terminal 30 are each connected via a network N. Here, the network N is a wired or wireless communication line, for example, the Internet.

[0045] 5 is a diagram showing an example of a warehouse W to which the information processing system 100 is applied. Here, in the warehouse W, route information is generated based on images taken by the work terminal 20 carried by a worker.

[0046] As shown in FIG. 5, warehouse W is equipped with a plurality of storage shelves 40. Each storage shelf 40 is equipped with a plurality of shelves 41 for storing products. In the example shown in FIG. 5, a shelf position identification code 42 is attached to the left end of each shelf 41. That is, the space partitioned by each shelf 41 becomes a storage space for storing products 43. Note that the storage space of each shelf 41 may be divided, and a shelf position identification code 42 may be attached to each divided space. Products 43 are arranged in the storage positions of each shelf 41. Product identification codes 44 are attached to each product 43.

[0047] Carrying the work terminal 20, the worker stores products brought into the warehouse in various storage locations, and retrieves designated products from storage locations according to instructions on slips and other documents. While moving around the warehouse W during work, the worker can use the work terminal 20 carried by the worker to continuously capture images including the shelf position identification code 42 and product identification code 44. The information processing system 100 uses the images including the shelf position identification code 42 and product identification code 44 sent from the work terminal 20 to generate route information that provides real-time guidance on the route from the worker's current location to the location of the product being worked on.

[0048] <Work terminal 20> The work terminal 20 is an information terminal that transmits and receives data. The work terminal 20 is, for example, a smartphone or tablet, and outputs information about the product to be worked on to the worker. FIG. 6 is a block diagram showing the configuration of the work terminal 20 in FIG. 4.

[0049] The work terminal 20 includes an imaging unit 21, a storage unit 22, a memory 23, a communication unit 24, an input / output unit 25, and a processing unit 26. The imaging unit 21 is an imaging device that takes images in accordance with the control of the processing unit 26. The imaging unit 21 continuously takes images including a shelf position identification code 42 and a product identification code 44. These images are input to the server 10.

[0050] The storage unit 22 is an example of a storage device that includes a non-volatile memory such as a flash memory or an SSD (Solid State Drive). The storage unit 22 stores a program that implements processes including a process for transmitting captured images and a process for notifying a worker of received information. The process for notifying a worker may include (1) a process for displaying the work content to be performed by the worker, (2) a process for displaying detailed information about the product that is the target of the worker's work (work target information), and (3) a process for displaying route information that guides the route from the location of the work terminal to the storage location of the target product within the warehouse W.

[0051] The memory 23 is a volatile storage device such as RAM (Random Access Memory), and is a storage area for temporarily storing information when the processing unit 26 is operating. The communication unit 24 is a communication interface with the network N. The communication unit 24 may also establish a short-range wireless communication connection with the server 10 and the management terminal 30 to perform communication. Here, various standards such as Bluetooth (registered trademark), BLE (Bluetooth Low Energy), and UWB (Ultra Wide Band) can be applied to the short-range wireless communication.

[0052] The input / output unit 25 is, for example, a touch panel, and includes a display device and an input device. The processing unit 26 is a processor that controls the hardware of the work terminal 20. The processing unit 26 loads a program from the storage unit 22 into the memory 23 and executes it. In this way, the processing unit 26 realizes the functions of a transmission / reception processing unit 261 and a display processing unit 262.

[0053] The transmission / reception processing unit 261 transmits an image including the shelf position identification code 42 and the product identification code 44 to the server 10. The transmission / reception processing unit 261 also receives various information necessary for the worker to perform the work from the server 10. The display processing unit 262 displays the received various information on the display device so that the worker can see it.

[0054] The display device can display, for example, route information received from the server 10, guiding the route from the location of the work terminal to the location of the product to be worked on. The worker can move to the location of the product to be worked on while visually checking the displayed route. The display device can also display the work content to be performed by the worker and the work target information, as described above. The work content to be performed by the worker may include, for example, storing the product in a storage shelf or picking up the product from a storage shelf. The work target information may include, for example, the product name, size, weight, and quantity of the product to be worked on.

[0055] The method of presenting the route information to the worker is not limited to a visual method such as displaying it on a display device. For example, the route information may be presented to the worker by voice. Therefore, the input / output unit 45 may include a speaker or the like.

[0056] Furthermore, the worker can use the input device to input information about the work he or she has performed as needed. For example, when the worker has completed picking up the designated product, the worker can input completion information indicating that the work has been completed. For example, a check box for inputting whether the work has been completed may be displayed on the touch panel screen of the work terminal 20. The completion information is sent to the server 10. This allows the manager to confirm that the worker has completed the work, and makes it possible to assign a new work to the worker who is now free.

[0057] <Administration terminal 30> The management terminal 30 is an information terminal used by the manager of the warehouse W, and transmits and receives data to and from the server 10 via the network N by wireless communication. The management terminal 30 is, for example, a PC, a smartphone, a tablet, etc. FIG. 7 is a block diagram showing the configuration of the management terminal 30 of FIG. 4.

[0058] The management terminal 30 includes a storage unit 31, a memory 32, a communication unit 33, an input / output unit 34, and a processing unit 35. The management terminal 30 mainly issues instructions to and manages the server 10. The storage unit 31 is an example of a storage device that includes a non-volatile memory such as a flash memory or an SSD (Solid State Drive). The storage unit 31 stores programs that have executed processes including processes for determining the work content to be performed by workers. The memory 32 is a storage area such as RAM for temporarily storing information when the processing unit 35 is operating. The communication unit 33 is a communication interface with the network N.

[0059] The input / output unit 34 may include, for example, a display device and an input device such as a keyboard or a mouse. The input / output unit 34 may be a touch panel in which the display device and the input device are integrated. The input / output unit 34 may include a known code reader. The manager can use the input / output unit 34 to input information identifying the goods to be received. The information about the goods to be received may be used when instructing a worker to store the goods in a storage shelf. The manager can also input information identifying the goods to be shipped. The information about the goods to be shipped may be used when instructing a worker to remove the goods from the storage shelf. Such product identification information is used to update the product information database, which will be described later.

[0060] The processing unit 35 is a processor that controls the hardware of the management terminal 30. The processing unit 35 loads a program from the storage unit 31 into the memory 32 and executes it. As a result, the processing unit 35 realizes the functions of a work instruction generation unit 351, a product management unit 352, a layout management unit 353, and a spatial map management unit 354.

[0061] The work instruction generation unit 351 generates work instructions including worker information that identifies the worker who will perform the work and the work content for the worker, and outputs them to the server 10. When creating work instructions for the worker to store the received goods on the storage shelf, the work instruction generation unit 351 can specify a storage location for storing the goods, for example, according to the product identification information of the received goods (information on the size of the goods, instructions from the shipper, etc.).

[0062] If a storage shelf has multiple storage spaces of different sizes, the work instruction generation unit 351 may identify a storage space with a small difference in size from the product as the storage location for the product. The work instruction generation unit 351 may also identify a storage location for the product to be stored, taking into account the sizes of products already stored on the shelf. The work instruction generation unit 351 may also identify a location that further satisfies information regarding the product's weight as the storage location. For example, if a shelf is configured with multiple levels, the work instruction generation unit 351 may identify the bottom level as the storage location for the product when the weight of the product is greater than a predetermined value.

[0063] The product management unit 352 outputs the above-mentioned product identification information to the server 10 and updates the product information database. The layout management unit 353 outputs layout management information, such as the arrangement of storage shelves 40 in the warehouse W and the size allocation of storage space in each storage shelf 40, entered by the manager to the server 10 and updates the layout database, which will be described later. The spatial map management unit 354 updates the spatial position information DB 114, which will be described later.

[0064] <Server 10> The server 10 is an information processing device that realizes functions such as a function of generating route information (route information generation function) that guides the route from the current location of the worker to the location where the product should be stored, a function of pinpointing the location of the product (location identification function), and a function of determining the validity of the relative locations of the storage shelf, product, and worker terminal (validity determination function).The server 10 may be composed of multiple servers, and each functional block may be realized by multiple computers.

[0065] Fig. 8 is a block diagram showing the configuration of the server 10 of Fig. 4. As shown in Fig. 8, the server 10 includes a storage unit 11, a memory 12, a communication unit 13, and a processing unit 200. The storage unit 11 is an example of a storage device such as a hard disk or a flash memory. The storage unit 11 stores a program 111, a layout DB (database) 112, a product information DB 113, and a spatial position information DB 114. The program 111 is a computer program that implements part of the above-mentioned path information generation function, position identification function, and validity determination function.

[0066] The layout DB 112 manages layout information that indicates the arrangement of structures such as aisles and storage shelves within the warehouse. FIG. 9 is a diagram showing an example of layout information. The layout information may include the attachment positions of barcodes that indicate shelf position identification information within the warehouse. The layout information is, for example, a plan view showing the aisles of the warehouse. As shown in FIG. 9, the warehouse may be divided into multiple areas. The "scale" is used to match the scale of the layout information with that of the image input from the work terminal 20. For example, the "scale" may indicate the unit length in the layout information.

[0067] The layout of the aisles in each area can be expressed as data indicating x and y coordinate values, with (0, 0) on the plane coordinate system as the origin. This origin position information is used when overlaying the estimated position information of the products and work terminals stored in the spatial position information DB 114. The layout information is also used in processing by the association determination unit 240, which will be described later, such that the position of the work terminal (worker) must be in an aisle and the position of the products must be on a storage shelf other than the aisle. The layout information can also be used when the route information generation unit 232, which will be described later, generates route information and determines that the worker cannot pass through places other than the aisle.

[0068] The product information DB 113 manages various information related to products. The product information DB 113 can store the above-mentioned product identification information. Fig. 10 is a diagram showing an example of the product identification information. As shown in Fig. 10, the product identification information includes general information related to the product, such as the bar code attached to each product, the shipper, and the date of arrival / departure.

[0069] The spatial position information DB 114 manages spatial position information that indicates the relative positions in three dimensions of products and work terminals 20, estimated by a spatial position information generation unit 220 of the processing unit 200, which will be described later. FIG. 11 is a diagram showing an example of spatial position information. As shown in FIG. 11, the area and X, Y, and Z coordinate values ​​of the estimated position of each work terminal 20 and each product are shown. The spatial position information also includes a barcode attached to each product. In other words, the spatial position information DB 114 holds link information with the layout DB 112 and product information DB 113, and coordinate information of the products and work terminals in three-dimensional space.

[0070] The layout DB 112, product information DB 113, and spatial position information DB 114 are managed via the aggregation unit 230, which will be described later, and mainly via the position information integration unit 231. The reason for management via the aggregation unit 230 is that, for example, the product information DB 113 is updated in response to input from the product management unit 352 of the management terminal 30 and is also rewritten when a worker completes picking or storage work, requiring coordinated information updates. Note that the configurations of the layout DB 112, product information DB 113, and spatial position information DB 114 are not limited to the examples described above.

[0071] The memory 12 is a volatile storage device such as a RAM, and is a storage area for temporarily storing information when the processing unit 200 is operating. The communication unit 13 is a communication interface with the network N. The processing unit 200 is a processor that controls each component of the server 10. The processing unit 200 loads a program 111 from the storage unit 11 into the memory 12 and executes the program 111. In this way, the processing unit 200 realizes the functions of an acquisition unit 210, a spatial position information generation unit 220, an aggregation unit 230, and an association determination unit 240.

[0072] The acquisition unit 210 includes an image analysis unit 211 and a code reading unit 212. Images captured by the imaging unit 21 of the work terminal 20 are input to the image analysis unit 211 in a time-series manner. The image analysis unit 211 analyzes the input images and automatically detects the position of the barcode. The code reading unit 212 reads the number (a string of numbers) included in the detected barcode. This allows the acquisition unit 210 to acquire multiple pieces of product identification information attached to the product and shelf position identification information attached to each storage position of the storage shelf 40.

[0073] The spatial position information generation unit 220 includes a work terminal 3D position estimation unit 221, a product identification unit 222, and a product 3D position estimation unit 223. The work terminal 3D position estimation unit 221 estimates the three-dimensional position of the work terminal 20 that captured the image from the position of the barcode in the image acquired by the acquisition unit 210. The product identification unit 222 checks whether or not there is a barcode indicating product identification information of the product among the barcodes acquired by the acquisition unit 210. The product identification unit 222 compares the product identification information acquired from the image with the product identification information in the product information DB 113 to identify the product.

[0074] The product 3D position estimation unit 223 estimates the position of the product identified by the product identification unit 222 in three dimensions from the position of the work terminal 20 and the position of the barcode in the image acquired by the acquisition unit 210. In other words, the spatial position information generation unit 220 estimates the three-dimensional relative positions of the product and the work terminal based on the product identification information and shelf position identification information acquired from the image input from the work terminal 20, and generates three-dimensional spatial position information.

[0075] The method of estimating the 3D position used by the work terminal 3D position estimation unit 221 and the product 3D position estimation unit 223 is not particularly limited. As described above, for example, a method can be used in which information on the size of the identification code and the attachment position of the identification code is registered in advance, and coordinates are estimated based on the position and distortion of the arrangement of pixels representing the identification code or storage shelf in the image. Also, a method of performing photogrammetry from the frame difference between consecutive acquired images can be adopted. SLAM, which uses a 3D sensor to estimate the self-position and create a map of the space, can be used.

[0076] The aggregation unit 230 includes a position information integration unit 231, a route information generation unit 232, and a work information generation unit 233. The position information integration unit 231 integrates information from the layout DB 112, product information DB 113, and spatial position information DB 114, and overlays the estimated positions of products, work terminals, etc. Hereinafter, the information from each database integrated and overlaid on each other will be referred to as a "spatial map."

[0077] That is, the location information integrating unit 231 functions as a spatial map creating unit that integrates information from each database to create a spatial map. A "spatial map" represents products, work terminals 20, storage shelves 40, shelves 41, aisles, etc., using coordinate values ​​in an XYZ coordinate system. Therefore, the location information integrating unit 231 can identify the storage location of the work object as coordinate information on the spatial map. In this way, the location information integrating unit 231 can integrate and overlay multiple pieces of information from each database, which were not managed simultaneously in the comparative example, to extract pinpoint three-dimensional location information for products and work terminals (workers).

[0078] The route information generation unit 232 generates route information using a spatial map to guide the route from the location of the work terminal to the location of the product identified based on the product identification information. Specifically, when the route information generation unit 232 receives a work instruction from the management terminal 30, including the above-mentioned worker information and work content, it uses a spatial map to calculate the route required for the worker to perform the work. Specifically, the route information generation unit 232 uses the work target information, including the product to be worked on and its quantity, from the product information DB 113, the estimated positions of the work terminal 20 and the product to be worked on from the spatial position information DB 114, and the aisle information within the warehouse from the layout DB 112, to determine the route the worker will take when performing the work, taking into account the accurate positions of both the work terminal 20 and the product to be worked on. The route information generation unit 232 can generate route information using coordinate information in the above-mentioned XYZ coordinate system.

[0079] The position information integration unit 231 can update the spatial position information DB 114 based on the position information of the barcode input from the spatial position information generation unit 220 and the estimated position information of the product and the work terminal 20. Note that the accuracy of updating the spatial position information DB 114 can be further improved based on the validity determination result of the two types of position information input from the association determination unit 240. The association determination unit 240 will be described in detail later.

[0080] The work information generation unit 233 transmits to the work terminal 20 work information necessary for the worker to perform the work, such as work instructions given from the management terminal 30, route information calculated by the location information integration unit 231, and a spatial map obtained as a result of integrating each database. For example, the work instructions may include work object information indicating detailed information about the identified work target product and the quantity of work target product to be picked up. Based on the work information, the work terminal 20 displays the work content to be performed by the worker, the route to be taken when performing the work, and the work object information.

[0081] The route information may be information that displays on a spatial map a route from the position of the work terminal 20 to the storage location of the target product identified based on the product identification information. The route information may be displayed superimposed on an image captured by the work terminal 20 according to at least one of a text display, an augmented reality display, and a mixed reality display. The route information may also include stop position information that indicates the stop position of the worker that is displayed on the spatial map when the work terminal 20 is located a predetermined distance from the storage location of the target product.

[0082] The association determination unit 240 includes a relative position validity determination unit 241 and a time series validity determination unit 242. The relative position validity determination unit 241 compares the current spatial map with a spatial map that is temporally earlier than the current spatial map to determine the validity of the relative positional relationships between the product, the work terminal 20, the product identification information, and the shelf position identification information. Specifically, the relative position validity determination unit 241 determines whether there are any abnormalities in the associations of relative positions, such as the positions of the aisle and the worker, the positions of the shelves, the positions of the shelves and the products, and the positions of the products, based on the positions of the barcodes in the image, the estimated positions of the work terminal 20 and the products estimated by the spatial position information generation unit 220, and the database information integrated and superimposed by the aggregation unit 230.

[0083] The time series validity determination unit 242 compares the current spatial map with a spatial map taken chronologically before the current spatial map to determine the validity of the positional relationship in time series. Specifically, the time series validity determination unit 242 determines whether the time series information of the worker's movements, i.e., the barcode information read from the temporally consecutive images taken by the worker while moving, contains any abnormal content. The processing by the association determination unit 240 makes it possible to pinpoint and ensure highly accurate data on the two pieces of position information for the worker and the product.

[0084] Here, the flow of the information processing method according to the present disclosure will be described with reference to Figures 12 to 15. Here, the flow of the picking work will be described. Figure 12 is a diagram showing the flow of preparation for operation of product management in the information processing system 100.

[0085] The manager operates the input / output unit 34 to input layout information indicating the arrangement of structures within the warehouse and the positions within the warehouse where barcodes indicating shelf position identification information are attached (S001). The manager also sets the origin of the coordinate system of the spatial map (S0002). Note that the origin of the coordinate system of the spatial map does not need to be set by the manager, and may be automatically set to a corner of the map, for example.

[0086] The manager then operates the input / output unit 34 to input product identification information (barcode, product name, quantity, etc.), which is data linked to information required to identify the products handled by the worker (S0003). This forms the basis of the spatial map. As the spatial map continues to be updated in the subsequent flow, the positions of the worker terminal (worker) and the product positions are managed. Note that step S0003 is repeatedly performed during operation, and the product identification information is always kept up to date. Step S0003 may be automated, for example, based on product inventory information.

[0087] 13 is a diagram showing the flow in which a manager issues picking work instructions to a worker. First, the manager operates the input / output unit 34 to specify one or more types of products to be worked on and the required quantity (S101). The manager may select the products to be worked on from a list, or may select them in a video game-like manner by operating in a digital twin space that is a three-dimensional spatial map.

[0088] Next, the manager operates the input / output unit 34 to determine which worker to issue the work instruction to (S102). The manager may specify a specific worker, or may automatically assign a worker who is in a position that makes it easy to pick the product specified in S101, or a worker who has free time. The work instruction, including the instruction content and worker information, is processed by the aggregation unit 230. The route information generation unit 232 generates a route using a spatial map to guide the worker from the location of the work terminal to the storage location of the product to be worked on, which was identified based on the product identification information (S103). Thereafter, the work information generation unit 233 transmits work information necessary for the worker to perform the work, including the work instruction and route information, to the work terminal 20 (S104).

[0089] The worker who has the work terminal that received the work information can then carry out the picking work in accordance with the work content. The processing of the information processing system 100 at this time can be divided into a flow in which the worker operates the work terminal 20 to view the work information (FIG. 14), and an internal system processing flow that takes place during that time (FIG. 15).

[0090] 14 shows a worker workflow in which a worker operates the work terminal 20 to view work information. First, the received work information is displayed on the display device of the work terminal 20 carried by the worker (S201). The worker can accept the work instruction by operating the input / output unit 25 (S202). If the work instruction includes the task of picking up multiple items, a flow that is repeated until one item is acquired (S220) and a flow that is repeated until all items are acquired (S210) are executed.

[0091] In the flow (S220) that is repeated until one product is acquired, first, route information is displayed on the display device of the work terminal 20 (S221). The worker can start moving according to the displayed route (S222). As the worker moves, the internal processing (FIG. 15) described below is repeatedly executed, and the estimated position of the work terminal 20 in the spatial position information DB 114 is updated, thereby repeatedly updating the route information.

[0092] Furthermore, when the worker moves to the location of the product to be picked and the estimated location of the work terminal 20 is updated to a location within a predetermined processing range from the estimated location of the product by the internal processing shown in FIG. 15 , the flow of S220 is exited. Then, the input / output unit 25 displays work target information indicating the product to be worked on and the required quantity (S211). The worker can check the displayed information and perform the picking. At this time, if detailed information such as the date of entry is required, the worker may operate the input / output unit 25 to request the server 10 to access the product information DB 113 and read the detailed information (S212). If detailed information is requested (S212, YES), the work terminal 20 can obtain and display the detailed information from the server 10 (S213). At this time, the server 10 may also read a work procedure manual (manual) or the like and send it to the work terminal 20.

[0093] When the worker has completed all the tasks displayed on the input / output unit 25, the worker can input completion information through the input / output unit 25 (S214). This allows the worker to move on to the next item to be worked on. When all items to be worked on have been picked, this flow ends. Note that the information processing system 100 may continue route guidance until the picked up items are taken to the designated location.

[0094] Next, the internal system processing that is performed while the worker is working will be described with reference to Fig. 15. Fig. 15 shows the flow of the internal system processing that is performed while the worker is working. While the worker is working, the imaging unit 21 of the acquisition unit 2 continuously captures images and continues to transmit them to the server 10. The acquisition unit 210 of the server 10 continuously acquires images over time and analyzes the barcodes in the images (S301).

[0095] If identification information, which is product identification information or shelf position identification information, cannot be acquired from the barcode (S302, NO), image acquisition and analysis of the barcode in the image (S301) continue. If identification information can be acquired from the barcode (S302, YES), the working terminal 3D position estimation unit 221 estimates the position of the working terminal 20 using all of the identification information in the image (S303).

[0096] Thereafter, the association determination unit 240 determines the validity of the estimated position of the work terminal 20 (S304). The association determination unit 240 determines the validity negatively if multiple barcodes with impossible combinations are read out based on the estimated position of the work terminal 20 and the relative positional relationship between the barcodes attached to the products in the image and the barcodes attached to the shelf boards 41, or if a comparison of the time-series information indicates that the worker moved at a speed that would normally be impossible or to an impossible height. If the estimated position of the work terminal 20 is not valid (S304, NO), the data is discarded (S305).

[0097] If the estimated position of the work terminal 20 is valid (S304, YES), the position information integration unit 231 updates the position of the work terminal 20 in the spatial map in the spatial position information DB 114 (S306). The position information integration unit 231 can update the spatial map, for example, every time an image is acquired from the work terminal 20. In response to this, the route information generation unit 232 updates the route information (S307) and outputs it to the work terminal 20. The work terminal 20 can display the updated route information (S221 in FIG. 14).

[0098] Furthermore, if the position of the work terminal 20 is valid and the product barcode has been acquired (S308, YES), the product 3D position estimation unit 223 estimates the product position (S309). However, if the product barcode has not been acquired (S308, NO), the process returns to S301. Furthermore, as in the case of the work terminal 20 (S304), the association determination unit 240 determines the validity of the estimated product position (S310). If the estimated product position is invalid (S310, NO), that is, if the package is not where it should be, the association determination unit 240 can issue invalid position information to the management terminal 30 (S311).

[0099] If the estimated product position is valid (S310, YES), it is confirmed whether the product for which product identification information has been acquired is the product to be worked on as instructed in Fig. 13 (S312). If this product is a product to be worked on (S312, YES), the work information generation unit 233 notifies the work terminal 20 of the work target information (S313, S211). If this product is not a product to be worked on (S312, NO), the process returns to S301 again.

[0100] So far, an example of picking work has been described. In the above-mentioned picking work, the route information indicates a route from the location of the work terminal to the target product based on a barcode indicating product identification information. In storage work, the route information may indicate a route from the location of the work terminal to the package storage location based on a barcode indicating shelf location identification information indicating the shelf location. In this case, after the target product is stored, it is determined whether the product has been stored in the specified location, and if there are no problems, the work is completed.

[0101] As described above, according to the present disclosure, by simply continuing to take images with the work terminal, it is possible to utilize barcodes attached to items or within the warehouse to confirm the location of the worker terminal or items in three-dimensional coordinates, and to guide the worker from their current location to the work location.

[0102] The present disclosure has the following seven advantages, for example: (1) Not only can it be determined that a product is located near a specific storage shelf identified by shelf location identification information, but it can also be determined not only by the human eye but also by the system itself that the product is located in a pinpoint position using highly accurate three-dimensional location information. (2) By being able to pinpoint product location and product identification information in three-dimensional space without the need for tedious repeated barcode reading, warehouse operations can be carried out efficiently without workers having to search for products around storage shelves.

[0103] (3) The system reduces the work of reading barcodes, allows storage locations to be specified graphically, and automatically verifies the validity of the location while the worker is working, thereby reducing human errors such as failing to read barcodes and reducing the loss of products.

[0104] (4) By providing work instructions using images, such as route information, rather than text, even new workers who are unfamiliar with the layout of the warehouse can complete their tasks in a time similar to that of veteran workers. (5) Since there is little written information in the work instructions, even foreign workers can carry out their work without any misunderstandings in communication.

[0105] (6) Even if a product is placed at the back of a storage shelf that is not managed by shelf location identification information and is in a blind spot that is difficult for the human eye to find, the location of the product can be graphically grasped, making it easy to find. (7) It is possible to determine whether products that do not require topping or that have storage conditions that prohibit stacking are placed correctly, and whether products that have expired are being stored, while the worker is performing other tasks.

[0106] Other embodiments The work terminal 20 may be, for example, a smartphone with a camera or smart glasses with a camera. The work terminal 20 may also be a combination of a wearable camera and a small terminal such as a smartphone. The work terminal 20 may also be a picking cart used to pick products from storage locations, to which a web camera and tablet are attached.

[0107] In addition to the images captured by the work terminals of the present disclosure, barcode monitoring by fixed cameras installed in the warehouse may also be used in combination. The system of the present disclosure can also be used in conjunction with existing indoor positioning technology to improve the positional accuracy of work terminals and products.

[0108] In the above-described embodiment, the location information is managed using three-dimensional coordinates, but the three-dimensional coordinate information may be combined with a technology that identifies the storage location of a product using existing shelf location identification information. The system disclosed herein continuously performs inventory, so it can also be used to detect product loss (theft). In addition, the system may be installed on a patrol robot and used as an unmanned monitoring device.

[0109] The system disclosed herein is applicable to material management and picking work by workers in logistics warehouses and raw material warehouses where workers patrol the warehouse. The system disclosed herein can also be used to grasp the location of a patrol robot patrolling the warehouse and to grasp product packages. The system disclosed herein can contribute to improving the efficiency of back-of-house inventory management and stocking in retail stores such as supermarkets. The system disclosed herein is suitable, for example, for work sites where the types of inventory in a store change frequently and where workers, such as part-timers and casual workers, are frequently replaced. The system disclosed herein also enables part-timers to retrieve prescription medications at dispensing pharmacies.

[0110] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include RAM, read-only memory (ROM), flash memory, SSD or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0111] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0112] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0113] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix A1) an acquisition unit that acquires images taken by the work terminal in succession over time and acquires a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility; a spatial position information generation unit that estimates a three-dimensional relative position between the item and the work terminal in real time based on the item identification code and the storage location identification code, and generates three-dimensional spatial position information; a spatial map creation unit that creates a spatial map by integrating the spatial position information; a route information generating unit that generates route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code; Equipped with Information processing device. (Appendix A2) a determination unit that compares a current spatial map with a spatial map that is temporally older than the current spatial map to determine at least one of the validity of a relative positional relationship between the item, the work terminal, the item identification code, and the storage location identification code, and the validity of a positional relationship in time series; 10. The information processing device according to claim 1, (Appendix A3) the route information is information that displays on the spatial map a route from the position of the work terminal to a storage location of the item identified based on the item identification code; An information processing device according to appendix A1 or A2. (Appendix A4) the route information includes stop position information indicating a stop position of the worker to be displayed on the spatial map when the work terminal is located a predetermined distance from the storage position of the item; 10. The information processing device according to claim 9, wherein the information processing device is a (Appendix A5) The route information is displayed superimposed on an image captured by the work terminal in accordance with at least one of a text display, an augmented reality display, and a mixed reality display. 10. The information processing device according to claim 9, wherein the information processing device is a device for processing information. (Appendix A6) the spatial map creation unit identifies the storage position of the item as coordinate information on the spatial map; The route information generation unit generates the route information using the coordinate information. An information processing device according to any one of appendices A1 to A5. (Appendix A7) the spatial map creation unit updates the spatial map every time the image is acquired from the work terminal. An information processing device according to any one of appendices A1 to A6. (Appendix B1) a work terminal that takes continuous photographs in time to generate images including an item identification code attached to the item and a storage location identification code attached to each storage location in the warehouse facility; an information processing device to which the image is input; Including, The information processing device includes: an acquisition unit that acquires the images successively in time and acquires a plurality of the item identification codes and the storage location identification codes; a spatial position information generation unit that estimates a three-dimensional relative position between the item and the work terminal in real time based on the item identification code and the storage location identification code, and generates three-dimensional spatial position information; a spatial map creation unit that creates a spatial map by integrating the spatial position information; a route information generating unit that generates route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code; Equipped with Information processing system. (Appendix C1) The computer A process of acquiring images taken by the work terminal in succession over time, and acquiring a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility; a process of estimating in real time a three-dimensional relative position between the item and the work terminal based on the item identification code and the storage location identification code, and generating three-dimensional spatial position information; A process of integrating the spatial location information to create a spatial map; a process of generating route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code; To execute Information processing methods. (Appendix D1) A process of acquiring images taken by the work terminal in succession over time, and acquiring a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility; a process of estimating in real time a three-dimensional relative position between the item and the work terminal based on the item identification code and the storage location identification code, and generating three-dimensional spatial position information; A process of integrating the spatial location information to create a spatial map; a process of generating route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code; to the computer, program.

[0114] Some or all of the elements (e.g., configurations and functions) described in Appendix A1 to Appendix A7 that are dependent on Appendix A1 (device) may also be dependent on Appendix B1 (information processing system), Appendix C1 (method), and Appendix D1 (program) in the same dependency relationship as Appendix A1 to Appendix A7. Some or all of the elements described in any appendix may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0115] 1. Information processing equipment 2 Acquisition part 3 Spatial position information generation section 4. Spatial Mapping Department 5. Route information generation unit 10 Servers 11 Storage section 12 Memory 13 Communications Department 111 Program 112 Layout DB 113 Product information DB 114 Spatial location information DB 20 Work terminal 21 Imaging unit 22 Memory section 23 Memory 24 Communications Department 25 Input / output section 26 Processing section 261 Transmission and reception processing section 262 Display processing section 30 Management terminal 30 Management terminal 31 Storage section 32 memory 33 Communications Department 34 Input / output section 35 Processing section 351 Work instruction generation section 352 Product Management Department 353 Layout Management Department 354 Spatial Map Management Department 40 Storage Shelves 41 Shelf 42 Shelf location identification code 43 items 44 Product Identification Code 100 Information Processing Systems 101 Information Processing Systems 102 Acquisition Department 103 Spatial position information generation unit 104 Spatial Mapping Department 105 Route information generation unit 200 Processing section 210 Acquisition Department 211 Image Analysis Unit 212 Code reader 220 Spatial position information generation unit 221 Work terminal 3D position estimation unit 222 Product Specification Department 223 Product 3D position estimation section 230 Consolidation Department 231 Location information integration section 232 Route information generation unit 233 Work information generation section 240 Related Judgment Section 241 Relative position validity determination unit 242 Time Series Validity Judgment Unit W Warehouse

Claims

1. an acquisition unit that acquires images taken by the work terminal in succession over time and acquires a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility; a spatial position information generation unit that estimates a three-dimensional relative position between the item and the work terminal in real time based on the item identification code and the storage location identification code, and generates three-dimensional spatial position information; a spatial map creation unit that creates a spatial map by integrating the spatial position information; a route information generating unit that generates route information using the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code; Equipped with Information processing device.

2. a determination unit that compares a current spatial map with a spatial map that is temporally older than the current spatial map to determine at least one of the validity of a relative positional relationship between the item, the work terminal, the item identification code, and the storage location identification code, and the validity of a positional relationship in time series; The information processing device according to claim 1 .

3. the route information is information that displays on the spatial map a route from the position of the work terminal to a storage location of the item identified based on the item identification code; The information processing device according to claim 1 .

4. the route information includes stop position information indicating a stop position of the worker to be displayed on the spatial map when the work terminal is located a predetermined distance from the storage position of the item; The information processing device according to claim 3 .

5. the route information is displayed superimposed on an image captured by the work terminal in accordance with at least one of a text display, an augmented reality display, and a mixed reality display; The information processing device according to claim 3 .

6. the spatial map creation unit identifies the storage position of the item as coordinate information on the spatial map; The route information generation unit generates the route information using the coordinate information. The information processing device according to claim 1 .

7. the spatial map creation unit updates the spatial map every time the image is acquired from the work terminal. The information processing device according to claim 1 .

8. a work terminal that takes continuous photographs in time to generate images including an item identification code attached to the item and a storage location identification code attached to each storage location in the warehouse facility; an information processing device to which the image is input; Including, The information processing device includes: an acquisition unit that acquires the images successively in time and acquires a plurality of the item identification codes and the storage location identification codes; a spatial position information generation unit that estimates a three-dimensional relative position between the item and the work terminal in real time based on the item identification code and the storage location identification code, and generates three-dimensional spatial position information; a spatial map creation unit that creates a spatial map by integrating the spatial position information; a route information generating unit that generates route information using the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code; Equipped with Information processing system.

9. The computer A process of acquiring images taken by the work terminal in succession over time, and acquiring a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility; a process of estimating in real time a three-dimensional relative position between the item and the work terminal based on the item identification code and the storage location identification code, and generating three-dimensional spatial position information; A process of integrating the spatial location information to create a spatial map; a process of generating route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code; To execute Information processing methods.

10. A process of acquiring images taken by the work terminal in succession over time, and acquiring a plurality of item identification codes attached to the items and storage location identification codes attached to each storage location in the warehouse facility; a process of estimating in real time a three-dimensional relative position between the item and the work terminal based on the item identification code and the storage location identification code, and generating three-dimensional spatial position information; A process of integrating the spatial location information to create a spatial map; a process of generating route information that uses the spatial map to guide a route from the location of the work terminal to a storage location of the item identified based on the item identification code; to the computer, program.

Images