Simulation model generation method, simulation model generation program, and simulation model generation system

Abstract

This simulation model generating method includes: acquiring layout information relating to a work site from a management system; estimating the position of a worker at the work site on the basis of a first captured image captured by a first camera that captures images of the work site; calculating at least one of a movement speed of the worker and a work time of the worker in an area of the work site corresponding to the position, on the basis of the estimation result; and generating a simulation model of the work site on the basis of the layout information and the at least one of the movement speed and the work time of the worker in each of one or more areas of the work site.

Description

Simulation model generation method, simulation model generation program, and simulation model generation system

[0001] The present disclosure relates to a simulation model generation method, a simulation model generation program, and a simulation model generation system.

[0002] Patent Literature 1 discloses a multi-agent simulation system that performs model correction to reduce the degree to which the results of a multi-agent simulation deviate from reality. This system creates a simulation model using real-world personal data, such as personal attribute information based on a questionnaire.

[0003] Japanese Patent Application Publication No. 2014-174705

[0004] For example, simulation models used in digital twins are required to reproduce the real world, such as a work site, in real time with high accuracy. Therefore, it is preferable to use a wide variety of site data in real time to generate the simulation model. Site data includes not only data that is generally easy to obtain, such as drawing data, but also data that is difficult to measure or obtain, such as data related to the movements of workers on site. Such data that is difficult to measure or obtain may be necessary to generate a highly accurate simulation model. However, the multi-agent simulation system of Patent Document 1 does not consider the above case because it creates a simulation model using personal data accumulated in advance.

[0005] The present disclosure has been devised in view of the above-described conventional situation, and aims to generate a highly accurate simulation model for managing a work site.

[0006] The present disclosure provides a simulation model generation method that connects to a management system that stores layout information of a work site, acquires the layout information from the management system, estimates a position of a worker at the work site based on a first image captured by a first camera that images the work site, calculates at least one of a movement speed of the worker and a working time of the worker in an area of ​​the work site that corresponds to the position based on the estimation result, and generates a simulation model of the work site based on the layout information and at least one of the movement speed and the working time of the worker in each of one or more areas of the work site.

[0007] The present disclosure also provides a simulation model generation method that connects to a management system that stores layout information of a work site, acquires the layout information from the management system, estimates a position of a worker at the work site based on a first captured image taken by a first camera that images the work site, classifies work performed by the worker based on a second captured image taken by a second camera that images the worker's hands, calculates at least one of a movement speed of the worker and a work time for each work unit performed by the worker in the area based on the estimation result and the classification result, and generates a simulation model of the work site based on the layout information and at least one of the movement speed and the work time for each work unit.

[0008] The present disclosure also provides a simulation model generation program that causes a computer connected to a management system that stores layout information of a work site to execute the following processes: acquiring the layout information from the management system; estimating the position of a worker at the work site based on a first image captured by a first camera that images the work site; calculating at least one of the worker's movement speed and the worker's working time in an area of ​​the work site that corresponds to the position based on the estimation result; and generating a simulation model of the work site based on the layout information and at least one of the worker's movement speed and the worker's working time in each of one or more areas of the work site.

[0009] The present disclosure also provides a simulation model generation system including a management system that holds layout information of a work site, and a computing device connected to the management system, wherein the computing device acquires the layout information from the management system, estimates a position of a worker at the work site based on a first image captured by a first camera that images the work site, calculates at least one of a movement speed of the worker and a working time of the worker in an area of ​​the work site that corresponds to the position based on the estimation result, and generates a simulation model of the work site based on the layout information and at least one of the movement speed and the working time of the worker in each of one or more areas of the work site.

[0010] Any combination of the above components, and conversion of the expression of the present disclosure into a method, device, system, storage medium, computer program, etc., are also valid aspects of the present disclosure.

[0011] According to the present disclosure, a simulation model for managing a work site can be generated with high accuracy.

[0012] Schematic diagram for explaining the flow of processing for creating log data according to the embodiment 1. Schematic diagram for explaining the flow of processing for creating human information according to the embodiment 1. Schematic diagram for comparing the workflow in a conventional digital twin with the workflow in a digital twin according to the embodiment 1. Schematic diagram for explaining the creation of human information using a position estimation unit and a work classification unit according to the embodiment 1. Schematic diagram for explaining the flow of processing for creating log data according to the embodiment 1. Schematic diagram for explaining the flow of processing for creating human information according to the embodiment 1. Schematic diagram for comparing the workflow in a digital twin according to the embodiment 1.

[0013] Hereinafter, with reference to the drawings as appropriate, detailed descriptions will be given of embodiments that specifically disclose a simulation model generation method, a simulation model generation program, and a simulation model generation system according to the present disclosure. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

[0014] <First Embodiment> [System Configuration] First, a configuration example of a simulation model generation system 1 according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing a configuration example of the simulation model generation system 1 according to the first embodiment. The simulation model generation system 1 includes a computing device 10, a database 15, a position estimation camera 20, a handheld camera 30, a warehouse management system 40, a database 41, and a simulation device 50.

[0015] The arithmetic device 10 includes at least a processor 11 and a memory 14. The arithmetic device 10 may be configured using a general-purpose computer device such as a personal computer or a server computer. The arithmetic device 10 is capable of data communication with various devices included in the simulation model generation system 1 and external devices (not shown).

[0016] The processor 11 may be configured using, for example, a central processing unit (hereinafter referred to as "CPU"), a graphic processing unit (hereinafter referred to as "GPU"), a micro processing unit (hereinafter referred to as "MPU"), a digital signal processor (hereinafter referred to as "DSP"), or a field programmable gate array (hereinafter referred to as "FPGA"). The processor 11 realizes various functions of the arithmetic device 10 by reading various data and programs stored in the memory 14. For example, the processor 11 and the memory 14 cooperate to realize various functions of the position estimation unit 12 and the task classification unit 13, which will be described later.

[0017] The memory 14 is a storage unit for storing various data, programs, etc. The memory 14 may be configured from a volatile / non-volatile storage device such as a random access memory (hereinafter referred to as "RAM"), a read only memory (hereinafter referred to as "ROM"), a hard disk drive (hereinafter referred to as "HDD"), or a solid state drive (hereinafter referred to as "SSD").

[0018] The database 15 is configured using a non-volatile storage device, and stores various programs and data handled by the arithmetic device 10. The arithmetic device 10 and the database 15 may be configured as an integrated unit.

[0019] The position estimation camera 20 captures images of the warehouse interior and generates captured images. The simulation model generation system 1 may include multiple cameras as the position estimation camera 20. The position estimation camera 20 may be attached, for example, to a specific location in the warehouse where the worker is located, or may be attached to equipment used by the worker while working in the warehouse. For example, the position estimation camera 20 may be attached to a cart used by the worker for picking work so as to be able to capture images in the direction of travel of the cart. Hereinafter, the position estimation camera may be referred to as the first camera, and the captured image generated by the position estimation camera may be referred to as the first captured image.

[0020] The handheld camera 30 captures an image of the hands of a warehouse worker and generates a captured image. The simulation model generation system 1 may include multiple cameras as the handheld camera 30. The handheld camera 30 may be attached to a specific location on the warehouse worker, or to equipment used by the worker while working in the warehouse. For example, the handheld camera 30 may be attached to work clothes worn by the worker so as to capture an image of the worker's hands. Alternatively, the handheld camera 30 may be attached to a cart used by the worker for picking work so as to capture an image of the worker's hands. Hereinafter, the handheld camera may be referred to as the second camera, and the captured image generated by the handheld camera may be referred to as the second captured image.

[0021] Each of the position estimation camera 20 and the handheld camera 30 is configured with at least a lens and an image sensor as optical elements for generating an image. The lens receives light reflected by an object within the angle of view of the area captured by the position estimation camera 20 or the handheld camera 30, and forms an optical image of the object on the light-receiving surface of the image sensor, in other words, on the imaging surface. The image sensor is a solid-state imaging element, such as a charged coupled device (hereinafter referred to as "CCD") or a complementary metal oxide semiconductor (hereinafter referred to as "CMOS"). The image sensor converts the optical image formed on the imaging surface via the lens into an electrical signal at predetermined intervals. For example, if the predetermined interval is 1 / 30 of a second, the frame rate of the sensor 30 is 30 fps. Furthermore, the position estimation camera 20 and the handheld camera 30 may generate captured images by performing predetermined signal processing on the electrical signals at the above-mentioned predetermined time intervals. The captured images generated by the position estimation camera 20 and the handheld camera 30 may include still images and videos.

[0022] The warehouse management system 40 is a system for managing the inventory status of items managed in a warehouse and the transport of items into and out of the warehouse. The warehouse management system may also be referred to as a Warehouse Management System (hereinafter referred to as "WMS"). The warehouse management system 40 may be capable of managing one or more warehouses and may be capable of data communication with computer devices such as personal computers installed in the warehouses. The warehouse management system 40 may be configured using, for example, a general-purpose computer device such as a personal computer or a server computer. The simulation model generation system 1 may be equipped with multiple warehouse management systems.

[0023] The warehouse management system 40 stores and holds layout information for each of the one or more warehouses it manages. The warehouse management system 40 may also store and hold standard data on the time required for work performed in the warehouse. The time required for a certain work, in other words, the standard data on work time, is data defined as the time required for that work. The standard data on work time may be arbitrarily set by the user of the simulation model generation system 1, or may be the average value of actually measured work time.

[0024] The database 41 is configured using a non-volatile storage device and stores various programs and data handled by the warehouse management system 40. The warehouse management system 40 and the database 41 may be configured as an integrated unit.

[0025] The simulation device 50 includes at least a processor 51 and a memory 53. The simulation device 50 may be configured using a general-purpose computer device such as a personal computer or a server computer. The processor 51 realizes various functions of the simulation device 50 by reading various data and programs stored in the memory 53. For example, the processor 51 and the memory 53 work together to realize various functions of the digital twin simulation unit 52, which will be described later. A description of the processor 51 and the memory 53 will be omitted as it overlaps with the description of the processor 11 and the memory 14. The simulation device 50 and the arithmetic device 10 may be configured as an integrated unit. For example, a single computer device may be capable of realizing the various functions of the arithmetic device 10 and the simulation device 50.

[0026] The simulation model generation system 1 recreates a real warehouse in a virtual space using a digital twin, enabling simulations to verify various measures related to the real warehouse. The simulation model of the warehouse used in the digital twin is generated based on information stored in the warehouse management system 40. However, there are cases where the information stored in the warehouse management system 40 alone is not sufficient to generate a highly accurate simulation model. The simulation model generation system 1 acquires the missing data in such cases to generate a more accurate simulation model. For the sake of explanation below, it is assumed that the warehouse work performed by workers in the warehouse is picking work.

[0027] In this embodiment, the simulation will be described assuming that the target of the simulation is a warehouse. However, the present invention is not limited to this, and the target of the simulation may be a work site other than a warehouse, such as a factory or a retail store. In this case, the simulation model generation system 1 may include a management system that stores layout information and the like of the work site, instead of the warehouse management system 40.

[0028] [Simulation Model] Next, various types of information used to generate a simulation model will be described with reference to Fig. 2. Fig. 2 is a schematic diagram for explaining information used to generate a simulation model according to the first embodiment.

[0029] The warehouse management system 40 includes slot information for the warehouses it manages. A slot refers to the smallest unit of storage space for an item. For example, a slot is an individual space separated by a partition or shelf on a shelf installed in a warehouse. Therefore, one shelf may include multiple slots. Each slot in a warehouse is provided with slot information corresponding to that slot, regardless of whether or not an item is stored therein.

[0030] The slot information includes data such as a slot ID, slot position, slot size, pick position, pick order, and pick zone ID. The slot ID is an identifier for the slot, and a different slot ID is assigned to each slot. The slot position indicates the position of the slot in the warehouse and is expressed, for example, using three-dimensional coordinates. The slot size indicates the size of the slot and is expressed, for example, using the distance between two points in three-dimensional coordinate space. The pick position indicates the position when a worker picks an item stored in the slot and is expressed, for example, using three-dimensional coordinates. The pick order indicates the order in which a worker picks items stored in the slot and is expressed, for example, by a number. The worker performs work on slots that have been assigned the same pick zone ID. The work in question is, for example, picking an item placed in the slot.

[0031] In order to generate a simulation model of a warehouse, the simulation device 50 acquires slot information of the warehouse from the warehouse management system 40. However, in order to generate a simulation model of a warehouse, various information is required, such as information about the aisles within the warehouse or information about the walls of the warehouse, in addition to slot information that indicates the positions of shelves in the warehouse. If the warehouse management system 40 does not store various information other than slot information that is necessary for generating a simulation model, the simulation device 50 may generate various information necessary for generating a simulation model based on the acquired slot information.

[0032] The simulation device 50 may perform a data conversion process on the slot information to generate converted slot information, aisle information, wall information, area information, picklists, and inventory information. This is because the data structure of slot information may differ depending on the warehouse management system manufacturer or warehouse. When generating a simulation model, the simulation device 50 generalizes, or in other words, standardizes, the slot information through the data conversion process. This allows the simulation device 50 to generate a simulation model based on the generalized information, regardless of which warehouse management system the acquired slot information is stored in.

[0033] The conversion slot information includes a conversion slot ID, a conversion slot position, a conversion slot size, and a conversion pick position as data. The conversion slot ID is a slot ID that has been anonymized. The anonymization process prevents information about customers and the like from being obtained from the slot ID. An example of a conversion slot ID is an integer value such as "00001." The conversion slot position is a slot position that has been modified for generating a simulation model. The conversion slot size is a representation of the slot size in the distance unit system used in the simulation model. For example, the conversion slot size is expressed in meters. The conversion pick position is a pick position that has been modified in accordance with the modification of the slot position to the conversion slot position.

[0034] The aisle information includes data such as an aisle ID, start and end positions, aisle width, and whether or not there is a one-way restriction. Workers move through aisles that exist within a warehouse to perform work. The aisle ID is an identifier for the aisle. The start and end positions indicate the start and end points of the aisle and are expressed, for example, as two-dimensional coordinates. The aisle width indicates the width of the aisle. Whether or not there is a one-way restriction indicates whether the aisle is one-way.

[0035] The wall information includes data such as a wall ID, start and end point positions, wall height, and visibility. The wall ID is an identifier for a wall that exists within the warehouse. The start and end point positions indicate the positions of the ends corresponding to the horizontal start and end points of the wall, and are expressed, for example, in two-dimensional coordinates. The wall height indicates the height of the wall. The visibility indicates whether the wall is visible.

[0036] The area information further includes three pieces of information: a waiting area, a loading area, and a warehouse area, and each of these three pieces of information includes data on the start and end points. The warehouse area defines the area of ​​the entire warehouse to be simulated that is generated as a simulation model. The waiting area is an area where workers wait before starting work. The loading area is an area where workers perform loading work to ship one or more items picked by the worker from the warehouse to be simulated. The waiting area and loading area are each part of the warehouse area.

[0037] A pick list contains data such as a pick list ID, a conversion slot ID, an item ID, and the number of pick items. Here, an item refers to an item stored in a slot. The pick list contains information about which shelf, that is, which slot, a worker should pick which item from. The worker performs picking work based on the pick list. The pick list ID is an identifier for the pick list. The item ID is an identifier for the item. The number of pick items indicates the quantity of items to be picked.

[0038] The inventory information includes data such as a conversion slot ID, an item ID, and the number of inventory items. The number of inventory items indicates the number of items in stock before the simulation is executed.

[0039] In this embodiment, information related to the warehouse layout, such as slot information, may be referred to as layout information. Conversion slot information, aisle information, wall information, and area information are also layout information. Picklists and inventory information are each information necessary for picking simulations. The digital twin simulation unit 52 of the simulation device 50 can generate a simulation model using the layout information. Furthermore, the digital twin simulation unit 52 can execute a simulation using the generated simulation model using information necessary for picking simulations.

[0040] Furthermore, the digital twin simulation unit 52 may use standard data for the movement speed and task time of each worker in the simulation model. The standard data for the movement speed of each worker is data for the movement speed preset by the user. In this case, the movement speed of each worker in the simulation model will be constant, and the task time for a certain task will be constant regardless of the worker. The movement speed and task time of each worker may be referred to as "person information." When standard data is used for the person information, as described above, the movement speed of each worker in the simulation model will be constant, and the task time for a certain task will be constant regardless of the worker. In this case, the accuracy of the simulation model may be insufficient. However, the movement speed or task time may vary depending on each worker. Therefore, it may be difficult to obtain person information based on the data of each worker in a real warehouse. In this embodiment, the position estimation unit 12 and the task classification unit 13 of the calculation device 10 generate person information based on the data of each worker, and the digital twin simulation unit 52 of the simulation device 50 uses the generated person information to generate a highly accurate simulation model.

[0041] Next, referring to FIG. 3 , an example of a warehouse visualized in a virtual space is shown. FIG. 3 is a schematic diagram illustrating an example of a simulation model 60 according to the first embodiment. The simulation model 60 is a visualization of a warehouse to be simulated in a virtual space. The simulation model 60 includes multiple shelves 61, multiple aisles 62, a waiting area 63, a loading area 64, a warehouse area 65, walls 66, 67, 68, and 69, and a worker 70. In the simulation model 60, the shelf 61 is generated based on slot information. The shelf 61 includes multiple slots, and items are stored in each slot based on inventory information. The aisle 62 is generated based on the aisle information. The waiting area 63, the loading area 64, and the warehouse area 65 are generated based on area information. The walls 66, 67, 68, and 69 are generated to surround the entire warehouse based on the wall information. The worker 70 works based on a picklist during simulation. The worker 70 may be represented by a person or, as shown in FIG. 3 , by a material handling and transport vehicle such as a forklift. The simulation model 60 shown in FIG. 3 is an example and is not limited to this.

[0042] [Overall Processing] Next, the overall processing flow of the simulation model generation system 1 according to the first embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the processing flow of the simulation model generation system 1 according to the first embodiment.

[0043] The simulation device 50 acquires layout information from the warehouse management system 40 (step S80). For example, the simulation device 50 may acquire slot information of the warehouse to be simulated from the warehouse management system 40. The simulation device 50 may then perform a data conversion process on the slot information to generate converted slot information, aisle information, wall information, and area information. The simulation device may further generate information required for the simulation, such as a pick list and inventory information.

[0044] The simulation device 50 generates a simulation model using the layout information acquired from the warehouse management system 40 in step S80 and the personnel information set in the standard data (step S81). For example, the simulation device 50 sets the movement speed of the worker to be reproduced in the simulation and the work time required for the work performed by the worker to certain values.

[0045] The simulation device 50 determines whether the accuracy of the simulation model generated in step S81 is sufficient (step S82). For example, a predetermined standard for the accuracy of the simulation model may be arbitrarily set by a user, such as an administrator of the simulation model generation system 1. The simulation device 50 may then determine whether the accuracy of the simulation model satisfies the predetermined standard. In this case, the simulation device 50 may determine that the accuracy of the simulation model is sufficient if it satisfies the predetermined standard, and that the accuracy is insufficient if it does not. An example of the predetermined standard is that the difference between the actual warehouse throughput and the warehouse throughput resulting from the simulation is within a predetermined range. The predetermined standard may consist of multiple standards. For example, the standard used in the determination process of step S82 may be different from the standard used in the determination process of step S85, which will be described later. Alternatively, the simulation device 50 may accept a user's determination of whether the accuracy of the simulation model is sufficient.

[0046] If the simulation device 50 determines that the accuracy of the simulation model is sufficient (step S82: YES), the simulation model generation system 1 ends this processing flow.

[0047] If the simulation device 50 determines that the accuracy of the simulation model is insufficient (step S82: NO), the calculation device 10 calculates the work time and movement speed of the worker within the work area (step S83). Details of the processing of step S83 will be described later with reference to FIGS. 5 to 7. The work area is one of multiple areas into which a warehouse work site is divided. Note that the division method is not particularly limited, and the work site may be divided into areas of uniform size, or may be divided in accordance with the environment, equipment, etc. of the actual warehouse work site. Hereinafter, the work area may be simply referred to as an area.

[0048] By the process of step S83, human information including the working time and movement speed of the worker in the working area is generated.

[0049] The simulation device 50 generates a simulation model using the layout information acquired from the warehouse management system 40 in step S80 and the personnel information generated in step S83, i.e., the working time and movement speed of the worker within the work area (step S84).

[0050] The simulation device 50 determines whether the accuracy of the simulation model generated in step S84 is sufficient (step S85).

[0051] If the simulation device 50 determines that the accuracy of the simulation model is sufficient (step S85: YES), the simulation model generation system 1 ends this processing flow.

[0052] If the simulation device 50 determines that the accuracy of the simulation model is insufficient (step S85: NO), the calculation device 10 calculates the task time and movement speed of the worker for each task unit (step S86). Details of the processing of step S86 will be described later with reference to FIGS. 9 to 11. For example, if a picking task is broken down into task units, it is broken down into at least an operation of placing an object and an operation of picking up an object. Note that which motion or task constitutes a task unit may be specified, for example, by the user.

[0053] By the process of step S86, human information including the work time and movement speed of the worker for each task unit is generated.

[0054] The simulation device 50 generates a simulation model (step S87) using the layout information acquired from the warehouse management system 40 in step S80 and the personnel information generated in step S86, i.e., the work time and movement speed of each worker for each task unit. Then, the simulation model generation system 1 ends this processing flow.

[0055] Although not shown in the figure, the simulation device 50 may determine whether the accuracy of the simulation model generated in the processing of step S87 satisfies a specified standard. If it is determined that the accuracy of the simulation model does not satisfy the specified standard, the simulation model generation system 1 may repeatedly execute the processing of steps S86 and S87. In other words, the simulation model generation system 1 may repeatedly generate a simulation model using layout information and human information including the task time and movement speed of each worker for each task unit until it is determined that the accuracy of the newly generated simulation model satisfies the specified standard.

[0056] In this way, the simulation model is generated in stages until the accuracy of the simulation model meets a specified standard, in other words, the accuracy of the simulation model is improved in stages.

[0057] [Accuracy Improvement by Position Estimation] Next, with reference to FIGS. 5 to 8, an explanation will be given of an improvement in the accuracy of a simulation model by position estimation. As explained with reference to FIG. 4, the simulation device 50 generates a simulation model using layout information and person information set in the standard data. If the accuracy of the generated simulation model does not meet the specified standard, the calculation device 10 calculates the work time and movement speed of the worker within the work area. At this time, the position estimation unit 12 estimates the position of the worker at the work site based on the image captured by the position estimation camera 20. The work time and movement speed of the worker within the work area are calculated based on the estimation result. Details will be explained below.

[0058] FIG. 5 is a schematic diagram for explaining the creation of person information using the position estimation unit 12 according to the first embodiment.

[0059] In the example of Fig. 5, the work site 90 is represented by CAD data. The work site may be represented by data other than CAD data. The work site 90 is divided into multiple work areas. In other words, the work site 90 includes multiple work areas. For example, the work site 90 includes work area 4N and work area 5N.

[0060] The routes C1 and C2 in the work area 5N, and the route C3 that straddles the work area 4N and the work area 5N, are routes taken by the worker, or more precisely, the transition of the worker's estimated position. The position estimation unit 12 estimates the position of the worker at the work site based on images of the work site captured by the position estimation camera 20. Position estimation may be achieved by a known technique such as Visual Simultaneous Localization and Mapping (hereinafter referred to as "VSLAM").

[0061] The calculation device 10 estimates that a worker is working when the worker is moving within a single work area, for example, as in route C1 or route C2, in other words, when the worker's position estimated over a predetermined time period remains within a single work area. On the other hand, the calculation device 10 estimates that a worker is moving when the worker is moving across work areas, for example, as in route C3, in other words, when the worker's position estimated over a predetermined time period spans multiple work areas. Based on the estimation results, the calculation device 10 can calculate the worker's movement speed and the worker's working time within the work area.

[0062] Fig. 6 is a schematic diagram for explaining the flow of the log data creation process according to embodiment 1. Fig. 6 explains an example of the estimation result of the position of a worker in a work site 91 including work area 4N, work area 5N, and work area 6N.

[0063] For the sake of explanation, the worker's position is represented using a three-dimensional coordinate system (not shown) consisting of an X axis, a Y axis, and a Z axis. In the example of FIG. 6 , the worker's position is estimated in the following order: (0,0,0), (0,6,0), (2,7,0), (0,8,0), (3,8,0), and (3,13,0). In other words, it is estimated that the worker moved in the following order: (0,0,0), (0,6,0), (2,7,0), (0,8,0), (3,8,0), and (3,13,0). Note that the worker's position is estimated at a predetermined interval. In the example of FIG. 6 , the predetermined interval is 10 seconds.

[0064] The estimated results of the worker's location, etc., are saved as log data. Note that in this specification, saving is synonymous with storing and retaining. The log data includes the following items: time, location, area, worker ID, and task category. The time indicates the time the log data was created, in other words, the estimated time of the worker's location. The location indicates the coordinates of the estimated worker. The area indicates the estimated location of the worker, or more precisely, the area that includes the coordinates. The worker ID is an identifier for identifying the worker. Here, the task category indicates either "work" or "travel." For example, log data LD1 indicates the time: 10:00:00 (10:00:00), the location: (0,0,0), the area: Area 6N, the worker ID: 001, and the task category: "work."

[0065] The flow of the process for creating log data for a worker whose worker ID is 001 will be described. When creating the log data, the processor 11 of the computing device 10 sets the current time in the time field of the log data (step S100). In describing each process in the flowchart shown in FIG. 6, the creation of log data LD2 will be taken as an example. In step S100, the processor 11 sets 10:00:10 (10 hours, 00 minutes, 10 seconds) in the time field of the log data LD2.

[0066] The worker ID item may be set in advance in the log data, or, although not shown in the figure, a process for setting the worker ID item may be performed separately.

[0067] The processor 11 sets the result of the position estimation by the position estimation unit 12 in the position field of the log data (step S101). For example, the processor 11 sets (0, 6, 0) in the position field of the log data LD2.

[0068] The processor 11 sets the area corresponding to the estimated position in the area field of the log data (step S102). For example, the processor 11 sets 5N in the area field of the log data LD2.

[0069] The processor 11 determines whether the area set in step S102 has changed from the area of ​​the previous log data (step S103). The previous log data refers to the log data created immediately before the log data currently being created or processed. For example, the previous log data of log data LD2 is log data LD1.

[0070] If the processor 11 determines that the area set in step S102 has changed from the area of ​​the previous log data (step S103: YES), it sets "move" in the task classification item (step S104).

[0071] If the processor 11 determines that the area set in step S102 has not changed from the area of ​​the previous log data (step S103: NO), it sets the task in the task classification item (step S105).

[0072] For example, the processor 11 compares the area 5N in the log data LD2 set in step S102 with the area 6N indicated by the previous log data LD1 and determines that the area has changed. In this case, the processor 11 sets “move” in the task classification item of the log data LD2.

[0073] The processor 11 saves the setting contents (step S106), and then ends this processing flow, thereby creating the log data LD2.

[0074] The processor 11 creates log data LD3 at 10:00:20. When creating the log data LD3, 5N is set in the area field. Because the area indicated by the log data LD2 is 5N, work is set in the work classification field of the log data LD3.

[0075] In addition, in the processing of step S103, if there is no previous log data to refer to, such as when the processor 11 is creating log data LD1, the processor 11 may set the work classification to work, as in the example of Figure 6.

[0076] In this manner, log data LD1, log data LD2, log data LD3, log data LD4, log data LD5, and log data LD6 are created at predetermined intervals.

[0077] Next, a flow of processing in which the arithmetic device 10 creates personal information based on log data will be described with reference to Fig. 7. Fig. 7 is a schematic diagram for explaining the flow of processing in which personal information is created according to the first embodiment.

[0078] The processor 11 of the computing device 10 reads log data for a specific worker ID (step S110). Here, an example will be described in which the processor 11 reads the log data for worker ID 001, i.e., log data LD1, log data LD2, log data LD3, log data LD4, log data LD5, and log data LD6.

[0079] The processor 11 repeats the processes from step S112 onwards for all lines of the log data (step S111).

[0080] The processor 11 determines whether the task category is travel (step S112). That is, the processor 11 determines whether the task category is travel or work. For the sake of explanation, an example will be given in which the processor 11 executes the processes from step S112 onward on the log data LD2. In step S112, the processor 11 determines that the task category of the log data LD2 is travel.

[0081] If the processor 11 determines that the task category is movement (step S112: YES), it calculates the movement speed from the difference in time and the difference in position indicated by the previous log data and the current log data (step S113). The processor 11 calculates the difference between the time of log data LD1 and the time of log data LD2 to be 10 seconds. The processor 11 also calculates the difference between the position of log data LD1 and the position of log data LD2 to be 6 m. The processor 11 then calculates the movement speed to be 0.6 m / s from the calculated difference in time and difference in position.

[0082] The processor 11 saves the worker ID and the movement speed calculated in step S113 (step S114). As a result, for example, movement speed data SD1 is saved. The movement speed data SD1 indicates that the movement speed of the worker with worker ID 001 is 0.6 m / s. The processor 11 returns to step S112 and executes the processes from step S112 onwards using the log data LD3.

[0083] If processor 11 determines that the task category is not travel (step S112: NO), it determines whether the task category of the previous log data is travel or not (step S115). That is, if processor 11 determines that the task category is work, it determines whether the task category of the previous log data is travel or work. Here, an explanation will be given using log data LD3 as an example. In step S112, processor 11 determines that the task category of log data LD3 is not travel. Processor 11 proceeds to step S115 and determines whether the task category of the previous log data, i.e., log data LD2, is travel or work.

[0084] If the processor 11 determines that the task classification of the previous log data is "travel" (step S115: YES), it retains the time of the current log data as the task start time (step S116). The current log data refers to the log data currently being processed. If the log data LD3 is the current log data, the log data LD2 is the previous log data. If the processor 11 determines that the task classification of the log data LD2 is "travel," it retains the time of the log data LD3, 10:00:20 (10 hours, 00 minutes, 20 seconds), as the task start time.

[0085] If the processor 11 determines that the task classification of the previous log data is not movement (step S115: NO), the processor 11 proceeds to step S117.

[0086] The processor 11 determines whether the task classification of the next log data is "movement" (step S117). The next log data refers to the log data created immediately after the log data being created or processed. For example, the next log data for log data LD3 is log data LD4. Continuing with the explanation, we will use the processing of log data LD3 as an example. The task classification of log data LD4, which is the next log data for log data LD3, is "work," not "movement."

[0087] If the processor 11 determines that the task classification of the next log data is not movement (step S117: NO), the processor 11 returns to step S112 and executes the processes from step S112 onward with the next log data.

[0088] When processor 11 has executed the processes from step S112 onward with log data LD4, it proceeds to the processes of step S112, step S115, and step S117 in that order. In step S117, processor 11 determines that the task classification of the next log data, i.e., log data LD5, is not "movement." Then, processor 11 returns to step S112 and executes the processes from step S112 onward with log data LD5, which is the next log data.

[0089] When the processor 11 has executed the processes from step S112 onward on the log data LD5, the processor 11 proceeds to the process in the order of step S112, step S115, and step S117. In step S117, the processor 11 determines that the task classification of the next log data, i.e., the log data LD6, is movement.

[0090] If the processor 11 determines that the task classification of the next log data is travel (step S117: YES), it subtracts the task start time stored in step S116 from the time of the next log data to calculate the task time (step S118). The processor 11 subtracts 10:00:20 (10:00:20), which is the task start time stored, from 10:00:50 (10:00:50), which is the time of the log data LD6, to calculate a task time of 30 seconds.

[0091] The processor 11 saves the worker ID, the area, and the work time calculated in step S118 (step S119). This saves, for example, work time data TD1. The work time data TD1 indicates that the worker with worker ID 001 required 30 seconds to work in area 5N. The processor 11 returns to step S112 and executes the processes from step S112 onward using the log data LD6.

[0092] When the processor 11 has executed the processes from step S112 onward on the log data LD6, the processor 11 advances the process to step S112, step S113, and step S114 in that order. Then, the processor 11 generates the moving speed data SD2.

[0093] When the processor 11 has completed the process for all lines of the log data, the processor 11 ends the repetition of the process (step S120), and then ends this processing flow.

[0094] In addition, if the processor 11 executes the processing from step S112 onward using the first log data, such as log data LD1, for which there is no previous log data, the processing proceeds to step S112, step S115, and step S117 in that order, and then returns to step S112.

[0095] In this way, the processor 11 calculates the worker's movement speed and the work time in each area based on the log data. By accumulating movement speed data, for example, a probability distribution P1 is obtained. The probability distribution P1 represents the relationship between the movement speed of the worker with worker ID 001 and the frequency of movement at a specific movement speed. Furthermore, by accumulating work time data for each area, for example, a probability distribution P2 is obtained. The probability distribution P2 represents the relationship between the work time when the worker with worker ID 001 works in area 5N and the frequency of the specific work time. Data obtained using position estimation, such as movement speed data or work time data, may be collectively referred to as estimated data. By setting human information using estimated data instead of standard data, the simulation device 50 can generate a simulation model with high accuracy.

[0096] FIG. 8 is a schematic diagram for comparing the workflow in a conventional digital twin with the workflow in a digital twin according to the first embodiment.

[0097] In a simulation model constructed for a conventional digital twin, personnel information is set using standard data. For illustrative purposes, a case will be taken as an example in which a workflow from step S130 to step S133 is reproduced in a conventional digital twin. In this workflow, a worker works in work area 5N (step S130), moves from work area 5N to work area 4N (step S131), works in work area 4N (step S132), and moves from work area 4N to another area (step S133). The worker's movement speed when reproducing steps S131 and S133 is set based on the standard data. Furthermore, the worker's work time when reproducing steps S130 and S132 is set based on the standard data. Therefore, differences in movement speed between workers and differences in work time between workers' areas are not taken into account.

[0098] In a simulation model constructed for a digital twin using position estimation according to the first embodiment, person information is set using estimated data. Take, as an example, a case in which a workflow from steps S130A to S133A, which has the same content as the workflow from steps S130 to S133, is reproduced in a digital twin using position estimation. The worker's work time when reproducing step S130A is set based on probability distribution P3. Probability distribution P3 represents the relationship between the work time when the worker works in area 5N and the frequency at which a specific work time is required. Furthermore, the worker's work time when reproducing step S132A is set based on probability distribution P4. Probability distribution P4 represents the relationship between the work time when the worker works in area 4N and the frequency at which a specific work time is required. This allows the worker's work to be reproduced with higher accuracy. Furthermore, the worker's movement speed when reproducing steps S131A and S133A is set based on probability distribution P5. Probability distribution P5 represents the relationship between the worker's movement speed and the frequency at which the worker moves at a specific movement speed. This allows the movement of the worker to be reproduced with higher accuracy.

[0099] In the description of the first embodiment, an example is shown in which a simulation model is generated using estimated data for both the worker's working time and travel time, but this is not limiting. For example, estimated data may be used for either the worker's working time or travel time, and standard data prepared in advance may be used for the other of the worker's working time and travel time for which estimated data is not used.

[0100] In this way, the simulation device 50 connects to the warehouse management system 40, which stores layout information of the work site, and acquires the layout information. The calculation device 10 then estimates the position of the worker at the work site based on the first captured image captured by the first camera that captures the work site. The calculation device 10 then calculates at least one of the worker's movement speed and the worker's working time in an area of ​​the work site corresponding to the position based on the estimation result. The simulation device 50 then generates a simulation model of the work site based on the layout information and at least one of the movement speed and the worker's working time in each of one or more areas of the work site.

[0101] The calculation device 10 may also estimate the location at a predetermined cycle. Based on the estimation result, the calculation device 10 may determine the worker's task classification as either work in a specific area or movement from the specific area to a different area. Based on the determination result and the predetermined cycle, the calculation device 10 may then calculate the movement speed and the worker's task time in the area.

[0102] Furthermore, the simulation device 50 may determine whether the accuracy of the simulation model generated using the position estimation satisfies a specified standard. If the simulation device 50 determines that the simulation model does not satisfy the specified standard, the calculation device 10 and the simulation device 50 respectively execute the processes of steps S86 and S87 of the flowchart shown in FIG. 4. The following description will be made with reference to FIGS. 9 to 12. Note that in the description of FIGS. 9 to 12, portions that overlap with the description of FIGS. 5 to 8 will be omitted or simplified.

[0103] [Accuracy Improvement by Position Estimation and Task Classification] FIG. 9 is a schematic diagram for explaining the creation of person information using the position estimation unit 12 and task classification unit 13 according to the first embodiment.

[0104] As described with reference to FIG. 5 , the position estimation unit 12 estimates the position of the worker at the work site based on an image of the work site captured by the position estimation camera 20. In the example of FIG. 9 , the task classification unit 13 classifies the task being performed by the worker based on an image of the worker's hands captured by the handheld camera 30. For example, the memory 14 of the computing device 10 may pre-store an image of the worker's hands performing a specific task. The task classification unit 13 may compare the image acquired from the handheld camera 30 with the pre-stored image to classify the task being performed by the worker. Note that the method used by the task classification unit 13 to classify tasks is not limited to this, and any known technology may be used.

[0105] In the example of Figure 9, the task classification unit 13 classifies the task performed by the worker as a task of picking up an object while the worker is located on path C1. Furthermore, the task classification unit 13 classifies the task performed by the worker as a task of placing an object while the worker is located on path C2. Furthermore, the task classification unit 13 classifies the task performed by the worker as a task of moving while the worker is located on path C3.

[0106] The computing device 10 can calculate the movement speed of the worker and the task time of each task unit of the worker based on the position estimation result by the position estimation unit 12 and the task classification result by the task classification unit 13. Here, each task unit is the task of picking up an object and the task of putting an object down.

[0107] 10 is a schematic diagram for explaining the flow of the log data creation process according to embodiment 1. As in the explanation with reference to FIG. 6, an explanation will be given taking as an example the result of estimating the position of a worker in a work site 91 including work area 4N, work area 5N, and work area 6N.

[0108] The results of estimating the worker's position and the results of the task classification are saved as log data. When the task classification unit 13 classifies tasks, the task classification item in the log data indicates either a "task unit" such as picking up or putting down an object, or "movement." For example, log data LD7 indicates that the time is 10:00:00 (10:00:00), the location is (0,0,0), the area is Area 6N, the worker ID is 001, and the task classification is "picking up an object." Note that the worker's position is estimated and the task classification is performed at a predetermined interval. In the example of FIG. 10, the predetermined interval is 10 seconds.

[0109] When creating the log data, the processor 11 of the computing device 10 sets the current time in the time field of the log data (step S100).

[0110] The processor 11 sets the result of the position estimation by the position estimation unit 12 in the position field of the log data (step S101).

[0111] The processor 11 sets the area corresponding to the estimated location in the area field of the log data (step S102).

[0112] The processor 11 sets the result of the task classification by the task classification unit 13 in the task classification item of the log data (step S140).

[0113] The processor 11 saves the setting contents (step S106), and then ends this processing flow, thereby creating log data.

[0114] In this manner, log data LD7, log data LD8, log data LD9, log data LD10, log data LD11, and log data LD12 are created at predetermined intervals.

[0115] Next, a process flow in which the arithmetic device 10 creates personal information based on log data will be described with reference to Fig. 11. Fig. 11 is a schematic diagram for explaining the process flow in which personal information is created according to the first embodiment.

[0116] The processor 11 of the computing device 10 reads log data for a specific worker ID (step S110). Here, an example will be described in which the processor 11 reads the log data for worker ID 001, i.e., log data LD7, log data LD8, log data LD9, log data LD10, log data LD11, and log data LD12.

[0117] The processor 11 repeats the process from step S112 onwards for all lines of the log data (step S150). First, an example will be described in which the processor 11 executes the process from step S112 on the log data LD7.

[0118] The processor 11 determines whether the task classification is movement (step S112). The processor 11 determines that the task classification of the log data LD7 is not movement.

[0119] If processor 11 determines that the task classification is not movement (step S112: NO), processor 11 determines whether the task classification of the current log data is the same as the task classification of the previous log data (step S151). Because log data LD7 is the first log data, processor 11 determines that the task classification of the current log data is not the same as the task classification of the previous log data.

[0120] If processor 11 determines that the task classification of the current log data is not the same as the task classification of the previous log data (step S151: NO), processor 11 stores the time of the current log data as the task start time (step S116). Processor 11 stores the time of 10:00:00 (10:00:00) of log data LD7 as the task start time of the task of picking up an object.

[0121] Processor 11 determines whether the task classification of the current log data is the same as the task classification of the next log data (step S152). Because the task classification of log data LD8, which is the next log data of log data LD7, is movement, processor 11 determines that the task classification of the current log data is not the same as the task classification of the next log data.

[0122] If processor 11 determines that the task classification of the current log data is not the same as the task classification of the next log data (step S152: NO), processor 11 subtracts the time stored as the task start time of the task of picking up the object from the time of the next log data to calculate the task time (S118). Processor 11 subtracts 10:00:00 (10:00:00 seconds), which is the time stored as the task start time of the task of picking up the object, from 10:00:10 (10:00:10 seconds), which is the time of log data LD8, to calculate the task time of the task of picking up the object as 10 seconds.

[0123] Processor 11 saves the worker ID, the area, and the work time calculated in step S118 (step S119). As a result, for example, work time data TD2 is saved. Work time data TD2 indicates that the worker with worker ID 001 took 10 seconds to retrieve the object in area 6N. Processor 11 returns to step S112 and executes the processes from step S112 onward using log data LD8.

[0124] In step S112, the processor 11 determines that the task classification of the log data LD8 is travel.

[0125] If processor 11 determines that the task category is movement (step S112: YES), it calculates the movement speed from the time difference and position difference indicated by the previous log data and the current log data (step S113). Processor 11 calculates the difference between the time of log data LD7 and the time of log data LD8 to be 10 seconds. Processor 11 also calculates the difference between the position of log data LD7 and the position of log data LD8 to be 6 m. Processor 11 then calculates the movement speed to be 0.6 m / s from the calculated time difference and position difference.

[0126] The processor 11 saves the worker ID and the movement speed calculated in step S113 (step S114). As a result, for example, movement speed data SD3 is saved. The movement speed data SD3 indicates that the movement speed of the worker with worker ID 001 is 0.6 m / s. The processor 11 returns to step S112 and executes the processes from step S112 onward using the log data LD9.

[0127] Processor 11 proceeds with the process in the order of step S112, step S151, step S116, step S152, step S118, and step S119. Although not shown in the figure, this saves work time data indicating that the worker with worker ID 001 required 10 seconds to place an object in area 5N. Processor 11 returns to step S112 and executes the processes from step S112 onwards using log data LD10.

[0128] Processor 11 proceeds with the process in the order of step S112, step S151, step S116, and step S152. Because the task classification of log data LD10 and the task classification of log data LD11 are both picking up an object, processor 11 determines in step S152 that the task classification of the current log data is the same as the task classification of the next log data.

[0129] If the processor 11 determines that the task classification of the current log data is the same as the task classification of the next log data (step S152: YES), the processor 11 returns to step S112 and executes the processes from step S112 onward on the next log data. The processor 11 returns to step S112 and executes the processes from step S112 onward on the log data LD11.

[0130] Processor 11 proceeds with the process in the order of step S112 and step S151. Because the task classification of log data LD10 and the task classification of log data LD11 are both picking up an object, processor 11 determines in step S151 that the task classification of the current log data is the same as the task classification of the previous log data.

[0131] If processor 11 determines that the task category of the current log data is the same as the task category of the previous log data (step S151: YES), processor 11 proceeds to step S152. Because the task category of log data LD11 is “picking up an object” and the task category of log data LD12 is “moving,” processor 11 determines in step S152 that the task category of the current log data is not the same as the task category of the next log data.

[0132] In step S118, processor 11 subtracts the time stored as the work start time from 10:00:50 (10:00 minutes 50 seconds), which is the time of the next log data, log data LD12. The time stored as the work start time is 10:00:30 (10:00 minutes 30 seconds), which is the time of log data LD10. As a result, the work time for the work of picking up the object is calculated to be 20 seconds.

[0133] In step S119, processor 11 saves the worker ID, area, and work time. As a result, work time data TD3 is saved. The work time data TD3 indicates that the worker with worker ID 001 took 20 seconds to retrieve the object in area 5N. Processor 11 returns to step S112 and executes the processes from step S112 onward using log data LD12.

[0134] When the processor 11 has executed the processes from step S112 onward using the log data LD12, the processor 11 advances the process to step S112, step S113, and step S114 in that order. Then, the processor 11 generates moving speed data SD4.

[0135] When the processor 11 has completed the process for all lines of the log data, the processor 11 ends the repetition of the process (step S153), and then ends this processing flow.

[0136] In this way, the processor 11 calculates the worker's movement speed and the task time for each task unit in each area based on the log data. By accumulating the movement speed data, for example, a probability distribution P6 is obtained. The probability distribution P1 represents the relationship between the movement speed of the worker with worker ID 001 and the frequency of movement at a specific movement speed. Furthermore, by accumulating the task time data for each task unit in each area, for example, a probability distribution P7 is obtained. The probability distribution P7 represents the relationship between the task time when the worker with worker ID 001 performs the task of picking up an object in area 5N and the frequency at which the specific task time is required. Note that by accumulating the task time data, a probability distribution that does not distinguish between areas (for example, the probability distribution P8 shown in FIG. 12 ) may also be obtained. For example, if the task unit is the task of picking up an object, the probability distribution that does not distinguish between areas represents the relationship between the task time when the worker performs the task of picking up an object and the frequency at which the specific task time is required.

[0137] By setting human information using estimated data obtained using position estimation and task classification, the simulation device 50 can generate a simulation model with higher accuracy than if human information were set using estimated data obtained using position estimation.

[0138] FIG. 12 is a schematic diagram for comparing workflows in a digital twin according to the first embodiment.

[0139] FIG. 12 shows a schematic diagram of a case where the workflow from step S130A to step S133A is reproduced in a digital twin using position estimation, similar to that shown in FIG. 8 .

[0140] Here, we will take an example where a digital twin using position estimation and task classification reproduces a workflow from step S130B to step S133B with the same content as the workflow from step S130A to step S133A.

[0141] In step S130B, when working in work area 5N, the worker performs tasks A, B, and C. Tasks A, B, and C are each a task unit. The work times for tasks A, B, and C are set based on probability distributions P8, P9, and P10. Probability distribution P8 represents the relationship between the work time when the worker performs task A and the frequency at which a specific task takes place. Probability distribution P9 represents the relationship between the work time when the worker performs task B and the frequency at which a specific task takes place. Probability distribution P9 represents the relationship between the work time when the worker performs task C and the frequency at which a specific task takes place. This allows the work of each task unit by the worker to be reproduced with higher accuracy.

[0142] Furthermore, for example, if the task configuration in task area 5N changes from task A, task B, and task C to task A and task D, the task time for task D is set based on probability distribution P11. Probability distribution P11 represents the relationship between the task time when a worker performs task D and the frequency with which a specific task takes time. In this way, by distinguishing the task time in a task area by task, it becomes possible to reproduce the tasks in that task area with high accuracy even if the tasks in that task area change.

[0143] Although detailed description will be omitted, the work in work area 4N in step S132B is also reproduced with high accuracy. The work time for each work unit may be differentiated for each work area. For example, if the work in work area 4N consists of work A and work B, the work times for work A and work B may be set based on probability distributions P8 and P9, respectively, or may be set based on a probability distribution of the work times for work A and work B limited to work area 4N.

[0144] Furthermore, the movement speed of the worker during reproduction in steps S131B and S133B is set based on a probability distribution P12. The probability distribution P12 represents the relationship between the movement speed of the worker and the frequency of movement at a specific movement speed. This allows the movement of the worker to be reproduced with higher accuracy than when the movement of the worker is reproduced based on standard data.

[0145] In the description of the first embodiment, an example is shown in which a simulation model is generated using estimated data for both the work time and travel time of each task unit of a worker, but this is not limiting. For example, estimated data may be used for either the work time or travel time of each task unit of a worker, and pre-prepared standard data may be used for the work time or travel time of each task unit of a worker for which estimated data is not used.

[0146] In this way, by setting human information using the position estimation results, the simulation model generation system 1 can generate a simulation model with higher accuracy than when setting human information using standard data. Furthermore, by setting human information using the position estimation results and the task classification results, the simulation model generation system 1 can generate a simulation model with higher accuracy than when setting human information using the position estimation results. This makes it possible to gradually improve the accuracy of the simulation model, for example, depending on the accuracy desired by the user.

[0147] In this way, if the simulation device 50 determines that the accuracy of the simulation model generated using the position estimation result does not satisfy a predetermined standard, the calculation device 10 may classify the work performed by the worker based on a second captured image captured by a second camera that captures an image of the worker's hands.The calculation device 10 may then calculate the work time for each task performed by the worker in the area based on the position estimation result and the task classification result.The simulation device 50 may then generate a simulation model based on layout information and at least one of the movement speed and the work time for each task.

[0148] Furthermore, the simulation device 50 may generate a simulation model based on layout information and at least one of the movement speed and the task time for each task unit in each of one or more areas.

[0149] 4, if the simulation device 50 determines that the accuracy of the simulation model is insufficient (step S82: NO), the processes of steps S83, S84, and S85 may be omitted, and the process may proceed to step S86. In other words, if the accuracy of the simulation model generated using the person information set in the standard data is insufficient, the accuracy improvement by position estimation and task classification described with reference to FIGS. 9 to 12 may be performed without performing the accuracy improvement by position estimation described with reference to FIGS. 5 to 8. Such settings for improving accuracy by position estimation and task classification without going through the stage of improving accuracy by position estimation may be performed in advance by the user.

[0150] In this manner, the simulation device 50 connects to the warehouse management system 40, which stores layout information of the work site, and acquires the layout information. The calculation device 10 then estimates the position of the worker at the work site based on a first captured image captured by a first camera that captures the work site. The calculation device 10 also classifies the work performed by the worker based on a second captured image captured by a second camera that captures the worker's hands. The calculation device 10 then calculates at least one of the worker's movement speed and the work time for each work unit performed by the worker in the area of ​​the work site corresponding to the position based on the estimation result and the classification result. The simulation device 50 may then generate a simulation model of the work site based on the layout information and at least one of the movement speed and the work time for each work unit performed by the worker in the area of ​​the work site.

[0151] (Summary of First Embodiment) The above description of First Embodiment discloses at least the following techniques. Note that, in parentheses, examples of corresponding components in First Embodiment are shown, but the present invention is not limited to these.

[0152] (Technology 1) A simulation model generation method connects to a management system that stores layout information of a work site, acquires the layout information from the management system, estimates the position of a worker at the work site based on a first image captured by a first camera that images the work site, calculates at least one of the worker's movement speed and the worker's working time in an area of ​​the work site that corresponds to the position based on the estimation result, and generates a simulation model of the work site based on the layout information and at least one of the movement speed and the worker's working time in each of one or more areas of the work site.

[0153] This makes it possible to generate a highly accurate simulation model for managing a work site. The simulation model generation method generates data on the movement speed or work time of a worker using position estimation technology, and generates a simulation model using that data, thereby improving the accuracy of a simulation model generated by setting, for example, the movement speed and work time of a worker using standard data.

[0154] (Technology 2) In the simulation model generation method according to Technology 1, the simulation model generation method may determine whether or not the accuracy of the simulation model satisfies a specified standard.

[0155] This allows a determination as to whether or not the accuracy of the simulation model desired by the user has been achieved.

[0156] (Technology 3) In the simulation model generation method described in Technology 2, if it is determined that the accuracy of the simulation model does not satisfy a standard, the simulation model generation method may classify the work performed by the worker based on a second image captured by a second camera that captures an image of the worker's hands, calculate the work time for each work unit performed by the worker in the area based on the estimation result and the classification result, and generate a simulation model based on the layout information, the movement speed, and the work time for each work unit.

[0157] As a result, if the accuracy of the simulation model desired by the user is not achieved, the simulation model generation method can improve the accuracy of the simulation model by classifying the work performed by workers, calculating the work time for each classified work unit, and using the calculated work time for each work unit.

[0158] (Technology 4) In the simulation model generation method described in Technology 3, the simulation model may be generated based on layout information, movement speed, and task time for each task unit in each of one or more areas.

[0159] This allows the simulation model generation method to generate a simulation model with higher accuracy using the task time for each task unit in a specific area, thereby improving the accuracy of the simulation model.

[0160] (Technology 5) In the simulation model generation method described in any one of Technologies 1 to 4, the location estimation is performed at a predetermined cycle, and the simulation model generation method may determine the task classification of the worker as either work in a specific area or movement from the specific area to an area different from the specific area based on the estimation result, and calculate the movement speed and the task time of the worker in the area based on the determination result and the cycle.

[0161] As a result, the simulation model generation method can calculate the movement speed of the worker and the working time of the worker in the area using a predetermined cycle.

[0162] (Technology 6) In the simulation model generation method described in Technology 3, if the accuracy of the work site simulation model generated based on the layout information, the movement speed, and the work time for each work unit does not satisfy a standard, the simulation model generation method may repeat the calculation of the movement speed, the calculation of the work time for each work unit, and the generation of the work site simulation model based on the layout information, the movement speed, and the work time for each work unit until it is determined that the accuracy of the newly generated work site simulation model satisfies the standard.

[0163] As a result, for example, if the accuracy of the generated simulation model is lower than the accuracy desired by the user, the calculation of the worker's movement speed and the work time for each task unit and the generation of a simulation model using the calculation results can be repeated until the accuracy desired by the user is achieved.

[0164] (Technology 7) Connect to a management system that stores layout information of the work site, acquire the layout information from the management system, estimate the position of the worker at the work site based on a first image taken by a first camera that takes an image of the work site, classify the work performed by the worker based on a second image taken by a second camera that takes an image of the worker's hands, calculate at least one of the worker's movement speed and the work time for each work unit performed by the worker in the area based on the estimation result and the classification result, and generate a simulation model of the work site based on the layout information and at least one of the movement speed and the work time for each work unit.

[0165] This makes it possible to generate highly accurate simulation models for managing work sites. The simulation model generation method generates data on the movement speed of workers or the work time for each task unit using position estimation technology and task classification technology, and generates a simulation model using this data, thereby improving the accuracy of a simulation model generated by setting, for example, the movement speed of workers and the work time for each task unit using standard data.

[0166] (Technology 8) A simulation model generation program causes a computer connected to a management system that stores layout information of a work site to execute the following processes: acquire the layout information from the management system; estimate the position of a worker at the work site based on a first image captured by a first camera that captures images of the work site; calculate at least one of the worker's movement speed and the worker's working time in an area of ​​the work site that corresponds to the position based on the estimation result; and generate a simulation model of the work site based on the layout information and at least one of the movement speed and the worker's working time in each of one or more areas of the work site.

[0167] As a result, the simulation model generation program can obtain the same effect as that of Technique 1.

[0168] (Technology 9) A simulation model generation system includes a management system that stores layout information of a work site, and a computing device connected to the management system. The computing device acquires the layout information from the management system, estimates the position of a worker at the work site based on a first image captured by a first camera that captures images of the work site, calculates at least one of the worker's movement speed and the worker's working time in an area of ​​the work site that corresponds to the position based on the estimation result, and generates a simulation model of the work site based on the layout information and at least one of the movement speed and the worker's working time in each of one or more areas of the work site.

[0169] As a result, the simulation model generation system can obtain the same effect as that of Technique 1.

[0170] The functions of the above-described embodiments can also be realized by supplying programs and applications for realizing the functions of the above-described embodiments to a system or device using a network or storage medium, etc., and having one or more processors in the computer of that system or device read and execute the programs.

[0171] Furthermore, the functions of the above-described embodiments may be realized by a circuit that realizes one or more functions (for example, an Application Specific Integrated Circuit (hereinafter referred to as "ASIC") or an FPGA).

[0172] Although the embodiments of the present disclosure have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the above-described embodiments may be combined in any manner as long as they do not deviate from the spirit of the invention.

[0173] This application is based on a Japanese patent application (Patent Application No. 2024-081893) filed on May 20, 2024, the contents of which are incorporated herein by reference.

[0174] The present disclosure is useful as a simulation model generation method, a simulation model generation program, and a simulation model generation system.

[0175] 1 Simulation model generation system 10 Arithmetic device 11, 51 Processor 12 Position estimation unit 13 Work classification unit 14, 53 Memory 15, 41 Database 20 Position estimation camera 30 Handheld camera 40 Warehouse management system 50 Simulation device 52 Digital twin simulation unit 60 Simulation model 90, 91 Work site 4N, 5N, 6N Work area (area)

Claims

1. A simulation model generation method comprising: connecting to a management system that stores layout information of a work site; acquiring the layout information from the management system; estimating a position of a worker at the work site based on a first image captured by a first camera that images the work site; calculating at least one of the worker's movement speed and the worker's working time in an area of ​​the work site corresponding to the position based on the estimation result; and generating a simulation model of the work site based on the layout information and at least one of the worker's movement speed and the worker's working time in each of one or more areas of the work site.

2. The simulation model generating method according to claim 1, further comprising determining whether the accuracy of the simulation model satisfies a specified standard.

3. The simulation model generation method according to claim 2, wherein, if it is determined that the accuracy of the simulation model does not satisfy the standard, the work performed by the worker is classified based on a second image captured by a second camera that captures an image of the worker's hands, the work time for each task unit performed by the worker in the area is calculated based on the estimation result and the classification result, and the simulation model is generated based on the layout information, the movement speed, and the work time for each task unit.

4. The simulation model generating method according to claim 3, wherein the simulation model is generated based on the layout information, the movement speed, and the task time for each task unit in each of the one or more areas.

5. The simulation model generation method according to claim 1, wherein the location estimation is performed at a predetermined cycle, and based on the estimation result, the task classification of the worker is determined as either work in a specific area or movement from the specific area to an area different from the specific area, and based on the determination result and the cycle, the movement speed and the working time of the worker in the area are calculated.

6. The simulation model generation method according to claim 3, wherein, if the accuracy of the simulation model of the work site generated based on the layout information, the movement speed, and the work time for each work unit does not satisfy the standard, the calculation of the movement speed, the calculation of the work time for each work unit, and the generation of the simulation model of the work site based on the layout information, the movement speed, and the work time for each work unit are repeated until it is determined that the accuracy of the newly generated simulation model of the work site satisfies the standard.

7. A simulation model generation method comprising: connecting to a management system that stores layout information of a work site; acquiring the layout information from the management system; estimating a position of a worker at the work site based on a first image taken by a first camera that images the work site; classifying work performed by the worker based on a second image taken by a second camera that images the worker's hands; calculating at least one of a movement speed of the worker and a work time for each work unit performed by the worker in the area based on the estimation result and the classification result; and generating a simulation model of the work site based on the layout information and at least one of the movement speed and the work time for each work unit.

8. A simulation model generation program that causes a computer connected to a management system that stores layout information of a work site to execute the following processes: acquiring the layout information from the management system; estimating the position of a worker at the work site based on a first image captured by a first camera that images the work site; calculating at least one of the worker's movement speed and the worker's working time in an area of ​​the work site that corresponds to the position based on the estimation result; and generating a simulation model of the work site based on the layout information and at least one of the worker's movement speed and the worker's working time in each of one or more areas of the work site.

9. A simulation model generation system comprising: a management system that holds layout information of a work site; and a computing device connected to the management system, wherein the computing device acquires the layout information from the management system; estimates the position of a worker at the work site based on a first image taken by a first camera that images the work site; calculates at least one of the worker's movement speed and the worker's working time in an area of ​​the work site corresponding to the position based on the estimation result; and generates a simulation model of the work site based on the layout information and at least one of the worker's movement speed and the worker's working time in each of one or more areas of the work site.

Images