Simulation system and program

Abstract

This simulation system is provided with: an acquisition unit that acquires sensor information detected by a plurality of sensors provided in an area in which a human worker moves and executes a job; a generation unit that generates, on the basis of the sensor information, model data obtained by modeling a robot for work in place of the human worker; and a display control unit that performs control for displaying the robot on the basis of the model data.

Description

Technical Field

[0001] The present disclosure relates to a simulation system and a program. Background Art

[0002] Patent Document 1 describes the posture control of a humanoid robot used for automatically performing operations on a production line in a factory.

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2019-093506 Summary of the Invention

[0006] Means for Solving the Technical Problem

[0007] The simulation system according to the first aspect includes: an acquisition unit that acquires sensor information detected by a plurality of sensors provided in an area where a human worker moves and performs operations; a generation unit that generates model data obtained by modeling a robot for replacing the human worker based on the sensor information; and a display control unit that controls the display of the robot based on the model data.

[0008] In the second aspect based on the first aspect, the generation unit updates the model data based on the sensor information detected in real time.

[0009] In the third aspect based on the first or second aspect, the simulation system further includes a selection unit that selects a robot for replacing the human worker from a plurality of robots with different specifications based on the sensor information, and the generation unit generates model data obtained by modeling the robot selected by the selection unit.

[0010] In the fourth aspect based on the third aspect, the selection unit derives operation information related to the operation of the human worker based on the sensor information and selects a robot based on the derived operation information.

[0011] In the fifth aspect based on the fourth aspect, the operation information includes at least one of information on the physique of the human worker, the weight of the object to be operated on, the number of objects to be operated on, the moving distance of the human worker, and the moving speed of the human worker.

[0012] In the sixth aspect based on the third aspect, the specifications include at least one of the size of the robot, the weight of the robot, the maximum moving speed of the robot, the maximum weight of the object that the robot can carry, and the maximum size of the object that the robot can carry.

[0013] The program according to the seventh mode causes the computer to function as an acquisition unit, a generation unit, and a display control unit of the simulation system according to the first mode.

[0014] The simulation system according to the eighth mode includes: an acquisition unit that acquires sensor information detected by a plurality of sensors, the plurality of sensors detecting a work space in which a human worker and a first robot respectively perform operations while moving; a generation unit that generates first model data obtained by modeling the operations of the human worker and second model data obtained by modeling the operations of the first robot based on the sensor information acquired by the acquisition unit; and a control unit that performs a first simulation based on the first model data and the second model data generated by the generation unit, the first simulation simulating the states in which the human worker and the first robot respectively perform the operations in the work space.

[0015] According to the eighth mode, it is possible to improve the accuracy of the first simulation for simulating the state in which a human worker and a first robot perform operations in a mixed manner.

[0016] Based on the eighth mode, in the ninth mode, when the result of the first simulation satisfies a specified condition, the control unit performs a process of proposing to execute at least one of a second simulation and a third simulation. The second simulation simulates a state in which the operation speed of the first robot is changed for the first simulation, and the third simulation simulates a state in which at least a part of the human worker is replaced with a second robot for the first simulation.

[0017] According to the ninth mode, when the result of the first simulation satisfies a specified condition, such as a condition that the operation speed of the entire work space does not reach the target, it is possible to enable the user to recognize that there are countermeasures such as changing the operation speed of the first robot and replacing at least a part of the human worker with a second robot.

[0018] Based on the eighth mode or the ninth mode, in the tenth mode, when performing the second simulation that simulates a state in which the operation speed of the first robot is changed for the first simulation, the control unit sets the operation speed of the first robot in the second simulation based on the operation speed of each human worker included in the first model data corresponding to each human worker.

[0019] According to the tenth mode, when performing the second simulation that simulates a state in which the operation speed of the first robot that performs operations in a mixed manner with a human worker is changed, it is possible to appropriately set the operation speed of the first robot based on the operation speed of the human worker.

[0020] In the eleventh method, based on the eighth method or the ninth method, when performing a third simulation of simulating a situation in which at least a part of the human workers are replaced with second robots for the first simulation, the control unit selects the human workers to be replaced with the second robots in the third simulation based on the operation speeds of the individual human workers included in the first model data corresponding to the respective human workers.

[0021] According to the eleventh method, when performing a third simulation of simulating a situation in which at least a part of the human workers working in cooperation with the first robot are replaced with second robots, the human workers to be replaced with the second robots can be appropriately selected based on the operation speeds of the individual human workers.

[0022] In the twelfth method, based on the eighth method or the ninth method, when performing a third simulation of simulating a situation in which at least a part of the human workers are replaced with second robots for the first simulation, the control unit selects the second robots to replace at least a part of the human workers in the third simulation from among a plurality of types of robots based on the first model data corresponding to the respective human workers.

[0023] According to the twelfth method, when performing a third simulation of simulating a situation in which at least a part of the human workers working in cooperation with the first robot are replaced with second robots, the second robots to replace a part of the human workers can be appropriately selected.

[0024] In the thirteenth method, based on the eighth method or the ninth method, when performing a third simulation of simulating a situation in which at least a part of the human workers are replaced with second robots for the first simulation, the generation unit generates third model data obtained by modeling the operation of the second robots, and the control unit also uses the third model data generated by the generation unit to perform the third simulation.

[0025] According to the thirteenth method, when performing a third simulation of simulating a situation in which at least a part of the human workers working in cooperation with the first robot are replaced with second robots, the accuracy of the third simulation can be further improved by also using the third model data obtained by modeling the second robots.

[0026] In the fourteenth mode, based on the eighth mode or the ninth mode, when performing the third simulation for simulating the situation of replacing at least a part of the human worker with the second robot for the first simulation, the control unit sets the operation speed of the second robot in the third simulation based on the operation speed of each human worker included in the first model data corresponding to each human worker.

[0027] According to the fourteenth mode, it is possible to appropriately set the operation speed of the second robot in the third simulation for simulating the situation of replacing at least a part of the human worker who works in combination with the first robot with the second robot, based on the operation speed of the human worker replaced by the second robot.

[0028] The program according to the fifteenth mode is executed by a computer, and the processing includes: acquiring sensor information detected by a plurality of sensors, where the plurality of sensors detect an operation space in which a human worker and a first robot respectively perform operations while moving; generating first model data obtained by modeling the operation of the human worker and second model data obtained by modeling the operation of the first robot based on the acquired sensor information; and performing a first simulation based on the generated first model data and second model data, where the first simulation simulates the situation in which the human worker and the first robot respectively perform the operations in the operation space.

[0029] According to the fifteenth mode, it is possible to improve the accuracy of the first simulation for simulating the situation where a human worker and a first robot perform operations in combination.

[0030] It should be noted that the above - disclosed summary does not list all the necessary features of the present disclosure. In addition, sub - combinations of these feature groups can also be the disclosed content. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a top - view of a floor of a warehouse where picking operations according to the first embodiment are performed.

[0032] Figure 2 It is a block diagram showing an example of the functional structure of an information processing device according to the first embodiment.

[0033] Figure 3 It is a flowchart showing an example of a processing routine executed by the information processing device according to the first embodiment.

[0034] Figure 4 It is a block diagram showing an example of the hardware structure of a computer that functions as an information processing device according to each embodiment in a schematic manner.

[0035] Figure 5 This is a diagram showing an example of data related to the specifications of the robots involved in each embodiment.

[0036] Figure 6 This is a top view of the floor of a warehouse where the picking operation according to the second embodiment is performed.

[0037] Figure 7 This is a block diagram showing an example of the functional structure of the information processing apparatus according to the second embodiment.

[0038] Figure 8 This is a flowchart showing an example of a processing routine executed by the information processing apparatus according to the second embodiment. Detailed Embodiments

[0039] Hereinafter, the present disclosure will be described by way of disclosed embodiments. However, the following embodiments do not limit the disclosure recited in the claims. In addition, the combinations of the features described in the embodiments are not necessarily all essential for the disclosed solutions.

[0040] [First Embodiment]

[0041] When aiming to significantly improve productivity by robotizing the operations of human workers in a work space such as a warehouse, it is difficult to simply replace human workers with robots and set operation speeds such as the moving speed and running speed of the robots. Therefore, the simulation system according to this embodiment is based on sensor information detected by a plurality of sensors provided as detection objects in a warehouse where human workers perform operations accompanied by movement. In a three-dimensional virtual space such as the metaverse, it fully simulates all operations such as the layout, various functions, number of people, automated machines, belt conveyors, various speeds, unloading, storage, picking, sorting, packaging, and loading of the warehouse to be robotized. Thus, it is possible to reproduce warehouse operations through digital twin and perform transfer simulation of robotizing warehouse operations.

[0042] Figure 1 This is a top view of the floor 50 of a warehouse, which is an example of an area where the human worker 52 moves and performs operations. The operation performed by the human worker 52 is, for example, a picking operation. The picking operation according to this embodiment refers to the work of collecting (picking up) required items. The human worker 52 as a picking worker has an indispensable role for shipping the items in the warehouse, and thus is arranged in all types of warehouses. The item picked up by the human worker 52 is an example of an object that is the operation target of the human worker 52.

[0043] For example, the main task is to collect specified items based on a pre-indicated list or order and transfer the aggregated items to the inspection person in charge and the packing person in charge. The larger the scale of the warehouse, the greater the variety and quantity of items stored, so a large number of human workers 52 move within the floor 50.

[0044] As Figure 1 shown, multiple shelves 54 are provided on the floor 50, and the spaces between the respective shelves 54 and between the wall of the floor 50 and the shelves 54 respectively form movement passages 56 for the human workers 52. The human workers 52 receive information on the list and order from the management control device that manages the floor 50 via the mobile terminal 64, move in the movement passages 56 according to the received information, and pick up the target items. Objects other than the shelves 54 such as belt conveyors may also be provided on the floor 50.

[0045] A plurality of sensors 12 are provided within the floor 50. The sensors 12 are arranged at multiple positions in such a way that there are no blind spots within the floor 50. The sensors 12 detect the human workers 52. In addition, the sensors 12 detect not only the human workers 52 but also various information such as the size, shape, quantity, and variety of items and the shelves 54 for reproducing the operations within the warehouse in a three-dimensional virtual space. As the sensors 12, the highest-performance cameras, solid-state lidars (LiDAR), multi-color laser coaxial displacement gauges, or other various sensor groups can be adopted. In addition, examples of the sensors 12 include vibration meters, thermal imagers, hardness meters, radars, LiDARs, high-pixel / long-focus / ultra-wide-angle / 360-degree / high-performance cameras, image recognition, faint sounds, ultrasounds, vibrations, infrared rays, ultraviolet rays, electromagnetic waves, temperature, humidity, spot artificial intelligence (AI) weather forecasts, high-precision multi-channel global positioning systems (GPS), low-altitude satellite information, or long-tail event AI data.

[0046] It should be noted that in addition to the above information, the sensors 12 can also detect images, distances, vibrations, heat, odors, colors, sounds, ultrasounds, ultraviolet rays, or infrared rays. In addition, examples of other information detected by the sensors 12 include the center-of-gravity movement of the human workers 52, the detection of the material of the floor on which the human workers 52 move, the detection of the external air temperature, the detection of the external air humidity, the detection of the vertical, horizontal, and diagonal tilt angles of the floor, and the detection of the moisture content.

[0047] The sensor 12 performs these detections, for example, every nanosecond. Private areas such as lounges and areas such as restrooms are set as not being the objects of monitoring by the sensor 12. In addition, in order to protect the privacy of each human worker 52, the face, body shape, ID, etc. of the human worker 52 are completely shielded.

[0048] Figure 2 FIG. is a block diagram showing an example of the functional structure of the information processing device 20 included in the analog system according to the present embodiment. The information processing device 20 includes an acquisition unit 30, a selection unit 31, a generation unit 32, and a display control unit 34. Robot data 40 is stored in the storage device included in the information processing device 20.

[0049] The robot data 40 includes data related to the specifications of a variety of robots with different specifications. In the present embodiment, the robot data 40 includes data related to the specifications of humanoid robots. As an example, as Figure 5 shown, for each model of the robot, the robot data 40 includes the size of the robot, the weight of the robot, the maximum moving speed of the robot, the maximum weight of the objects that the robot can carry, and the maximum size of the objects that the robot can carry as the specifications of the robot. The model of the robot is an example of the identification information for identifying the robot. In addition, the size of the robot includes the height of the robot, the length of the arm, and the length of the leg. It should be noted that the specifications of the robot may also include the number of joints of the arm and the number of joints of the leg, etc.

[0050] The acquisition unit 30 acquires the sensor information detected in real time by the plurality of sensors 12 provided in the floor 50.

[0051] The selection unit 31 derives operation information related to the operation of the human worker 52 based on the sensor information acquired by the acquisition unit 30. In the present embodiment, the selection unit 31 derives the physique of the human worker 52, the weight of the object to be operated, the number of objects to be operated, the moving distance of the human worker 52, and the moving speed of the human worker 52 as the operation information. These operation information are derived, for example, by analyzing the image obtained by photographing the floor 50 by the digital camera included in the sensor information. In addition, these operation information are derived, for example, using the distance from the sensor 12 to the human worker 52 or the object to be operated obtained by radar or LiDAR.

[0052] In addition, the selection unit 31 selects a robot for the operation to replace the human worker 52 from a variety of robots with different specifications. Specifically, the selection unit 31 selects a robot from the variety of robots included in the robot data 40 based on the derived operation information. For example, the selection unit 31 selects a robot that can perform the operation represented by the derived operation information from the variety of robots included in the robot data 40. For each human worker 52, the selection unit 31 derives operation information based on the sensor information and selects a robot based on the operation information. A robot that can perform the operation is, for example, a robot whose maximum weight of the item that can be carried is equal to or greater than the weight of the work item to be carried by the operation. In addition, a robot that can perform the operation is, for example, a robot whose maximum moving speed when carrying the work item is equal to or greater than the moving speed of the human worker 52.

[0053] In the case where there are a variety of robots that can perform the operation, the selection unit 31 may also select a robot with a size closest to the physique of the human worker 52. In addition, in the case where there are a variety of robots that can perform the operation, the selection unit 31 may also select a robot with a size closest to the average physique of all human workers 52. In addition, for example, the selection unit 31 may take the operation information as input, input the derived operation information into a learned model that outputs the model number of the robot most suitable for the operation represented by the operation information, and thereby select a robot. In this case, the learned model may be obtained in advance through machine learning using supervised data.

[0054] The above robot selection process performed by the selection unit 31 may not be executed every time the acquisition unit 30 acquires sensor information. In this case, the robot selection process performed by the selection unit 31 is, for example, executed based on the sensor information acquired during a preset time interval such as every 10 minutes.

[0055] The generation unit 32 generates model data obtained by modeling a robot for the operation to replace human workers 52 other than the manager based on the sensor information acquired by the acquisition unit 30. The model data includes the type and various specifications of the robot. Specifically, the generation unit 32 refers to the robot data 40 and generates model data obtained by modeling the robot selected by the selection unit 31.

[0056] In addition, when generating the model data of the robot, the generation unit 32 may average the body shapes of the human workers 52 and reproduce the robot corresponding to the human worker 52 with a simple serial number.

[0057] In addition, the generation unit 32 also generates model data obtained by modeling objects other than human workers 52 such as the shelves 54 and the moving passages 56 based on the sensor information acquired by the acquisition unit 30.

[0058] Whenever the acquisition unit 30 acquires sensor information, the generation unit 32 generates and updates the model data in real time.

[0059] Based on the model data generated by the generation unit 32, the display control unit 34 performs display control on a display device such as a liquid crystal display after arranging the robot and various objects in a three-dimensional virtual space. In addition, when the model data is updated by the generation unit 32, the display control unit 34 updates the display of the robot and various objects in the three-dimensional virtual space. Thus, the operations on the floor 50 of the warehouse are reproduced in the three-dimensional virtual space.

[0060] The information processing device 20 can estimate the incoming and outgoing quantities of items during peak hours, and cause the virtual avatar representing the robot in the three-dimensional virtual space to perform operations at the same speed as the operation speed of the human worker 52 who has envisioned the estimated peak-hour situation.

[0061] In addition, the information processing device 20 can also repeatedly perform simulations such as changing the layout inside the warehouse, the specifications of the robots, and the number of robots in the three-dimensional virtual space until the operation speed capable of achieving n times the incoming and outgoing quantities of items is perfectly realized. The magnification factor n in this case can also be specified by the user.

[0062] As described above, by repeatedly simulating the reproduction of operations inside the warehouse and the execution of operations at various operation speeds in the three-dimensional virtual space, countermeasures such as changing the layout inside the warehouse and changing the number of robots can be found. Through this simulation, it is possible to output at an incoming and outgoing speed about 3 to 20 times the current situation.

[0063] In addition, the user can also input the success rate of operations such as picking performed by the prototype robot into the information processing device 20, monitor when operations fail, and conduct tests on rescue operations performed by humans.

[0064] The information processing device 20 repeatedly executes Figure 3 the flowchart shown.

[0065] In step S10, the acquisition unit 30 acquires the sensor information detected in real time by a plurality of sensors 12 provided inside the floor 50.

[0066] In step S11, as described above, the selection unit 31 derives operation information related to the operations of the human worker 52 based on the sensor information acquired in step S10, and selects a robot based on the derived operation information.

[0067] In step S12, as described above, the generation unit 32 generates model data obtained by modeling a robot for a task to replace a human worker 52 and an object other than the human worker 52 based on the sensor information acquired in step S10 and the robot selected in step S11.

[0068] In step S14, based on the model data generated in step S12, the display control unit 34 controls the display of a display device such as a liquid crystal display after arranging the robot and various objects in a three-dimensional virtual space. When the processing of step S14 ends, the processing of the flowchart ends.

[0069] Figure 4 An example of the hardware configuration of a computer 1200 that functions as the information processing device 20 is schematically shown. A program installed in the computer 1200 enables the computer 1200 to function as one or more "units" of the device according to the present embodiment, or enables the computer 1200 to perform operations associated with the device according to the present embodiment or the one or more "units", and / or enables the computer 1200 to execute the process according to the present embodiment or a stage of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described in this specification.

[0070] The computer 1200 according to the present embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216 that are interconnected via a host controller 1210. The computer 1200 further includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive can be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 can be a hard disk drive, a solid state drive, etc. The computer 1200 further includes a ROM 1230 and traditional input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0071] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 from a frame buffer or the like provided in the RAM 1214 or within itself, and causes the image data to be displayed on the display device 1218.

[0072] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 within the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to the IC card.

[0073] The ROM 1230 stores therein a boot program executed by the computer 1200 at startup and / or a program dependent on the hardware of the computer 1200. The input / output chip 1240 can also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0074] The program is provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The program is read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214 or the ROM 1230 which are also examples of computer-readable storage media, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200 and enables cooperation between the programs and the various types of hardware resources described above. The apparatus or method may be configured by operating or processing information in accordance with the use of the computer 1200.

[0075] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 can execute a communication program loaded into the RAM 1214 and command the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads the transmission data stored in the transmission buffer provided in a recording medium such as the RAM 1214, the storage device 1224, the DVD-ROM or the IC card, and transmits the read transmission data to the network, or writes the received data received from the network to the reception buffer provided on the recording medium.

[0076] In addition, the CPU 1212 can cause all or a necessary part of a file or database stored in an external recording medium such as the storage device 1224, the DVD drive (DVD-ROM), the IC card, etc. to be read into the RAM 1214 and perform various types of processing on the data on the RAM 1214. Next, the CPU 1212 can write the processed data back to the external recording medium.

[0077] Various types of information such as various types of programs, data, tables, and databases can be stored in a recording medium to undergo information processing. The CPU 1212 can perform various types of processing on the data read from the RAM 1214 and write the results back to the RAM 1214. The various types of processing include various types of operations, information processing, conditional judgments, conditional branches, unconditional branches, information retrieval / replacement, etc., which are described throughout this disclosure and specified by the instruction sequences of the programs. In addition, the CPU 1212 can retrieve information in files, databases, etc. within the recording medium. For example, in the case where a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 can retrieve an entry that matches the condition specifying the attribute value of the first attribute from the plurality of entries and read the attribute value of the second attribute stored in the entry, thereby obtaining the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0078] The programs or software modules described above can be stored in a computer-readable storage medium on or near the computer 1200. In addition, a recording medium such as a hard disk or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, and thus the program can be provided to the computer 1200 via the network.

[0079] The blocks in the flowcharts and block diagrams in this embodiment can represent stages of a process of performing operations or "parts" of a device having the function of performing operations. The specific stages and "parts" can be implemented by dedicated circuits, programmable circuits supplied together with computer-readable instructions stored on a computer-readable storage medium, and / or processors supplied together with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuits can include digital and / or analog hardware circuits, and can also include integrated circuits (ICs) and / or discrete circuits. The programmable circuits can include, for example, reconfigurable hardware circuits such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs). The reconfigurable hardware circuits include logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, flip-flops, registers, and storage elements.

[0080] A computer-readable storage medium can include any tangible device capable of storing instructions executable by a suitable device. As a result, a computer-readable storage medium having instructions stored therein comprises a product including instructions that can be executed to generate units for performing the operations specified in a flowchart or block diagram. Examples of computer-readable storage media can include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media can include floppy (registered trademark) disks, magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (registered trademark) disk, memory stick, integrated circuit card, etc.

[0081] Computer-readable instructions can include any one of assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source code or object code described in any combination of one or more programming languages, the one or more programming languages including object-oriented programming languages such as Smalltalk (registered trademark), JAVA (registered trademark), C++, etc. and traditional procedural programming languages such as the "C" programming language or similar programming languages.

[0082] Computer-readable instructions may be provided to a processor or programmable circuitry of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus locally or via a local area network (LAN), a wide area network (WAN) such as the Internet, etc., causing the processor or programmable circuitry of the general-purpose computer, special-purpose computer, or other programmable data processing apparatus to execute the computer-readable instructions to generate units for performing the operations specified in the flowchart or block diagram. Examples of the processor include a computer processor, a processing unit, a microprocessor, a digital signal processor, a controller, a microcontroller, etc.

[0083] [Second Embodiment]

[0084] Next, a second embodiment of the present disclosure will be described. It should be noted that the same reference numerals are assigned to the same parts as those in the first embodiment, and the description thereof is omitted.

[0085] In the case of robotizing the operations of human workers in a work space such as a warehouse, it is unrealistic to replace all human workers engaged in the operations with robots at once because the error in the transfer simulation cannot be ignored. Therefore, the above-mentioned robotization is achieved by the following method: performing a transfer simulation in the case of replacing a part of human workers with robots, and based on the result, repeatedly performing the action of actually replacing a part of human workers with robots multiple times. Therefore, in the process of the above-mentioned robotization, a situation occurs where human workers and robots perform operations in the work space in a mixed manner.

[0086] Therefore, in the present embodiment, in the situation where human workers and robots perform operations in the work space in a mixed manner, as described in detail later, first, a first simulation is performed, which simulates the situation where human workers and a first robot perform operations with accompanying movement in the work space. In addition, according to the result of the first simulation, in the case where the operation speed of the entire floor 50 does not reach the target, etc., a second simulation or a third simulation is performed. The second simulation simulates the situation where the operation speed of the first robot is changed for the first simulation, and the third simulation simulates the situation where at least a part of the human workers is replaced with a second robot for the first simulation.

[0087] Figure 6 It is a top view of the floor 50 of a warehouse, which is an example of a work space where human workers 52 and robots 60 perform operations with accompanying movement respectively. It should be noted that at a timing before robotization, for example, a situation where only human workers 52 exist on the floor 50, and at a timing after robotization, for example, a situation where only robots 60 exist on the floor 50, butFigure 6 shows the situation at a certain timing during the robotization, that is, the situation where the human worker 52 and the robot 60 coexist on the 50th floor. In Figure 6 , the robot 60 arranged on the 50th floor is an example of the first robot in the present disclosure. It should be noted that in the present disclosure, the robot 60 only needs to be able to perform the same operation accompanied by movement as the human worker 52, and it can be a humanoid robot or a non-humanoid robot.

[0088] The operations performed by the human worker 52 and the robot 60 are, for example, the same picking operations as those in the first embodiment.

[0089] In the second embodiment, in addition to the human worker 52, the robot 60 also moves within the 50th floor. In addition, the sensor 12 detects not only the human worker 52 but also the robot 60. It should be noted that the sensor 12 is not limited to being provided within the 50th floor, and can also be loaded or installed on at least one of the human worker 52 and the robot 60 existing within the 50th floor.

[0090] Figure 7 is a block diagram showing an example of the functional structure of the information processing device 20 included in the simulation system according to the present embodiment. The information processing device 20 includes an acquisition unit 130, a generation unit 132, a control unit 134, and a storage unit 136. In the present embodiment, it is possible to select a robot 60 (an example of the second robot 60 in the present disclosure) that replaces the human worker 52 performing operations within the 50th floor from among a variety of robots 60 with different specifications, and robot data 40 is stored in the storage unit 136.

[0091] The robot data 40 is the same data as that in the first embodiment and includes data related to the specifications of a variety of robots 60. As an example, as Figure 5 shown, for each model of the robot 60, the robot data 40 includes the size of the robot 60, the weight of the robot 60, the maximum moving speed of the robot 60, the maximum weight of the items that the robot 60 can carry, and the maximum size of the items that the robot 60 can carry as the specifications of the robot 60. The model of the robot 60 is an example of the identification information for identifying the robot 60. In addition, the size of the robot 60 includes the height of the robot 60, the length of the arm, and the length of the leg. It should be noted that the specifications of the robot 60 may also include the number of joints of the arm and the number of joints of the leg, etc.

[0092] It should be noted that the present disclosure also includes a manner in which the robot 60 (the second robot 60) that replaces the human worker 52 operating in the floor 50 is limited to a robot 60 of a fixed specification and a single type. In this manner, the robot data 40 (the storage unit 136 that stores the robot data 40) can also be omitted, and the process of selecting the robot 60 that functions as the second robot 60 can be omitted.

[0093] The acquisition unit 130 acquires sensor information detected in real time by a plurality of sensors 12 provided in the floor 50.

[0094] Based on the sensor information acquired by the acquisition unit 130, the generation unit 132 generates first model data obtained by modeling the operations of each human worker 52 other than the manager existing in the floor 50, corresponding to each human worker 52. It should be noted that the first model data corresponding to each human worker 52 at least includes information on the operation speed of each human worker 52.

[0095] In addition, in the present embodiment, the first model data may also include, for example, operation information related to the operation of the human worker 52, such as the physique of the human worker 52, the weight of the object to be operated, the number of objects to be operated, the moving distance of the human worker 52, and the moving speed of the human worker 52. The generation unit 132 can derive the above operation information from the sensor information acquired by the acquisition unit 130. The above operation information can be derived, for example, by analyzing an image of the floor 50 taken by a digital camera included in the sensor information. In addition, the above operation information can be derived, for example, using the distance to the human worker 52 or the object to be operated obtained from sensors 12 such as radar or LiDAR.

[0096] In addition, when there is a robot 60 (the first robot 60) in the floor 50, based on the sensor information acquired by the acquisition unit 130, the generation unit 132 also generates second model data obtained by modeling the operations of the first robot 60 existing in the floor 50. The second model data may also include the type and various specifications of the first robot 60. In addition, when generating the second model data, the physique of the human worker 52 can be averaged, and the robot 60 can be reproduced by a simple serial number.

[0097] Further, in the case of performing the third simulation, the generation unit 132 also generates third model data obtained by modeling the second robot 60 (the robot 60 selected by the control unit 134 described later) that replaces the human worker 52 working in the floor 50, based on the robot data 40 stored in the storage unit 136. It should be noted that when generating the third model data, the generation unit 132 may also average the body shapes of the human workers 52 and reproduce the robot corresponding to the human worker 52 by a simple serial number.

[0098] In addition, the generation unit 132 also generates model data obtained by modeling objects other than the human worker 52 and the robot 60, such as the shelf 54 and the moving passage 56, based on the sensor information acquired by the acquisition unit 130. Whenever the acquisition unit 130 acquires sensor information, the generation unit 132 generates and updates the above-mentioned respective model data in real time.

[0099] In the case of performing any one of the first simulation to the third simulation, the control unit 134 generates a three-dimensional virtual space obtained by simulating the shelf 54, the moving passage 56, etc. in the floor 50, based on the respective model data generated by the generation unit 132. In addition, in the case of performing the first simulation or the second simulation, the control unit 134 arranges an object corresponding to the human worker 52 in the three-dimensional virtual space based on the first model data, and arranges an object corresponding to the first robot 60 in the three-dimensional virtual space based on the second model data. It should be noted that in the case of performing the second simulation, before arranging the object corresponding to the first robot 60 in the three-dimensional virtual space, the control unit 134 changes and sets the operation speed of the first robot 60 (updates the second model data) based on the operation speed of the human worker 52 included in the first model data.

[0100] Then, the control unit 134 performs a simulation of an operation accompanied by movement in the three-dimensional virtual space for each object corresponding to the human worker 52 or the first robot 60 arranged in the three-dimensional virtual space, based on the respective model data including the first model data and the second model data. Thereby, it is possible to implement the first simulation that accurately simulates the situation where the human worker 52 and the first robot 60 work in a mixed manner, or the second simulation that accurately simulates the situation where the operation speed of the first robot 60 is changed for the first simulation.

[0101] In the present embodiment, in the case of performing a third simulation in which at least a part of the human worker 52 who works within the floor 50 is replaced with the second robot 60, the control unit 134 selects the human worker 52 (the human worker 52 to be replaced) to be replaced with the second robot 60 based on the first model data. The human worker 52 to be replaced can be selected, for example, based on the operation speed of each human worker 52 included in the first model data corresponding to each human worker 52. As an example, a predetermined number of human workers 52 to be replaced can be selected in ascending order of operation speed. Thereby, the human worker 52 to be replaced can be appropriately selected based on the operation speed of the human worker 52 and the like.

[0102] In addition, in the case of performing the third simulation, the control unit 134 selects the second robot 60 that replaces the human worker 52 to be replaced from a variety of robots 60 with different specifications. Specifically, the control unit 134 selects the second robot 60 from a variety of robots 60 in which specifications and the like are set in the robot data 40 based on the operation information included in the first model data corresponding to the human worker 52 to be replaced. For example, the control unit 134 selects the second robot 60 that can execute the operation represented by the operation information from the variety of robots 60. An example of the second robot 60 that can execute the operation is a robot 60 whose maximum weight of the loadable item is equal to or greater than the weight of the item to be operated by the operation. In addition, another example of the second robot 60 that can execute the operation is a robot 60 whose maximum moving speed when transporting the item to be operated is equal to or greater than the moving speed of the human worker 52.

[0103] In the case where there are a variety of second robots 60 that can execute the operation, the control unit 134 may also select the robot 60 with the size closest to the physique of the human worker 52 to be replaced. In addition, in the case where there are a variety of second robots 60 that can execute the operation, the control unit 134 may also select the robot 60 with the size closest to the average physique of the human workers 52. In addition, for example, the control unit 134 may also input the operation information and input the operation information to the learned model that outputs the model of the robot 60 most suitable for the operation represented by the operation information, thereby selecting the second robot 60. In this case, the learned model can be obtained in advance by machine learning using supervised data.

[0104] The control unit 134 may not perform the selection of the second robot 60 every time the acquisition unit 130 acquires the sensor information. In this case, the selection of the second robot 60 by the control unit 134 may be performed, for example, at a preset time interval such as 10 minutes based on the sensor information acquired during the time interval.

[0105] Further, in the case of performing the third simulation, the control unit 134 arranges an object corresponding to a part of the human workers 52 who are working within the floor 50 and have not been replaced by the second robot in the current simulation (human workers 52 as non-replacement objects) in the three-dimensional virtual space based on the first model data corresponding thereto. Further, the control unit 134 arranges an object corresponding to the first robot 60 in the three-dimensional virtual space based on the second model data. Further still, the control unit 134 arranges an object corresponding to the second robot 60 that replaces the human worker 52 as the replacement object in the current simulation in the three-dimensional virtual space based on the third model data.

[0106] It should be noted that at this time, the control unit 134 may also set the operation speed of the second robot 60 based on the operation speed of the human worker 52 as the replacement object included in the first model data corresponding to the human worker 52 as the replacement object. In this case, the operation speed of the second robot 60 in the simulation can be appropriately set according to the operation speed of the human worker 52 as the replacement object.

[0107] Then, for each object corresponding to any one of the human worker 52, the first robot 60, and the second robot 60 arranged in the three-dimensional virtual space, the control unit 134 performs a simulation of an operation with accompanying movement in the three-dimensional virtual space based on each model data including the first model data to the third model data. Thereby, the third simulation for accurately predicting the situation where the human worker 52 as the replacement object among the human workers 52 working in combination with the first robot 60 is replaced by the second robot 60 is realized.

[0108] It should be noted that the control unit 134 may also appropriately display the progress, results, etc. of the simulation on a display device such as a liquid crystal display. Further, in the case where each model data is updated by the generation unit 132, the control unit 134 may also update the arrangement, etc. of each object corresponding to one of the human worker 52, the first robot 60, and the second robot 60 in the three-dimensional virtual space. Thereby, the operation on the floor 50 of the warehouse is reproduced in the three-dimensional virtual space. Further, the control unit 134 may also estimate the incoming and outgoing quantities of items at the peak time of the warehouse and estimate the situation at the estimated peak time through simulation.

[0109] In addition, the control unit 134 (information processing device 20) can also repeatedly perform simulations such as changing the layout inside the warehouse, the specifications of the robot 60, the number of robots 60, etc. in the three-dimensional virtual space until the operation speed that can achieve the incoming and outgoing quantities of items n times is perfectly realized. The magnification n in this case can be specified by the user or the operation speed can be set according to the ability of human workers.

[0110] In this way, by repeatedly simulating the reproduction of operations inside the warehouse and the execution of operations at various operation speeds in the three-dimensional virtual space, countermeasures such as changing the layout inside the warehouse and the number of robots 60 can be found. Through this simulation, the incoming and outgoing speeds can be output at about 3 to 20 times the current level.

[0111] In addition, the user can also input the success rate of operations such as picking performed by the prototype robot 60 into the information processing device 20 to monitor operations during operation failures and conduct tests on rescue operations performed by humans.

[0112] The information processing device 20 repeatedly executes Figure 8 the processing shown in the flowchart in

[0113] In step S110, the acquisition unit 130 acquires the sensor information detected in real time by a plurality of sensors 12 with the floor 50 as the detection object.

[0114] In step S112, as described above, the generation unit 132 generates first model data obtained by modeling the operations of human workers 52 existing in the floor 50 and second model data obtained by modeling the operations of the first robot 60 existing in the floor 50 according to the sensor information acquired in step S110. It should be noted that the processing of steps S110 and S112 is also repeatedly executed in parallel, for example, at a nanosecond cycle, during the processing of the subsequent steps S114 and later.

[0115] In step S114, as described above, the control unit 134 performs a first simulation of the situation where human workers 52 and the first robot 60 coexist, and outputs the execution result of the first simulation to a display device or the like.

[0116] If the execution result of the first simulation is output to a display device or the like, the user refers to the execution result of the first simulation to perform an operation of verifying the simulation result, such as an operation of verifying whether the operation speed of the entire floor 50 in the first simulation is appropriate (step S116).

[0117] In step S118, the control unit 134 determines whether the overall operation speed of floor 50 has reached the target based on the result of the verification operation input by the user via the input device, etc. If the determination in step S118 is affirmative, Figure 8 the execution of the process shown in the flowchart ends.

[0118] In addition, if the determination in step S118 is negative, the process transfers to step S120, and the control unit 134 outputs information recommending the execution of the second simulation and the third simulation as options for countermeasures to improve the operation speed to a display device or the like, thereby performing the process of proposing to the user. In the next step S122, the control unit 134 determines the countermeasure to improve the operation speed selected by the user and branches according to the determination result.

[0119] If the user selects to execute the second simulation as the countermeasure to improve the operation speed, the process transfers from step S122 to step S124. In step S124, as described above, the control unit 134 changes the second model data based on the operation speed of the human worker 52 or the like to change the operation speed of the first robot 60. Then, in the next step S126, the control unit 134 executes the second simulation in which the operation speed of the first robot 60 is changed for the first simulation and outputs the result of the second simulation to a display device or the like. If the process of step S126 ends, the process returns to step S116.

[0120] In addition, if the user selects to execute the third simulation as the countermeasure to improve the operation speed, the process transfers from step S122 to step S130. In step S130, as described above, the control unit 134 selects the human worker 52 to be replaced by the second robot 60 in this simulation based on the first model data corresponding to each human worker 52.

[0121] In the next step S132, the control unit 134 outputs the human worker 52 selected as the replacement object in step S130 to a display device or the like and performs a confirmation process for requesting the user to approve the number of the human workers 52 as the replacement object or the like.

[0122] In step S134, as described above, the control unit 134 selects the second robot 60 that replaces the human worker 52 as the replacement object based on the first model data corresponding to the human worker 52 as the replacement object.

[0123] In step S136, as described above, the generation unit 132 generates the third model data obtained by modeling the operation of the second robot 60 that replaces the human worker 52 as the replacement object in this simulation.

[0124] In step S138, in addition to the first model data and the second model data generated in step S112, the control unit 134 executes a third simulation after arranging an object corresponding to a certain one of the human worker 52, the first robot 60, and the second robot that are not replacement objects in the three-dimensional virtual space, based on the third model data generated in step S136. When the process of step S138 ends, the process returns to step S116.

[0125] As described above, the technology of the present disclosure has been described using the embodiments, but the technical scope of the present disclosure is not limited to the scope described in the above embodiments. Those skilled in the art should understand that various changes or improvements can be made to the above embodiments. As can be seen from the claims, embodiments with such changes or improvements are also included in the technical scope of the present disclosure.

[0126] It should be noted that the execution order of each process such as the actions, sequences, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings is not particularly specified as "before...", "earlier than...", etc., or as long as the output of the previous process is not used in the subsequent process, it can be implemented in any order. Regarding the action flow in the claims, the specification, and the drawings, even if it is described using "first", "next", etc. for convenience, it does not mean that it must be implemented in that order.

[0127] The entire disclosures of Japanese Patent Application No. 2022-169079 filed on October 21, 2022, Japanese Patent Application No. 2022-179649 filed on November 9, 2022, and Japanese Patent Application No. 2022-203354 filed on December 20, 2022 are incorporated herein by reference in their entirety.

[0128] Explanation of Reference Numerals

[0129] 12: Sensor, 20: Information Processing Device, 30: Acquisition Unit, 31: Selection Unit, 32: Generation Unit, 34: Display Control Unit, 40: Robot Data, 50: Floor, 52: Human Worker, 54: Shelf, 56: Movement Passage, 60: Robot, 64: Portable Terminal, 130: Acquisition Unit, 132: Generation Unit, 134: Control Unit, 136: Storage Unit, 1200: Computer, 1210: Host Controller, 1212: CPU, 1214: RAM, 1216: Graphics Controller, 1218: Display Device, 1220: Input / Output Controller, 1222: Communication Interface, 1224: Storage Device, 1230: ROM, 1240: Input / Output Chip

Claims

1. A simulation system, wherein, the simulation system includes: an acquisition unit that acquires sensor information detected by a plurality of sensors provided in an area where a human worker moves and performs operations; a generation unit that generates model data obtained by modeling a robot for replacing the operations of the human worker based on the sensor information; and a display control unit that controls the display of the robot based on the model data.

2. The simulation system according to claim 1, wherein, the generation unit updates the model data based on the sensor information detected in real time.

3. The simulation system according to claim 1 or 2, wherein, the simulation system further includes a selection unit that selects, based on the sensor information, a robot for replacing the operations of the human worker from among a plurality of robots having different specifications, and the generation unit generates model data obtained by modeling the robot selected by the selection unit.

4. The simulation system according to claim 3, wherein, the selection unit derives operation information related to the operations of the human worker based on the sensor information, and selects a robot based on the derived operation information.

5. The simulation system according to claim 4, wherein, the operation information includes at least one of information on the physique of the human worker, the weight of the object to be operated on, the number of objects to be operated on, the moving distance of the human worker, and the moving speed of the human worker.

6. The simulation system according to claim 3, wherein, the specifications include at least one of the size of the robot, the weight of the robot, the maximum moving speed of the robot, the maximum weight of the object that the robot can carry, and the maximum size of the object that the robot can carry.

7. A program, wherein, the program causes a computer to function as the acquisition unit, the generation unit, and the display control unit of the simulation system according to claim 1.

8. A simulation system, wherein, the simulation system includes: an acquisition unit that acquires sensor information detected by a plurality of sensors that detect an operation space in which a human worker and a first robot respectively move and perform operations; a generation unit that generates first model data obtained by modeling the operations of the human worker and second model data obtained by modeling the operations of the first robot based on the sensor information acquired by the acquisition unit; and a control unit that performs a first simulation based on the first model data and the second model data generated by the generation unit, the first simulation simulating the states in which the human worker and the first robot respectively perform the operations in the operation space.

9. The simulation system according to claim 8, wherein, In a case where the result of the first simulation satisfies a specified condition, the control unit performs processing of proposing to execute at least one of a second simulation and a third simulation. The second simulation simulates a situation where the operation speed of the first robot is changed with respect to the first simulation, and the third simulation simulates a situation where at least a part of the human workers is replaced with a second robot with respect to the first simulation.

10. The simulation system according to claim 8 or 9, wherein, in a case of performing the second simulation that simulates a situation where the operation speed of the first robot is changed with respect to the first simulation, the control unit sets the operation speed of the first robot in the second simulation based on the operation speed of each of the human workers included in the first model data corresponding to each of the human workers.

11. The simulation system according to claim 8 or 9, wherein, in a case of performing the third simulation that simulates a situation where at least a part of the human workers is replaced with a second robot with respect to the first simulation, the control unit selects the human workers to be replaced with the second robot in the third simulation based on the operation speed of each of the human workers included in the first model data corresponding to each of the human workers.

12. The simulation system according to claim 8 or 9, wherein, in a case of performing the third simulation that simulates a situation where at least a part of the human workers is replaced with a second robot with respect to the first simulation, the control unit selects the second robot to be replaced from at least a part of the human workers in the third simulation based on the first model data corresponding to each of the human workers from among a plurality of types of robots.

13. The simulation system according to claim 8 or 9, wherein, in a case of performing the third simulation that simulates a situation where at least a part of the human workers is replaced with a second robot with respect to the first simulation, the generation unit generates third model data obtained by modeling the operation of the second robot, and the control unit also uses the third model data generated by the generation unit to perform the third simulation.

14. The simulation system according to claim 8 or 9, wherein, in a case of performing the third simulation that simulates a situation where at least a part of the human workers is replaced with a second robot with respect to the first simulation, the control unit sets the operation speed of the second robot in the third simulation based on the operation speed of each of the human workers included in the first model data corresponding to each of the human workers.

15. A program, wherein, the program is for causing a computer to execute processing including: acquiring sensor information detected by a plurality of sensors, the plurality of sensors detecting an operation space in which a human worker and a first robot respectively perform operations while moving therewith; Based on the acquired sensor information, generate first model data obtained by modeling the operation of the human worker and second model data obtained by modeling the operation of the first robot. Based on the generated first model data and second model data, perform a first simulation, which simulates the conditions of the human worker and the first robot performing the operation in the operation space respectively.

Images