Method and data processing system for multi-state simulation to verify the safety of industrial scenarios

Abstract

A method and a data processing system for simulating and processing collision avoidance management of a production plant area, performed by a data processing system, are disclosed. During an industrial scenario, such as a production process, an assembly process, material handling, or item sorting, at least possible spatial trajectories of objects, such as humans, production components, stationary and mobile robots, automated guided vehicles (AGVs), and the like, are determined in a predefined area, and a static 4D structure of the area is determined, wherein the possible positions of objects in the area are determined based on their positions and the time at which they will be at those positions, in order to identify potential collision events and to remedy those potential collision events.

Description

Technical Field

[0001] The present disclosure generally relates to computer-aided design, visualization, and manufacturing ("CAD") systems, product lifecycle management ("PLM") systems, production data management systems, and similar systems that manage data for use in industrial scenarios, such as production processes and other processes (collectively referred to as manufacturing execution systems ("MES") or manufacturing operations management systems ("MOM")). Background Art

[0002] In the planning of industrial scenarios, computer simulation techniques are used in which the physical scene of a physical environment is modeled by the virtual scene of a virtual simulation environment. The physical or real scene can be, for example, a facility, a manufacturing plant, an industrial scene, or any other physical scene that can benefit from being modeled in a virtual environment for the purpose of industrial simulation.

[0003] The real scene can include a variety of real objects related to the facility. Examples of real objects include, but are not limited to, personnel, workers, equipment, tools, containers, materials, finished or semi-finished products, and other objects present in the real scene. Real objects are represented by virtual objects in the virtual simulation environment. Virtual objects are typically defined by three-dimensional (3D) virtual models. Examples of 3D virtual models include, but are not limited to, CAD models, CAD-like models, point cloud models, and other types of 3D computer models.

[0004] In a real scene, real objects have positions and orientations, and when the real objects are moved or repositioned within the real scene during the industrial scene, the positions and orientations of the real objects may change / move. When modeling industrial facilities in a virtual simulation environment, a common requirement is that the positions and orientations of virtual objects in the virtual scene accurately reflect the positions and orientations of real objects in the real scene during the industrial scene.

[0005] It is assumed that conventional stationary robots with manipulators, industrial robots, lightweight robots such as mobile robots, and AGVs or any autonomous industrial equipment such as forklifts have a certain risk of colliding with humans. They are designed to work around humans without safety guarding, so safety mechanisms are provided to reduce any possible injuries.

[0006] Therefore, due to the complexity of the physical environment and the dynamic nature of processes in both spatial and temporal domains, industrial robots require intensive planning, simulation, and validation. Today, industrial robots are often confined to fenced-off locations due to safety constraints, which prevents any interaction with humans during the process. This also reduces the need for sensory information. Today, validation and simulation tasks are already extremely complex and require high-performance computers with CPU, memory, and GPU resources.

[0007] This planning, simulation, and verification challenge is extremely important in loosely constrained stochastic environments, including typical industrial robots, autonomous robots, and humans without fences.

[0008] Robots can harm humans and damage other robots and equipment. Furthermore, robots can be prevented from completing their tasks. Therefore, new robots require extensive sensory information about their environment and respond dynamically. This introduces an additional element of non-deterministic behavior when humans are introduced into industrial scenarios. Furthermore, autonomous robots introduce complex temporal and spatial behaviors.

[0009] Because of safety and loose targeting, the software that operates the robot—as well as other asynchronous and probabilistic behavioral changes in the environment—needs to be validated before entering the workplace.

[0010] Furthermore, autonomous robots may behave completely differently in identical situations, depending on sensory information from the environment that is not perfectly repeatable and has tolerances and random properties (e.g., cloudy natural lighting will be processed differently by visual sensors than sunny lighting).

[0011] Human subjects introduce additional challenges, as they are unpredictable (they don't have "algorithms") and therefore more prone to illogical responses and varying degrees of sensitivity (no "known sensor parameters"). Summary of the Invention

[0012] It is difficult to design environments and planning processes to minimize the risk of collisions between robots and / or AGVs and humans or other objects. Furthermore, the behavior of objects, particularly humans, is nondeterministic and often not precisely predefined, making it difficult to accurately time their movements in order to correctly calculate all possible collision risks. Therefore, improved techniques are needed.

[0013] Various disclosed embodiments include simulation and processing methods, as well as corresponding systems and computer-readable media. A multi-state simulation method for verifying the safety of an industrial scene within a predetermined area and within a predetermined time period, performed by a data processing system, is disclosed. A static 4D or 5D structure of a predefined area is determined, wherein possible locations of objects are determined based on their positions and the times at which they will be at those locations. Furthermore, possible spatial trajectories of objects, such as humans, production parts, stationary and mobile robots, automated guided vehicles (AGVs), etc., are determined within the predefined area during an industrial scene, such as a production process, assembly process, material handling, or item sorting. The positions and times of the objects are updated within the static 4D or 5D structure based on the trajectories. Possible matches of the objects occurring at different locations at different times are then identified, and the probability of the objects occurring simultaneously in the same volume is evaluated against a predefined threshold. Matches with probabilities below the predefined threshold are optionally pruned. Detailed kinematic calculations are performed on matches with probabilities above the predefined threshold to confirm whether they meet a predefined safety threshold. Matches that do not meet the predefined safety threshold are reported accordingly.

[0014] In another example, a data processing system is provided. The data processing system includes a processor and an accessible memory, and is specifically configured to execute a multi-state simulation method for verifying the safety of an industrial scenario within a predetermined area and a predefined time period, the method comprising the following steps:

[0015] a) determining a static 4D or 5D structure of the predefined area, wherein a possible location of an object is determined based on the location of the object and the time at which the object will be at the location;

[0016] b) determining possible spatial trajectories of objects, such as humans, production parts, stationary and mobile robots, AGVs, etc., in the predefined area during the industrial scenario, such as a production process, an assembly process, material handling, item sorting, etc.;

[0017] c) updating the position and the time of the object in the static 4D or 5D structure according to the trajectory;

[0018] d) identifying possible matches of the objects that appear at different locations at different times, and evaluating the probability of the objects appearing in the same volume at the same time against a predefined threshold; optionally, pruning matches with a probability below the predefined threshold;

[0019] e) performing detailed kinematic calculations on those matches with a probability above the predefined threshold to confirm whether these matches meet a predefined safety threshold; and

[0020] f) Report those matches that do not meet the predefined security threshold.

[0021] In another example, a non-transitory computer-readable medium is provided. The non-transitory computer-readable medium is encoded with executable instructions that, when executed, cause one or more data processing systems to perform a multi-state simulation method for verifying the safety of an industrial scenario within a predetermined area and a predefined time period, the method comprising the following steps:

[0022] a) determining a static 4D or 5D structure of the predefined area, wherein a possible location of an object is determined based on the location of the object and the time at which the object will be at the location;

[0023] b) determining possible spatial trajectories of objects, such as humans, production parts, stationary and mobile robots, AGVs, etc., in the predefined area during the industrial scenario, such as a production process, an assembly process, material handling, item sorting, etc.;

[0024] c) updating the position and the time of the object in the static 4D or 5D structure according to the trajectory;

[0025] d) identifying possible matches of the objects that appear at different locations at different times, and evaluating the probability of the objects appearing in the same volume at the same time against a predefined threshold; optionally, pruning matches with a probability below the predefined threshold;

[0026] e) performing detailed kinematic calculations on those matches with a probability above the predefined threshold to confirm whether these matches meet a predefined safety threshold; and

[0027] f) Report those matches that do not meet the predefined security threshold.

[0028] Regarding the calculation of the probability that an object will be at different spatial locations at different points in time, it is possible to define this probability by several factors:

[0029] Assuming that industrial scenarios, such as industrial manufacturing processes and logistics processes, are repeatable and, in most cases, have a defined cycle time, it is possible to identify cycles based on the overall workflow plan (the cycle time does not have to be exact). This defines the time range of the objects involved in the industrial scenario (in extreme cases, the time range may change completely). Discretization can be achieved for predeterminable time units, such as the time unit for the fastest breakthrough of an AGV from maximum speed to zero. Regarding the spatial range of the objects, all channels and any free areas can be calculated for each autonomous device when all points of interest are reached (for example, through a direct shortest path method).

[0030] For each device / object, the spatiotemporal probability of being in space within a certain time unit can be a normal distribution along the path that assigns the maximum probability to the object's circulation at the center of the channel (or on a predetermined path) at its nominal speed, while the orthogonal (lateral) position relative to the object's trajectory also follows a normal distribution. The probability of a moving object can be considered as a Gaussian distribution of movement. A self-learning strategy is also applied over time, and this Gaussian distribution can be refined to more precise values ​​as it learns from the quality of the match between the object's predetermined spatial and temporal position and the object's actual spatial and temporal position during the execution of the object's workflow.

[0031] As a good example, two adjacent objects can have, for example, overlapping Gaussian distributions at different locations and times in a 4D or 5D structure in a predefined region. For each point in the overlapping spatial region, the product of the two probabilities can be calculated, allowing pruning of those locations where the product is below a predefined threshold.

[0032] The foregoing has been a fairly broad overview of the features and technical advantages of the present disclosure so that those skilled in the art may better understand the detailed description below. Other features and advantages of the present disclosure that form the subject matter of the claims will be described hereinafter. Those skilled in the art will appreciate that they can easily use the disclosed concepts and specific embodiments as a basis for modifying or designing other structures for the same purpose of implementing the present disclosure. Those skilled in the art will also appreciate that such equivalent constructions do not depart from the spirit and scope of the present disclosure in its broadest form.

[0033] Before proceeding to the following detailed description, it may be helpful to set forth definitions of certain words or phrases used throughout this patent document: the terms "include" and "comprise" and their derivatives mean including, without limitation; the term "or" is inclusive, meaning and / or; the phrases "associated with" and "associated therewith" and their derivatives may mean including, being included within, interconnected with, containing, being contained within, being connected to or connected with, being coupled to or coupled with, being communicable with, cooperating with, interleaved, juxtaposed, being proximate to, being bound to or bound with, having, having the property of, and the like; and the term "controller" means any device, system, or portion thereof that controls at least one operation, whether such device is implemented in hardware, firmware, software, or some combination of at least two of the foregoing. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether local or remote. Definitions for certain words and phrases are provided throughout this patent document, and those skilled in the art will understand that such definitions apply to many, if not most, prior instances, as well as future uses of such defined words and phrases. Although certain terms may include a wide variety of embodiments, the appended claims may expressly limit these terms to specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, wherein like reference numerals designate like objects, and wherein:

[0035] Figure 1 illustrates a block diagram of a data processing system in which embodiments may be implemented;

[0036] Figure 2 A schematic diagram of a multi-state simulation method for verifying the safety of an industrial scenario within a predetermined area and a predetermined time period is illustrated;

[0037] Figure 3 A schematic diagram illustrating an example of multiple objects moving during execution of an industrial scenario;

[0038] Figure 4 A schematic diagram illustrating another example of multiple objects moving during execution of an industrial scenario; and

[0039] Figure 5 The present invention illustrates a flow chart of a multi-state simulation method for verifying the safety of an industrial scene within a predetermined area and a predetermined time period. DETAILED DESCRIPTION

[0040] Discussed below Figures 1 to 5 The various embodiments used to describe the principles of the present disclosure in this patent document are merely illustrative and should not be construed in any way to limit the scope of the present disclosure. Those skilled in the art will appreciate that the principles of the present disclosure can be implemented in any appropriately arranged device. Many of the innovative teachings of the present application will be described with reference to exemplary, non-limiting embodiments.

[0041] Previous techniques for properly managing collision avoidance in areas of a production plant relied on on-site analysis, where a robot was brought to a workplace and a production engineer visually assessed the robot's operation to determine whether it was likely to collide with a human during its operation. This analysis was tedious, lengthy, error-prone, and not well-suited to the task of preventing injuries to humans operating alongside the robot within the same production area.

[0042] Implementations according to the present disclosure provide numerous benefits, including, but not limited to: providing a user-friendly way to systematically simulate collision management by examining object paths and object interactions in predefined areas. Furthermore, for example, production processes involving robotic and human operations within the same area of ​​a production plant can be systematically planned to at least partially avoid collisions while or to modify the production process within a simulated environment; enabling user-friendly control of the position and orientation of object operations in an intuitive manner so that industrial simulation and planning are accurate simulations of real-world processes; and facilitating the use of industrial simulation packages (such as Process Simulate and Human Simulation provided by Siemens Product Lifecycle Management Software, Inc. (Plano, Texas)) in the workplace to perform virtual simulations for ongoing production simulations for non-professional users.

[0043] Embodiments may be particularly advantageous for software packages that include CAD environments, including but not limited to NX, Process Simulation, Solid Edge, and other packages offered by Siemens Product Lifecycle Management Software, Inc. (Plano, Texas), or by other software vendors. Embodiments integrated with CAD systems can conveniently provide a complete design and simulation environment.

[0044] Figure 1Illustrated is a block diagram of a data processing system 100, in which embodiment can be implemented as, for example, a MOM system that is specially configured to perform a process as described herein by software or otherwise, and is particularly implemented as each system in a plurality of interconnections and communication systems as described herein. The illustrated data processing system 100 can include a processor 102 that is connected to a secondary buffer / bridge 104, which is then connected to a local system bus 106. The local system bus 106 can be, for example, a peripheral component interconnect (PCI) architecture bus. Also connected to the local system bus in the illustrated example is a main memory 108 and a graphics adapter 110. The graphics adapter 110 can be connected to a display 111.

[0045] Other peripheral devices such as a local area network (LAN) / wide area network / wireless network (e.g., WiFi) adapter 112 may also be connected to the local system bus 106. An expansion bus interface 114 connects the local system bus 106 to an input / output (I / O) bus 116. The I / O bus 116 connects to a keyboard / mouse adapter 118, a disk controller 120, and an I / O adapter 122. The disk controller 120 may be connected to a storage device 126, which may be any suitable machine-usable or machine-readable storage medium, including but not limited to non-volatile hard-coded type media such as read-only memory (ROM) or electrically erasable programmable read-only memory (EEPROM), magnetic tape storage devices, and user-recordable type media such as floppy disks, hard drives, and compact disk read-only memories (CD-ROMs) or digital versatile disks (DVDs), as well as other known optical, electrical, or magnetic storage devices.

[0046] Also connected to I / O bus 116 in the example shown is an audio adapter 124 to which speakers (not shown) may be connected for playing sound. A keyboard / mouse adapter 118 provides a connection for a pointing device (not shown) such as a mouse, trackball, track pointer, touch screen, or the like.

[0047] It will be understood by those skilled in the art that Figure 1 The hardware illustrated in the figures may vary for a particular implementation. For example, other peripheral devices, such as optical disk drives, may be used in addition to or in place of the illustrated hardware. The illustrated examples are provided for illustrative purposes only and are not meant to imply architectural limitations on the present disclosure.

[0048] A data processing system according to an embodiment of the present disclosure may include an operating system that utilizes a graphical user interface. The operating system allows for the simultaneous presentation of multiple display windows within the graphical user interface, wherein each display window provides an interface to a different application or to a different instance of the same application. A cursor within the graphical user interface may be manipulated by a user using a pointing device. The cursor position may be changed and / or an event, such as a mouse button click, may be generated to initiate a desired response.

[0049] If appropriately modified, one of various commercial operating systems may be used, such as Microsoft Windows, a product of Microsoft Corporation located in Redmond, Washington. TM A version of or an open source operating system such as Linux OS. The operating system is modified or created according to the disclosed content as described.

[0050] LAN / WAN / wireless network adapter 112 can connect to network 130 (not part of data processing system 100), which can be any public or private data processing system network or combination of networks known to those skilled in the art, including the Internet. Data processing system 100 can communicate via network 130 with a server system 140, which is also not part of data processing system 100 but can be implemented, for example, as a separate data processing system 100.

[0051] One or more of the processor 102, memory 108, and the simulation program running on the processor 102 receives input via one or more of the local system bus 106, the adapter 112, the network 130, the server 140, the interface 114, the I / O bus 116, the disk controller 120, the storage device 126, etc. As used herein, receiving may include retrieving from the storage device 126, receiving from another device or process, receiving via interaction with a user, or otherwise receiving.

[0052] Figure 2 The figure illustrates a multi-state simulation method for verifying the safety of industrial scenarios within a predefined area and time period. This is because it is virtually impossible to account for all possible variations in industrial scenarios within a reasonable timeframe, given the complex configuration of objects and their trajectories. Furthermore, any scenario variation introduces additional combinations. Simulating a single, precise scenario when all behaviors are completely predetermined is also useless, as variations in the behavior of autonomous devices can lead to different sequences that still achieve the desired goal.

[0053] The idea here is to test how the entire environment behaves in different situations as quickly as possible using reasonable computing resources that must be scaled (e.g., via cloud HPC). Since each detailed and kinematic simulation takes up a lot of time and computer power, it is possible to reduce the detailed simulation to only the areas where there is suspicion of possible physical interactions (e.g., collisions). As described below, the overall goal is to limit the detailed simulation to those situations in which different objects or "devices" come dangerously close to each other during the execution of the industrial scenario.

[0054] Furthermore, to rationalize the number of possible scenarios, pruning is applied to reduce the number of computed states in each scenario and exclude scenarios with low probability from further detailed consideration. Furthermore, since the states are modeled statically over time in 4 dimensions and by generating probability fields in these volumes, fast computation and pruning of non-random states are possible.

[0055] Because this is a multi-sequence system where autonomous robots can "choose" to "behave" differently based on non-fully deterministic algorithms (e.g., ML-based non-fully deterministic algorithms), a sequence-by-sequence validation approach for each object would still be resource-intensive. To validate the different possible sequences, the simulation system stores all possible sequences in its own storage. While it is highly likely that most sub-volumes of state will share some sub-sequences for different objects, this means that the same part of the model can be used for different sequences, eliminating duplicate computations. For example, if objects A and B share the same space for both sequences X and Y, it is only necessary to perform a kinematic simulation once for both sequences and share the results between the sequences that resulted in the same clustering of objects A and B during the execution of the industrial scenario.

[0056] Figure 2 The key elements of the present invention are schematically illustrated. First, a sequence generator 200 is used to generate different possible sequences for objects (equipment, such as robots, conveyors, AGVs, etc.) in an industrial scenario, such as a production process. At the end of this step, a sequence is selected for further processing based on possible constraints or criteria, such as KPIs, product delivery dates, availability of required resources and materials, etc. Based on the object's instructions for creating sequences, this step delivers a number of possible solutions 240, 250, 260 for the object and generates a workflow for the object, in this case, a robot 210, a mobile robot 220, and an AGV 230.

[0057] Based on this sequence generation, a static 5D structure (X, X, Z, T, S) will be created for the environment for the industrial scenario, where the position of an object will be determined by the object's position X, Y, Z and the time T that the object is at this position and the related sequence S. This static 5D structure will be considered the starting point of the simulation and will be called the environment reference.

[0058] For each object, the simulation will execute the workflow given by the sequence and will then calculate the possible positions and times throughout the entire process defined by the sequence. The results of this simulation will be used to update the environment reference. From the perspective of the position tree 270, the results are displayed for the objects being in different positions in space and time during the workflow. Because objects will not necessarily be in the positions they are thought to be in due to minor environmental changes, each volume in the environment reference will have a certain probability that the object will actually be in it. Nevertheless, all possible positions of the objects are stored in the updated environment reference, revealing that some objects will share the same subvolume with a probability density similar to that in other sequences. These objects and their position responses are therefore stored in the same volume of the environment reference for further, more in-depth evaluation.

[0059] Now, once all containers (volumes, subvolumes) in the context reference are updated, a procedure is applied to traverse the complete structure of the context reference and find those objects that are in the same space at the same time (for the same sequence or for another sequence). Since these objects are in the same "container", these objects will be found. These results are in Figure 1 The final location tree 280 is shown in FIG2 without applying the concept of probability. For matches of objects that are in the same space at the same time, a probability value is calculated that the two objects actually share the same space at the same time. This probability value is compared with a predefined threshold, which helps to prune those matches with probability values ​​below the predefined threshold. Figure 1 In the example, those matches with probability lower than 1% are pruned.

[0060] For matches with probability values ​​above a predefined threshold, detailed kinematic calculations are performed on the objects from a specific sequence to ensure the safety and correctness of the industrial scenario environment during execution. If more than one sequence shares the same combination of objects with similar probabilities in space and time, the kinematic simulation is performed only once and the results are shared between the two sequences (or even more than two sequences).

[0061] All simulation results from all volumes containing more than one object are collected. A final conclusion is then drawn to determine whether the industrial scenario can be executed safely and correctly, and a final report is issued, demonstrating the correctness and safety of the industrial scenario. In cases where the simulation still produces a match that cannot be considered to meet safety requirements and the probability of failure is high, a correct report is issued, identifying the relevant uncertain locations and times in the environment.

[0062] Figure 3 An example of a sub-manufacturing scenario is now illustrated in detail. Two AGVs 310 and 320, a CNC machine tool 330, and two stationary robotic arms 340 and 350 are presented as resources. The P value is defined as a probability threshold that requires a match of objects with a high probability of being in the same space at the same time as the threshold P to undergo deep kinematic calculations to check for actual collisions.

[0063] Now assume a relatively simple workflow in the sequence of this industrial scenario. Robot 340 is loading two AGVs 310, 320 (respectively, in sequence) with material to be taken to robot 350. Robot 350 is unloading the material to CNC machine 330 and will obtain the generated result object.

[0064] Two possible subsequences are generated. In both, the time is 130 seconds after the start of the sequence. In both, AGV 310 approaches robot 350 while robot 350 is unloading AGV 320. Therefore, the two subsequences are in the same volume at the same time (note that the volume is actually 4-dimensional, so time is considered an additional dimension). At each coordinate in the volume, a probability of the AGV being there is formed based on the physical parameters of the AGV's activity. The volume boundaries can be defined by the speed, possible orientation and 3D pose of the AGV or robot, and the probability of the boundaries is low while the probability of the core is high. Therefore, due to the robot speed and possible lack of sensor coverage, the probability of risk (p) is above a certain threshold of the risk of collision with robot 350 during the unloading process.

[0065] Because p > P, a risk analysis is required, which means performing in-depth kinematic calculations. In practice, assume that the risk analysis indicates that the risk will only occur if AGV 320 is delayed, for example due to some internal maintenance cycle, and arrives 5 seconds later than robot 350. Therefore, this event is assigned a probability of p = 1.

[0066] For further consideration, let's assume that the center of the volume of robot 350 has Cartesian coordinates (3, 2, 4). The environment reference (X, Y, Z, T, S) for this location will look like this: For volume (3, 2, 4), at time 130, for sequence 1 and sequence 2:

[0067] (3, 2, 4, 130, 1) and (3, 2, 4, 130, 2)

[0068] There will be two objects at (3, 2, 4, 130, 1) and (3, 2, 4, 130, 2): AGVs 310 and 320. For AGV 320, there will be a probability P (P > p) that AGV 320 will be there. So, in summary, this means that for volume (3, 2, 4) at time 130, there is a collision probability p in both sequence 1 and sequence 2. Clearly, from this simple consideration, it's clear that there are two objects at the same time 130, simply by examining the environment reference, i.e., the structure in volume (3, 2, 4, 130).

[0069] Since all other variables are the same and there is no need to repeat the calculation twice, a deep calculation is performed for both sequences to find the collision. If the result of the calculation is that the collision is unavoidable, a report is generated, indicating the probability p of a collision and failure in this manufacturing process that will occur in both sequences.

[0070] Because p always remains smaller than P in other sequences, the other options in the environment reference of this volume are not calculated. Therefore, it can be observed that no deep kinematic calculations need to be performed, thus saving computational resources.

[0071] Figure 4 Another simple example of a method according to the present invention is illustrated. The resources presented in this example are an AGV with an attached robotic arm 410 and two stationary robots 420 and 430. The workflow for this example is as follows: Robot 420 is loading material from point A1 (e.g., the destination end of a first conveyor) to point B1 (e.g., a first stack). Robot 430 is loading material from point A2 (e.g., the destination end of a second conveyor) to point B2 (e.g., a second stack). AGV 410, tasked with assembling components between robots 420 and 430, is moving from point C to point D.

[0072] For risk analysis, assume the following scenario: If point B1 or point B2, e.g., the first stack and the second stack, are higher than 1.5 meters, which is 5 loads of material, then due to the volume of the robotic arm, the AGV 410 may be able to collide with the relevant robot during loading. It is taken into account in the control program of the AGV 410 that the AGV will change its path to bypass the first stack. In this case, the AGV 410 will be closer to the second stack (from B1 to B2, or in the opposite direction).

[0073] According to the original plan, two possible subsequences are generated: one subsequence at time 5 and the other subsequence at time 12. At these times, the two stacks are high. At both times, the AGV 410 tries to avoid the robot 420 and is very close to point B2 where the robot 430 is located. The combined probability of that happening is p2, where p2 > P. The volume of the AGV 410 is (2, 2, 2). The environmental reference (X, Y, Z, T, S) at that location will look like this:

[0074] For the volume (2, 2, 2), at time 5 and time 12, for sequences 1 and 2:

[0075] (2, 2, 2, 5, 1) and (2, 2, 2, 12, 2)

[0076] At (2, 2, 5, 1) and (2, 2, 2, 12, 2), there will be two objects: the AGV 410 and the robot 430. For the AGV 410, there will be a probability of p2 greater than P that the AGV 410 will be at that location at that time. So, overall, this means that for the volume 2, 2, 2 at time 5 and time 12, there is a collision probability of p2 in sequences 1 and 2.

[0077] Since it is not necessary to repeat the calculation twice because all other variables are the same, the depth calculation for finding a collision will be performed for one of the sequences. If the result of the calculation is that the collision is inevitable, a relevant report will be generated stating that the probability of a collision and a failure in this manufacturing process (which will occur in both sequences) is 0.1.

[0078] Since in other sequences, p < P, the other options in the ER for this volume are not calculated, so no depth kinematic calculation is required, thus saving computational resources.

[0079] Figure 5 FIG. 500 illustrates a flowchart of a multi-state simulation method for verifying the safety of an industrial scenario within a predetermined area and within a predefined time period. The method includes the following actions:

[0080] At act 510 , a static 4D or 5D structure of an environment is determined, wherein possible locations of objects are determined based on their locations and the times at which they will be at those locations.

[0081] In action 520 , possible spatial trajectories of objects, such as humans, production parts, stationary and mobile robots, AGVs, etc., are determined in a predefined area during an industrial scenario, such as a production process, assembly process, material handling, item sorting, etc.

[0082] At act 530 , the position and time of the object are updated in the static 4D or 5D structure according to the trajectory.

[0083] In action 540, possible matches of objects that appear at different locations at different times are identified, and the probability of the objects appearing at the same time in the same volume is evaluated against a predefined threshold. Optionally, matches with probabilities below a predefined threshold may be pruned at this stage.

[0084] At act 550 , detailed kinematic calculations are performed on those matches with probabilities above a predefined threshold to confirm whether the matches meet a predefined safety threshold.

[0085] At act 560, those matches that do not meet the predefined security threshold are reported accordingly.

[0086] Of course, those skilled in the art will recognize that certain steps in the above-described processes may be omitted, performed concurrently or sequentially, or performed in a different order unless specifically indicated or required by the sequence of operations.

[0087] Those skilled in the art will recognize that, for the sake of simplicity and clarity, the entire structure and operation of all data processing systems suitable for use with the present disclosure are not illustrated or described herein. Instead, only those data processing systems that are unique to the present disclosure or necessary for understanding the present disclosure are illustrated and described. The remainder of the structure and operation of data processing system 100 may conform to any of the various current implementations and practices known in the art.

[0088] It is important to note that while the present disclosure includes a description in the context of a fully functional system, those skilled in the art will understand that at least some of the mechanisms of the present disclosure can be distributed in the form of instructions contained in a machine-usable, computer-usable, or computer-readable medium in any of a variety of forms, and that the present disclosure applies equally regardless of the particular type of instruction or signal-bearing medium or storage medium used to actually perform the distribution. Examples of machine-usable / readable or computer-usable / readable media include: non-volatile hard-coded type media, such as read-only memory (ROM) or electrically erasable programmable read-only memory (EEPROM), and user-recordable type media, such as floppy disks, hard drives, and compact disk read-only memories (CD-ROMs) or digital versatile disks (DVDs).

[0089] Although the exemplary embodiments of the present disclosure have been described in detail, those skilled in the art will understand that they can make various changes, substitutions, variations and alterations as disclosed herein without departing from the spirit and scope of the disclosure in its broadest form.

[0090] Nothing in this application should be read as implying that any particular element, step, or function is essential to the claims scope: the scope of the patented subject matter is limited only by the allowed claims.

Claims

1. A multi-state simulation method for verifying the safety of an industrial scenario within a predefined area and a predefined time period, wherein: The industrial scenario is a production process and / or an assembly process and / or a material handling process and / or an item sorting process, and wherein the industrial scenario involves movement of objects within a predefined area and a predefined time period, and the method comprises the following steps: a) determining a static 5D structure of the predefined area, wherein a possible position of the object is determined based on the position of the object and the time when the object will be at the position; b) determining, in the predefined area, different sequences of objects in the industrial scene according to spatial trajectories of the objects, the objects comprising at least one of a human, a production component, a fixed and mobile robot, and an AGV during the industrial scene; c) selecting one of the different sequences for further processing based on constraints and / or predetermined criteria, wherein the constraints and / or predetermined criteria include at least one of a KPI, a delivery date for the product, and availability of required resources and materials; d) updating the position and time of the object for the selected sequence according to the spatial trajectory in the static 5D structure; e) for a selected sequence, identifying matches of objects that appear at different locations at different times, and evaluating the probability of the objects appearing in the same volume at the same time against a predefined threshold; and pruning matches with a probability below the predefined threshold; f) performing detailed kinematic calculations on the matches with a probability higher than the predefined threshold to confirm whether the matches with a probability higher than the predefined threshold meet a predefined safety threshold; and g) reporting matches that do not meet said predefined security threshold.

2. The method according to claim 1, wherein For matches that do not meet the predefined safety threshold, the spatial trajectory is modified, and steps b) to g) are repeated.

3. A data processing system (100) for multi-state simulation for verifying the safety of an industrial scenario within a predefined area and a predefined time period, the data processing system comprising a processor (102) and an accessible memory (108), wherein: The industrial scenario is a production process and / or an assembly process and / or a material handling process and / or an article sorting process, and wherein the industrial scenario involves movement of objects within a predefined area and a predefined time period, and the data processing system (100) is configured to perform the following steps: a) determining a static 5D structure of the predefined area, wherein a possible position of the object is determined based on the position of the object and the time when the object will be at the position; b) determining, in the predefined area, different sequences of objects in the industrial scene according to spatial trajectories of the objects, the objects comprising at least one of a human, a production component, a fixed and mobile robot, and an AGV during the industrial scene; c) selecting one of the different sequences for further processing based on constraints and / or predetermined criteria, the constraints and / or predetermined criteria comprising at least one of a KPI, a delivery date for a product, and availability of required resources and materials; d) updating the position of the object and the time for the selected sequence according to the spatial trajectory in the static 5D structure; e) for a selected sequence, identifying matches of objects that appear at different locations at different times, and evaluating the probability of the objects appearing in the same volume at the same time against a predefined threshold; and pruning matches with a probability below the predefined threshold; f) performing detailed kinematic calculations on the matches with a probability higher than the predefined threshold to confirm whether the matches with a probability higher than the predefined threshold meet a predefined safety threshold; and g) reporting matches that do not meet said predefined security threshold.

4. The data processing system (100) according to claim 3, wherein: For matches that do not meet the predefined safety threshold, the spatial trajectory is modified, and steps b) to g) are repeated.

Images