Method and system for determining position of plurality of lidar sensors of industrial hazardous area

Abstract

A system and method for determining positions of a plurality of lidar sensors for reliably detecting traversal of a hazardous area in an industrial environment. The system receives data regarding a geometry of a hazardous area, data regarding a set of moving objects within the area, and data regarding a set of fixed objects within the area. The system receives data regarding a total swept volume of all swept volumes combining all motion operations of all moving objects of the unit. The system creates a set of zone slices including slices of zone boundaries and a set of obstacle-shaped slices, and determines positions of a set of configured sensors on the boundary slices, such that any crossing of the hazardous area slice by a moving entity greater than a minimum detectable size can be detected by at least one configured sensor, even by detecting a blocking effect in consideration of a combination of the set of obstacle-shaped slices.

Description

Technical Field

[0001] The present disclosure generally relates to computer-aided design, visualization, and manufacturing ("CAD") systems, product lifecycle management ("PLM") systems, product data management ("PDM") systems, production environment simulations, and similar systems that manage data about products and other items (collectively, "product data management" systems or PDM systems). These systems may include components that facilitate the design and simulation testing of product structures and product manufacturing. Background Art

[0002] In industrial manufacturing, many facilities are often densely populated, for example, by several robots, other equipment, and mobile or mobile entities (e.g., like people or automated guided vehicles (AGVs)). Hazards to mobile entities and equipment are a well-known and critical issue. To prevent such hazards, a typical conventional solution involves erecting safety fences around hazardous areas in industrial facilities.

[0003] Modern industry is trying to remove the need for such safety fences, for example by adopting smaller and slower robots or by using new technical solutions for large and powerful robots.

[0004] For example, light detection and ranging ("LiDAR") sensors can be used for safety coverage so that traditional fences can be eliminated. LiDAR sensors positioned along the boundaries of a danger zone can be used to detect the passage of a mobile entity through such a danger zone and activate corresponding emergency actions, such as signaling, sounding an alarm, generating an emergency stop signal, stopping the robot, etc.

[0005] However, hazardous areas often contain both mobile and stationary industrial objects that can block the detection range of lidar sensors. Consequently, currently known technologies unfortunately do not provide industrial workers with a reliable and optimal solution for determining where to position lidar sensors on the boundaries of hazardous areas in lidar-based safety setups.

[0006] Improved techniques for determining the position of lidar sensors to reliably detect risky zone crossings in industrial environments are desired. Summary of the Invention

[0007] Various disclosed embodiments include methods, systems, and computer-readable media for determining the positions of multiple lidar sensors for reliably detecting the traversal of a hazardous area in an industrial environment; wherein the hazardous area includes a set of mobile objects and a set of fixed objects, and wherein traversal of the hazardous area by a mobile entity must be detected by at least one lidar sensor. The method includes receiving data regarding the geometry of the hazardous area, data regarding the set of mobile objects within the area, and data regarding the set of fixed objects within the area, wherein the multiple sensors are to be positioned at the boundary of the hazardous area. The method also includes receiving data regarding a total swept volume (a total swept volume) combining all motion operations of all mobile objects of a unit. The method also includes receiving data regarding a minimum size of a mobile entity for which traversal of the area is to be detected; hereinafter referred to as a minimum detectable size. The method also includes receiving or determining data for configuring the multiple lidar sensors. The method also includes determining a set of regional obstacle shapes as a superposition of the total swept volume shape and the shapes of the set of fixed objects; whereby a regional obstacle located between a given sensor and a given solid portion has the effect of blocking sensor detection of the solid portion. The method further includes creating a set of area segments or slices comprising an area boundary and the set of obstacle-shaped segments or slices. The method further includes determining, for each area slice, a position of a set of configured sensors on the boundary slice such that any traversal of the hazardous area slice by a mobile entity larger than a minimum detectable size is detectable by at least one of the configured sensors even by taking into account a combined detection blocking effect of the set of obstacle-shaped slices.

[0008] The features and technical advantages of the present disclosure have been summarized quite broadly above so that those skilled in the art can better understand the detailed description below. Hereinafter, the other features and advantages of the present disclosure that form the subject matter of the claims will be described. It will be understood by those skilled in the art that they can easily use the disclosed concepts and specific embodiments as the basis for modifying or designing other structures for achieving the same purpose of the present disclosure. It will also be appreciated by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the present disclosure in its broadest form.

[0009] Before proceeding to the following detailed description, it may be helpful to set forth definitions of certain words or phrases used throughout this patent document: the terms "include" and "comprise" and their derivatives mean including, without limitation; the term "or" is inclusive, meaning and / or; the phrases "associated with" and "associated therewith" and their derivatives may mean including, included within, interconnected with, contain, contained within, connected to or connected with, coupled to or coupled with, communicable with, cooperating with, interleaved, juxtaposed, proximate, bound to or bound with, having, having the property of, and the like; and the term "controller" means any device, system, or portion thereof that controls at least one operation, whether such device is implemented in hardware, firmware, software, or some combination of at least two of hardware, firmware, and software. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether local or remote. Definitions for certain words and phrases are provided throughout this patent document, and those of ordinary skill in the art will understand that such definitions apply in many, if not most, instances to prior and future uses of such defined words and phrases. Although some terms may encompass a wide variety of embodiments, the appended claims may expressly limit these terms to specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, wherein like reference numerals represent like objects, and wherein:

[0011] Figure 1 A block diagram of a data processing system is shown in which embodiments may be implemented.

[0012] Figure 2 A flow chart for determining positions of multiple lidar sensors for reliably detecting the crossing of a hazardous area in an industrial environment according to disclosed embodiments is schematically illustrated.

[0013] Figure 3 is a diagram schematically illustrating an example of an industrial hazardous area.

[0014] Figure 4 is a diagram schematically illustrating an example of an industrial hazardous area that can be covered via a lidar sensor according to the disclosed embodiments.

[0015] Figure 5is a diagram schematically illustrating an example of a danger zone slice according to the disclosed embodiment.

[0016] Figure 6 is a diagram schematically illustrating an exemplary embodiment of a two-dimensional ("2D") slice of a hazard zone model.

[0017] Figure 7 is a diagram schematically illustrating an exemplary embodiment of inputs and outputs of a module for determining the positions of a plurality of lidar sensors for industrial hazardous areas.

[0018] Figure 8 is a diagram schematically illustrating an example embodiment of slicing a data set for machine learning ("ML") training.

[0019] Figure 9 is a diagram schematically illustrating an exemplary embodiment of the use of a genetic algorithm for determining the position of a lidar sensor. DETAILED DESCRIPTION

[0020] Discussed below Figures 1 to 9 The various embodiments used to describe the principles of the present disclosure in this patent document are merely illustrative and should not be construed in any way to limit the scope of the present disclosure. Those skilled in the art will appreciate that the principles of the present disclosure can be implemented in any appropriately arranged device. Many innovative teachings of the present application will be described with reference to exemplary, non-limiting embodiments.

[0021] Previous technologies were unable to determine the positions of multiple lidar sensors in hazardous areas in an optimal and efficient manner. For example, previous technologies were based on manual / non-automatic positioning, trial and error, and required too much time and effort.

[0022] The embodiments disclosed herein provide numerous technical benefits, including but not limited to the following examples.

[0023] Embodiments enable calculation of the number of required lidar sensors for covering industrial hazardous areas.

[0024] Embodiments enable determining where to place each lidar sensor in order to obtain complete and safe coverage of hazardous areas, thereby preventing harm to human life or equipment.

[0025] Embodiments enable determining where to place each lidar sensor in an automatic and efficient manner.

[0026] The embodiments are based on real robotic tasks performed by robots operating in hazardous areas.

[0027] The implementation ensures safe and optimal coverage by the workstation's lidar sensor.

[0028] The implementation is based on a swept volume and, therefore, the solution found is time independent.

[0029] Embodiments enable digitally planning and verifying LiDAR-based security setups for workstations.

[0030] Figure 1 A block diagram of a data processing system 100 is shown in which embodiments may be implemented as a PDM system, for example, configured by software or otherwise specifically to perform the processes described herein, and in particular as each of the plurality of interconnection and communication systems described herein. The data processing system 100 shown may include a processor 102 connected to a L2 cache / bridge 104, which in turn is connected to a local system bus 106. The local system bus 106 may be, for example, a Peripheral Component Interconnect (PCI) architecture bus. In the example shown, main memory 108 and a graphics adapter 110 are also connected to the local system bus. The graphics adapter 110 may be connected to a display 111.

[0031] Other peripheral devices such as a local area network (LAN) / wide area network / wireless (e.g., WiFi) adapter 112 may also be connected to the local system bus 106. An expansion bus interface 114 connects the local system bus 106 to an input / output (I / O) bus 116. The I / O bus 116 connects to a keyboard / mouse adapter 118, a disk controller 120, and an I / O adapter 122. The disk controller 120 may be connected to a storage device 126, which may be any suitable machine-usable or machine-readable storage medium, including but not limited to non-volatile hard-coded type media such as read-only memory (ROM) or electrically erasable programmable read-only memory (EEPROM), magnetic tape storage devices, and user-recordable type media such as floppy disks, hard drives, and compact disk read-only memories (CD-ROMs) or digital versatile disks (DVDs), as well as other known optical, electrical, or magnetic storage devices.

[0032] In the example shown, an audio adapter 124 is also connected to I / O bus 116. Speakers (not shown) can be connected to audio adapter 124 for playing sounds. A keyboard / mouse adapter 118 provides a connection for a pointing device (not shown) such as a mouse, trackball, trackpointer, touch screen, etc.

[0033] It will be understood by those of ordinary skill in the art that Figure 1The hardware shown in the drawings may vary for a particular implementation. For example, other peripheral devices such as optical disk drives may be used in addition to or in place of the hardware shown. The examples shown are provided for illustrative purposes only and are not meant to imply architectural limitations to the present disclosure.

[0034] A data processing system according to an embodiment of the present disclosure may include an operating system that utilizes a graphical user interface. The operating system allows for the simultaneous presentation of multiple display windows within the graphical user interface, wherein each display window provides an interface to a different application or to a different instance of the same application. A cursor within the graphical user interface may be manipulated by a user using a pointing device. The cursor position may be changed and / or an event, such as a click of a mouse button, may be generated to trigger a desired response.

[0035] One of various commercially available operating systems may be used, with appropriate modifications, such as Microsoft Windows, a product of Microsoft Corporation of Redmond, Washington. TM Modify or create an operating system according to the present disclosure as described.

[0036] LAN / WAN / wireless adapter 112 can connect to network 130 (not part of data processing system 100), which can be any public or private data processing system network or combination of networks known to those skilled in the art, including the Internet. Data processing system 100 can communicate via network 130 with server system 140, which is also not part of data processing system 100 but can be implemented as, for example, a separate data processing system 100.

[0037] Figure 2 A flow chart is shown for determining the positions of multiple lidar sensors for reliably detecting the crossing of a hazardous area in an industrial environment according to the disclosed embodiments. Such a method can be performed, for example, by Figure 1 However, the "system" in the following process may be any device configured to perform the described process.

[0038] The danger zone includes a set of mobile objects and a set of fixed objects. The crossing of the danger zone by a mobile entity must be detected by at least one lidar sensor.

[0039] At act 205 , data regarding the geometry of a danger zone, data regarding the set of mobile objects within the zone, and data regarding the set of fixed objects within the zone are received, wherein a plurality of sensors are to be positioned on a boundary of the danger zone.

[0040] At action 210, data is received regarding a total swept volume of all swept volumes for all motion operations of all mobile objects of the grouped unit. In an embodiment, the total swept volume is generated based on the data of the motion operations of the group of mobile objects via a virtual simulation system. Examples of virtual simulation systems include, but are not limited to, computer-aided robotic tools, Process Simulate (a product of the Siemens PLM software suite), robotic simulation tools, and other systems for industrial simulation. For example, the CAR tool generates a total swept volume within a hazardous robotic area by simulating all received robotic operations of all operating robots.

[0041] At act 215 , data regarding a minimum size of a mobile entity for which traversal of the area is to be detected is received; this is hereinafter referred to as the minimum detectable size. Examples of mobile entities include, but are not limited to, a person or an automated guided vehicle (AGV). Examples of the minimum detectable size may be the size of a person's hand or the size of a head.

[0042] At act 220 , data for configuring a plurality of lidar sensors is received or determined.

[0043] At act 225, a set of area obstacle shapes is determined as a superposition of the total swept volume shape and the fixed object group shape.Area obstacles located between a given lidar sensor and a given solid part have the effect of blocking sensor detection of the solid part.

[0044] At act 230 , a set of region slices is created that includes a slice of the region boundary and a slice of the set of obstacle shapes.

[0045] At action 235, for each area slice, a set of configured sensors are positioned on the boundary slices so that any crossing of the hazardous area slice by a mobile entity larger than the minimum detectable size can be detected by at least one configured sensor even by taking into account the combined detection blocking effects of the set of obstacle shape slices.

[0046] In embodiments, sensor positions are determined via an optimization algorithm. As will be readily appreciated by those skilled in the art, the choice of optimization and algorithm type depends on the given and received parameters and the variables to be determined and / or optimized. For example, in embodiments, the number of sensors and their configuration are given, and optimization then involves finding the optimal position of the sensors to ensure detection of area crossings of minimum-sized entities. In other embodiments, the number of sensors or sensor configuration is determined, and optimization then involves finding the optimal number, position, and configuration of sensors to ensure detection of area crossings of minimum-sized entities.

[0047] In embodiments, the sensor group positions can be determined by applying an ML training module. The input to the ML training module includes at least data regarding a region boundary slice, data regarding an obstacle shape group, and data regarding a minimum detectable size. The output of the training module includes at least the configured sensor group positions. In embodiments, the ML training module is trained using an input training dataset comprising at least data regarding a region boundary slice and data regarding an obstacle shape slice, and an output training dataset comprising at least the configured sensor group positions along the region boundary. In embodiments, the obstacle shape slices used in the training dataset may include randomly defined shapes.

[0048] In an embodiment, the sensor group locations may be determined by applying a genetic algorithm.

[0049] In embodiments, the number of sensors is received or predefined. In other embodiments, the number of sensors is determined. In embodiments, if the step of determining the sensor group position does not return a valid result, fine-tuning can be applied by increasing the number of sensors and / or by changing the sensor configuration data.

[0050] In embodiments, the terms “received / receive / receiving” as used herein may include retrieving from a storage device, receiving from another device or process, receiving via interaction with a user, or receiving in other ways.

[0051] Algorithm of the exemplary embodiment

[0052] In an exemplary embodiment, the following Figures 3 to 9 The main algorithmic stages and steps for determining the positions of multiple lidar sensors for reliably detecting the crossing of hazardous areas in an industrial environment are shown.

[0053] Figure 3is a diagram schematically illustrating an example of an industrial hazardous area 301. The boundary 302 of the hazardous area 301 is defined by a fence. The hazardous area 301 includes two robots and other equipment, devices, or objects, some of which are moving and some of which are not moving and are therefore in fixed positions. The fence depicted on the area boundary 302 is depicted below for illustrative purposes and is therefore intended to be the boundary of the hazardous area rather than a physical fence that blocks the passage of mobile entities. Therefore, the area boundary 302 can be considered an "invisible lidar safety fence" rather than a physical fence. The area boundary 302 defines the hazardous area 301. A lidar sensor (not shown) will be positioned on the area boundary 302. For example, the sensor can be placed on the floor or mounted on a pole, column, and / or tripod at different heights, so that the area boundary is conveniently defined not by a physical fence, but by the portion of the area boundary 302 that is invisible and covered by the sensor. Whenever a mobile entity like a person or an AGV vehicle crosses this boundary 302, a LiDAR sensor (not shown) should be able to detect such crossing so that LiDAR-based safety stations can be planned in industrial facilities.

[0054] Algorithm implementation may include one or more of the following main stages:

[0055] i) loading the virtual study onto the CAR tool;

[0056] ii) For each robotic operation, a swept volume (SV) is created;

[0057] iii) creating a set of lidar sensors;

[0058] iv) combining the virtual representation of the study, the virtual representation of the swept volume, and the virtual representation of the LiDAR sensor in a combined three-dimensional ("3D") model in which the fixed objects and the swept volume are superimposed (see Figure 4 );

[0059] v) Create a set of <<2D Slices>> segments that include the area to be covered, the sensor, the slices of the swept volume + the device (see Figure 5 and Figure 6 );

[0060] vi) Apply the algorithm (see Figure 7 ) to determine the LiDAR position based on the above data and the minimum detectable size of the detectable mobile entity received. The LiDAR positioning algorithm module can be based on the ML algorithm (see Figure 8 Examples of ML training data types) and / or genetic algorithms (see Figure 9 ).

[0061] In the first stage i), the data received from the virtual study are loaded onto the CAR tool. For example, the virtual study includes Figure 3 A virtual depiction of a danger zone 302 is shown. The danger zone includes mobile objects such as a robot and fixed equipment objects such as a table base. The virtual study includes a virtual depiction of the geometry of the danger zone boundary 302.

[0062] In the second phase ii), for each robotic operation of each robot, a total swept volume is generated. In an embodiment, the swept volume is generated by the CAR tool by considering all robotic operations of the robot and by considering all swept volumes of other mobile industrial objects and devices. In other embodiments, the total swept volume can be retrieved from a storage device or received from an external source without the need for the CAR tool to generate it.

[0063] In the third stage iii), a set of N lidar sensors is created based on the received configuration data, and their positions along the boundary of the danger zone are not yet determined. In some embodiments, the number of sensors N is received from a storage device, through interaction with a user, or in other ways. In other embodiments, the number of sensors N is calculated algorithmically. Examples of lidar configuration data include, but are not limited to, detection range, coverage sector in degrees, number of rays or angles between rays, and other lidar parameters.

[0064] In a fourth stage iv), a combined 3D virtual representation 401 is generated by combining the virtual data received or generated during the previous stages i) to iv), such as Figure 4 Schematically illustrated in the figure. Figure 4 is a diagram schematically illustrating an industrial hazardous area 401 with a swept volume and a lidar sensor according to an exemplary embodiment. By considering the motion volumes of the two robots and all possible robotic operations of the robotic unit, a swept volume 403 of the two robots can be generated within the CAR tool. Examples of robotic operations include, but are not limited to, welding, drilling, laser, cutting, coating, cleaning, picking, measuring and other operations. In an exemplary embodiment, the data describing the robot operations may include robot targets, such as <Cartesian+robot configuration> or <joint values>; commands to be executed on these robot targets; and the data may thereby be provided in the form of a text file or in the form of a 3D virtual object including such data. In Figure 4 , an exemplary lidar sensor 404 is depicted as a sector with a black circle and a set of beam arrays emanating therefrom. The robot swept volume 403 or fixed objects 405 act as obstacles by blocking the detection range of the sensor beam, thereby reducing the area covered by the sensor.

[0065] In a fifth stage v), the combined 3D virtual representation is sliced ​​505 into a set of segment slices 506. In the embodiment shown, the slices 506 have a 2D shape, in other embodiments (not shown) the slices 506 may have a 3D shape. Figure 5 Schematically, a 2D slice 506 with an obstacle 508 is shown taken from the combined 3D model 401 of the danger zone. Figure 5 , a front view of the combined 3D model 401 of the hazard zone is depicted, wherein horizontal lines 505 show representations of slices 506 taken at different heights on the combined 3D model 401, wherein the swept volume 303 is also depicted. In an embodiment, the lidar position problem is solved as a mathematical optimization problem implemented by slicing the segments into 2D or 3D segments or slices according to the 3D zone model.

[0066] exist Figure 5 In the lower part of FIG, four sketches of different 2D slices 506 are shown for illustration purposes. Each slice 506 includes a region boundary line 507 and three obstacles 508, which in each slice have different shapes depending on the height at which the slice cut 505 is made. For example, the slice cuts can start at a height of 20 cm and each cut can be repeated every 15 cm until a height of 140 cm is reached. Note that the four slices 506 are only graphical representations for illustration purposes and do not directly correspond to Figure 4 The shape of the regional 3D model 401. For example, even when Figure 4 When the boundary 302 in FIG. 5 has a polygonal shape, the boundary line 507 also has an elliptical shape, and the boundary shape 507 of each slice may change depending on the height at which the cut is made. Similarly, the obstacle slice 508 is a fixed object slice or a graphical representation from the swept volume slice for illustration purposes and is not related to the Figure 4 The swept volume of the 3D model corresponds directly to the fixed object. Obstacle 508 is filled with a dashed pattern and blocks sensor detection of the light beam of the lidar sensor 404.

[0067] In a sixth stage vi), an algorithm for determining the optimal positions of the set of lidar sensors is applied. Figure 6 FIG2 is a diagram schematically illustrating a danger zone slice / segment including lidar sensors and obstacle slices. N lidar sensors 404 are positioned at yet-to-be-determined locations along zone boundary 507 such that a minimum-sized mobile entity (not shown) can be detected by at least one beam of one lidar sensor 404 when crossing the zone boundary.

[0068] In an embodiment, it may be convenient to enable a user to provide input and exclude selected portions of the area boundary where sensors cannot be placed, such as reflecting areas of a station where devices should preferably not be placed. In such cases, the algorithm can calculate sensor positions on the area boundary outside of the excluded boundary portion and provide corresponding resulting positions. An example of an excluded area portion 610 is shown with a dashed line. Thus, in Figure 6 In the illustrative embodiment shown in FIG, the boundary 507 of the area where the lidar sensor can be applied is a continuous line 507 that does not include the dashed portion 610.

[0069] Examples of algorithms that may be applied to optimize the position of the lidar sensor include, but are not limited to, machine learning algorithms such as reinforcement learning, genetic algorithms, other optimization algorithms, and combinations thereof.

[0070] Figure 7 The inputs / outputs of module 701 for determining the positions of a plurality of lidar sensors according to the disclosed embodiments are schematically shown.

[0071] Module M LP 701 determines at least the position of each lidar sensor provided as output data 703. In an embodiment, the position of the lidar sensor can be defined by position and orientation coordinates (X, Y, Z, RX, RY, RZ). In an embodiment, the input data 702 of the lidar positioning module 701 can include one or more of the following data: the 2D geometry of the danger zone 507; the 2D shape of the obstacle 508 at the relevant height of each slice cut; the configuration data of the lidar sensor 404 (e.g., the coverage sector in degrees or radians, the number of rays, or the angle between rays); the number N of lidar sensors, the minimum detectable size of mobile entities. In an embodiment, the algorithm executed by the lidar positioning module 701 calculates the output data 703 based on the received input data 702 and based on a heuristic method by ignoring uncovered areas smaller than a predefined size. In an embodiment, using the heuristic method, the algorithm considers "uncovered" areas that are too small to be entered by a person or any other mobile entity as if these small areas were "covered", so that complete coverage is achieved. In an embodiment, the algorithm heuristically considers the small portion of the uncovered area that is inaccessible to mobile entities of minimum detectable size, as if full sensor coverage were achieved.

[0072] In embodiments, the input data 702 to the lidar position calculation module 701 includes the number N of lidar sensors. In other embodiments, the number N of sensors is calculated by the module 701 and is therefore included in the output data 703 but not in the input data 702. In general, the number N of sensors can be part of the input data 702—e.g., a user asks the lidar positioning module 701 where to position the N lidar sensors—or the number N of sensors is part of the output data 703—e.g., a user asks how many sensors should be used and where they should be positioned.

[0073] As will be readily understood by the skilled person, in embodiments, the region slices may have a 2D or 3D shape depending on the format of the height at which the slices are cut.

[0074] In embodiments where the height used to cut the slices 505 is in a numerical format (e.g., 45 cm), the resulting slices 506 have a 2D shape and the optimization algorithm solves a 2D problem. In other embodiments where the height used to cut the slices (not shown) is in a numerical range / interval format (e.g., 44 cm to 46 cm), the resulting slices 506 have a 3D shape, causing the optimization algorithm to solve a 3D problem.

[0075] In an embodiment, the input data 702 includes a virtual study, a lidar configuration. In an embodiment, the output data 703 includes lidar positions for safety coverage, thereby accounting for obstacles formed by the swept volume of each moving object as well as obstacles formed by fixed objects.

[0076] Example Implementation of ML Algorithms

[0077] In an exemplary embodiment, the module M for laser radar positioning LP The step of solving an ML algorithm (e.g., a reinforcement learning problem) is included. For example, assuming that the number of lidar sensors N is to be determined, the state may include changing the number of lidars and their locations, and the reward function may include a higher score for a larger coverage area and, optionally, for a smaller number of sensors. Heuristically, in an embodiment, small portions of uncovered areas that are physically inaccessible to humans may be treated as if these areas were fully covered.

[0078] In embodiments, training datasets used to train ML algorithms may be collected from real-world scenarios, or they may be synthetically generated with or without the use of simulated systems. Figure 8is a diagram schematically illustrating exemplary slices of a dataset for ML training according to the disclosed embodiments. Upper region slice 801 is an example of a slice in which obstacle 508 is generated by slicing a 3D model of a hazardous area having a swept volume. Lower region slice 802 is an example of a slice in which obstacle 808 is, for example, a synthetically generated fake obstacle based on various criteria, which may be user-defined or learned from collected historical facility unit data.

[0079] In a first exemplary embodiment of synthetic data generation, the main training data generation phase includes one or more of the following steps: using several virtual robots; defining the position of each robot; for each robot, generating some random tasks (e.g., positions); playing the simulation and generating a swept volume; slicing the swept volume into multiple 2D slices 801; exporting each SV slice 801 to a different test case; for each test case, defining the sensor data plus the area to be covered and running the algorithm, and collecting the output data set as training data for the artificial intelligence (AI) algorithm for modeling the lidar localization module 701.

[0080] In a second exemplary embodiment of synthetic data generation, the following steps are included: a "fake" list of 2D map slices 802 with fake obstacles 808 is generated, and such slices 802 are used as a training dataset to train the AI ​​algorithm. Advantageously, this does not require the use of simulations and / or the definition of virtual robots. Advantageously, this second exemplary embodiment can be a faster way to generate a training dataset for the AI ​​algorithm model of the lidar positioning module 701.

[0081] Exemplary Implementation of Genetic Algorithms

[0082] In an exemplary embodiment, the laser radar is used to locate M LP The module includes steps for solving a genetic algorithm problem. In an embodiment, the sequence or gonium of items is a list of lidar sensors that are a single solution. In an embodiment, according to a first technique, each item—a lidar sensor—is evaluated by how it adds net coverage to the coverage of the previous sensor. In an embodiment, according to a second technique, each solution (sequence of lidar sensors) is evaluated by how much total coverage it achieves. Embodiments may include a combination of the first and second techniques.

[0083] Figure 9FIG2 is a diagram schematically illustrating the use of an exemplary genetic algorithm for determining lidar sensor position according to the disclosed embodiments. A representation of a 2D slice of a danger zone 506 includes the lidar sensor 404 and an obstacle 508 as previously described. Sensors SA, SB 404 have a coverage sector (in degrees or radians) whose detection area is blocked by obstacle 508.

[0084] For example, assume that for a first sensor SA at a given location (X, Y, Z, RX, RY, RY), its coverage area is calculated 901 and turns out to be 25% of the total risk area to be covered. A second sensor S is added 902 at another location where the calculated coverage is 20%. B , which brings an additional net coverage of 5% (=25%-20%). Therefore, the two sensors S A 、S B The total coverage area 903 of 404 is 30% (25% + 5%). For example, assuming there is a third sensor S C (not shown), the first sensor S A The value of the second sensor S is its coverage area. B The value is its coverage area minus the value of the previous sensor S A Covered area, the third sensor S C The value is its coverage area minus the sensor S A and S B The coverage area of ​​a combination of sensors is calculated. In this way, sensors with greater coverage are passed to the next generation, also with some mutations, so as not to provide a more global and less localized solution. With each iteration, the genetic algorithm continuously tries and evaluates different sensor positions. The algorithm starts with N sensors and attempts to achieve full coverage of the area. In embodiments, if no solution is found, the number of sensors is increased to N+1, and the algorithm tries again until full coverage is achieved. Heuristically, in embodiments, small portions of uncovered area that are physically inaccessible to humans can be treated as if they were fully covered. In embodiments, unlike reinforcement learning algorithms, where the solution provided can be considered a single overall solution, with genetic algorithms, each sensor or sensor sequence or combination thereof is evaluated individually. In fact, while the order of sensor sequences is generally irrelevant or irrelevant in reinforcement learning algorithms, with genetic algorithms, the order is crucial in evaluating the value of adding each additional sensor on top of the other existing lidar sensors.

[0085] In summary, Table 1 below provides a high-level comparison of features of an AI-based approach and a genetic algorithm approach for calculating the position of a lidar sensor, according to an embodiment.

[0086]

[0087] Table 1: High-level comparison of AI-based algorithms and genetic algorithms

[0088] Examples of algorithms in module 701 include, but are not limited to, AI / ML algorithms, genetic algorithms, or any other algorithm that solves and optimizes output solutions 703 via heuristics for small uncovered areas given input data 702. In embodiments, module 702 may include a set of submodules that may also run multiple algorithms in parallel, for example, by collecting results, analyzing all results, and returning a pool of best-chosen solutions 703. In embodiments, module 702 may begin by applying an ML algorithm and use its generated results as a use case for the genetic algorithm, i.e., the genetic algorithm considers the ML algorithm result as one of the primary options and attempts to find a better option.

[0089] Of course, those skilled in the art will recognize that certain steps in the above-described processes may be omitted, performed concurrently or sequentially, or performed in a different order unless specifically indicated or required by the sequence of operations.

[0090] Those skilled in the art will recognize that, for the sake of simplicity and clarity, the entire structure and operation of all data processing systems suitable for use with the present disclosure are not shown or described herein. Instead, only such data processing systems as are unique to the present disclosure or necessary for understanding the present disclosure are shown and described. The remainder of the structure and operation of data processing system 100 may conform to any of various current implementations and practices known in the art.

[0091] It is important to note that while the present disclosure includes a description in the context of a fully functional system, those skilled in the art will understand that at least portions of the present disclosure can be distributed in the form of instructions contained in a machine-usable, computer-usable, or computer-readable medium in any of a variety of forms, and that the present disclosure applies equally regardless of the particular type of instruction or signal-bearing medium or storage medium used to actually perform the distribution. Examples of machine-usable / readable or computer-usable / readable media include non-volatile hard-coded type media such as read-only memory (ROM) or electrically erasable programmable read-only memory (EEPROM), and user-recordable type media such as floppy disks, hard drives, and compact disk read-only memories (CD-ROMs) or digital versatile disks (DVDs).

[0092] Although the exemplary embodiments of the present disclosure have been described in detail, those skilled in the art should understand that they can make various changes, substitutions, alterations and alterations as disclosed herein without departing from the spirit and scope of the disclosure in its broadest form.

[0093] Nothing in this application should be read as implying that any particular element, step, or function is an essential element that must be included in the claims scope: the scope of the patented subject matter is defined only by the allowed claims.

Claims

1. A method for determining the position of a plurality of lidar sensors by a data processing system, the plurality of lidar sensors being used to reliably detect the crossing of a hazardous area in an industrial environment; The danger zone includes a set of mobile objects and a set of fixed objects, and wherein traversal of the danger zone by a mobile entity must be detected by at least one lidar sensor; a) receiving data regarding the geometry of the danger zone, data regarding the set of mobile objects within the zone, and data regarding the set of fixed objects within the zone, wherein the plurality of sensors are to be positioned on a boundary of the danger zone; b) receiving data on a total swept volume combining all swept volumes of all motion operations of all mobile objects of the unit; c) receiving data on the minimum size of a mobile entity whose crossing of said area is to be detected; hereinafter referred to as minimum detectable size; d) receiving or determining data for configuring the plurality of lidar sensors; e) determining a set of regional obstacle shapes as a superposition of the total swept volume shape and the shapes of the set of fixed objects; whereby regional obstacles located between a given sensor and a given solid portion have the effect of blocking sensor detection of the solid portion; f) creating a set of region slices, which include slices of the region boundary and slices of the set of obstacle shapes; g) for each area slice, determining the positions of a set of configured sensors on said boundary slice such that any crossing of said hazardous area slice by a mobile entity larger than said minimum detectable size can be detected by at least one configured sensor even by taking into account a combined detection blocking effect of the set of obstacle shape slices.

2. The method according to claim 1, wherein The step of determining the position of the set of sensors is obtained by applying an ML training module, wherein the module receives as input at least data about the area boundary slice, data about the set of obstacle shapes, data about minimum detectable sizes, and wherein the module returns as output the position of at least the set of configured sensors.

3. The method of claim 1, wherein the step of determining the position of the sensor group is obtained by applying a genetic algorithm. The method of claim 1 , wherein the number of sensors is received or is to be determined.

5. The method according to claim 1, wherein The total swept volume is generated via simulation of data manipulated according to motion of the group of moving objects.

6. The method of claim 1, if the step of determining the sensor group position does not return a valid result, applying fine tuning by increasing the number of sensors and / or by changing the sensor configuration data.

7. The method according to claim 2, wherein: The ML training module is trained using an input training data set and an output training data set, wherein the input training data set includes at least data about the area boundary slice and data about the obstacle shape slice, and the output training data set includes at least the position of the configured sensor group on the area boundary.

8. The method according to claim 7, wherein: The obstacle shape slices used for the training dataset may include randomly defined shapes.

9. A data processing system comprising: processor; as well as Accessible memory, the data processing system is particularly configured to: a) receiving data on the geometry of the danger zone, data on the set of mobile objects within the zone, and data on the set of stationary objects within the zone, wherein the plurality of sensors are to be positioned on the boundary of the danger zone; b) receiving data on a total swept volume combining all swept volumes of all motion operations of all mobile objects of the unit; c) receiving data on the minimum size of a mobile entity whose crossing of said area is to be detected; hereinafter referred to as minimum detectable size; d) receiving or determining data for configuring the plurality of lidar sensors; e) determining a set of regional obstacle shapes as a superposition of the total swept volume shape and the shapes of the set of fixed objects; whereby regional obstacles located between a given sensor and a given solid portion have the effect of blocking sensor detection of the solid portion; f) creating a set of region slices, which include slices of the region boundary and slices of the set of obstacle shapes; g) for each area slice, determining the positions of a set of configured sensors on said boundary slice such that any crossing of said hazardous area slice by a mobile entity larger than said minimum detectable size can be detected by at least one configured sensor even by taking into account a combined detection blocking effect of the set of obstacle shape slices.

10. The data processing system according to claim 9, wherein: The step of determining the position of the set of sensors is obtained by applying an ML training module, wherein the module receives as input at least data about the area boundary slice, data about the set of obstacle shapes, data about minimum detectable sizes, and wherein the module returns as output the position of at least the set of configured sensors.

11. The data processing system according to claim 9, wherein: The step of determining the position of the sensor group is obtained by applying a genetic algorithm.

12. The data processing system according to claim 9, wherein: The number of sensors is received or is to be determined.

13. The data processing system according to claim 9, wherein: The total swept volume is generated via simulation of data manipulated according to motion of the group of moving objects.

14. The data processing system according to claim 9, wherein: The ML training module is trained using an input training data set and an output training data set, wherein the input training data set includes at least data about the area boundary slice and data about the obstacle shape slice, and the output training data set includes at least the position of the configured sensor group on the area boundary.

15. A non-transitory computer-readable medium encoded with executable instructions that, when executed, cause one or more data processing systems to: a) receiving data on the geometry of the danger zone, data on the set of mobile objects within the zone, and data on the set of fixed objects within the zone, wherein: The plurality of sensors are to be positioned on the boundary of the danger zone; b) receiving data on a total swept volume combining all swept volumes of all motion operations of all mobile objects of the unit; c) receiving data on the minimum size of a mobile entity whose crossing of said area is to be detected; Hereinafter referred to as minimum detectable size; d) receiving or determining data for configuring the plurality of lidar sensors; e) determining a set of regional obstacle shapes as a superposition of the total swept volume shape and the fixed object group shape; Thus, an obstruction of the area between a given sensor and a given solid part has the effect of blocking sensor detection of that solid part; f) creating a set of region slices, which include slices of the region boundary and slices of the set of obstacle shapes; g) for each area slice, determining the positions of a set of configured sensors on said boundary slice such that any crossing of said hazardous area slice by a mobile entity larger than said minimum detectable size can be detected by at least one configured sensor even by taking into account a combined detection blocking effect of the set of obstacle shape slices.

16. The non-transitory computer readable medium of claim 15, wherein: The step of determining the position of the set of sensors is obtained by applying an ML training module, wherein the module receives as input at least data about the area boundary slice, data about the set of obstacle shapes, data about minimum detectable sizes, and wherein the module returns as output the position of at least the set of configured sensors.

17. The non-transitory computer readable medium of claim 15, wherein: The step of determining the position of the sensor group is obtained by applying a genetic algorithm.

18. The non-transitory computer readable medium of claim 15, wherein: The number of sensors is received or is to be determined.

19. The non-transitory computer readable medium of claim 15, wherein: The total swept volume is generated via simulation of data manipulated according to motion of the group of moving objects.

20. The non-transitory computer readable medium of claim 15, wherein: The ML training module is trained using an input training data set and an output training data set, wherein the input training data set includes at least data about the area boundary slice and data about the obstacle shape slice, and the output training data set includes at least the position of the configured sensor group on the area boundary.

Images