Security monitoring

Abstract

The invention relates to security monitoring. A method for distributed training of a generic AI model (32) for security monitoring of an operating area (12) is provided in which at least one respective first sensor (16) generates sensor data by monitoring a respective operating area (12) at a plurality of locations (10), and at a respective location (10) of the plurality of locations (10), at least one second sensor (16) generates sensor data by monitoring a respective operating area (12). A respective classical security system monitors the respective operating area (12) and performs a security assessment using at least one respective second sensor (22), and trains the respective local AI model (18) using the sensor data and the security assessment of the respective location (10). Here, the training results of the local AI model (18) are transmitted to an orchestration server (28), and the orchestration server (28) generates or improves the generic AI model (32) on the basis of the training results.

Description

[0001] The present invention relates to a method and a safety system for distributed training of a general AI model for safety monitoring of an operating area.

[0002] Current safety technologies are based on classical evaluations. "Classical" is the opposite of machine learning methods or artificial intelligence (AI) methods. One aspect that hinders certification is the lack of a legal and regulatory basis. In safety technology, high requirements are placed on reliability and are specified in relevant standards, but these standards currently do not allow functions to be implemented with the aid of artificial intelligence. Safety standards such as the EN13849 machine safety standard and the EN61496 electrosensitive protective equipment (ESPE) standard require measures such as safety hardware or functional monitoring, thereby ensuring that hardware failures are detected and the evaluation runs according to the programming. The protection logic is limited to very simple functions that can be implemented using deterministic analysis algorithms. A well-known example in this regard is the monitoring of a protected field where the operator is not allowed to enter. On the other hand, there are tasks, especially in the field of image evaluation, such as object classification, especially person recognition, which can only be solved by artificial intelligence. Therefore, such functions are currently not available in safety technology applications.

[0003] In addition to formal obstacles, implementing safety functions with the aid of artificial intelligence or AI models also poses practical technical challenges. Common training methods require a large number of training examples to be provided in a central location. For this purpose, before using an AI model, a large amount of data from very different application areas must be collected, annotated, and their integrity and balance (Ausgewogenheit) evaluated. This requires a considerable amount of work, especially considering the high reliability requirements. Particularly valuable training data is obtained directly from the subsequent application environment. However, the subsequent application environment is usually an operating production environment or a logistics environment, and for confidentiality reasons, within the scope permitted by data protection laws, it is not desirable to disclose this data. Moreover, even if the data can be obtained, subsequent annotation will be very time-consuming and requires an accurate understanding of the corresponding data detection situation.

[0004] After training the AI model, in safety technology applications, a proof of functionality is required that achieves a reliability comparable to existing safety standards and thus far exceeds the conventional verification of traditional AI models. Due to the black-box nature of AI models, a theoretical proof based on the evaluation algorithm is not possible. At this point, release tests can sometimes be carried out under laboratory conditions, but ultimately it is also expected that tests must be carried out on-site. This in turn requires access to the operating area, which is often hindered by confidentiality reasons, and in any case, requires a considerable amount of additional work.

[0005] Also to be considered is the problem of data drift. Even if the training data represented a complete and balanced representation of the real operating environment during training, afterwards, due to the gradual changes in the environment, the reliability of the AI model may decline. Conventionally, stable detection features that are not affected by data drift are used here for argumentation. This does not work in AI models because the training itself selects the features, and these features are simply unknown.

[0006] The concept of federated learning is well-known in the field of machine learning. This involves training being carried out distributively among multiple computing nodes, in such a way that these computing nodes jointly build a robust model. Federated learning has not been developed for security technologies and no suggestions have been made in this regard so far.

[0007] The sensors used in security technologies are usually optoelectronic sensors, especially increasingly including cameras and more recently also 3D cameras. There are various techniques for 3D cameras, including time-of-flight methods, stereo methods, and projection methods or plenoptic cameras. However, as mentioned before, what is pursued is a relatively simple concept, such as the requirement that the protection field remains unobstructed, and even such classical image evaluation methods are extremely complex to implement. EP 3 859 382 A1 proposes a radio positioning system for locating radio transponders carried by persons. This radio positioning system is compared with the position determination of radar, ultrasonic sensors, or laser scanners. Therefore, reliable person identification is only possible after such sensor fusion, and the evaluation remains completely classical.

[0008] DE 10 2017 105 174 B4 discloses a method for generating training data for an artificial neural network, in which the recorded image data is automatically evaluated by a security sensor as security-critical or non-security-critical. However, this alone does not solve the problem mentioned at the beginning, because such training data still needs to be collected centrally, the security functions of the trained neural network must be verified, and the neural network is not resistant to data drift.

[0009] In US2021 / 0063578 A1, a deep neural network is used for the classification of objects for an autonomous vehicle. The labels of the camera are transferred to the lidar data. Among a long list of training methods, federated learning is only mentioned once and then not mentioned again.

[0010] US2022 / 0332335 A1 relates to the use of neural networks and federated learning for data analysis of vehicles.

[0011] Therefore, the object of the present invention is to provide an improved training method for AI models that can be used in safety technology applications.

[0012] This object is achieved by a method and a safety system for the distributed training of a general AI model for the safety monitoring of an operating area according to claim 1 or 12, respectively. The terms "safety" and "security" mean taking measures to control failures to a specified safety level. For example, such safety levels are divided into SIL 1 to SIL 4 (Safety Integrity Level) or PL a to PL e (Performance Level). For typical cases, this means complying with the conditions of relevant safety standards for machine safety or electrically sensitive protective equipment. For the implementation of safety functions using artificial intelligence, no such standards exist, and proof of equivalent reliability must be provided. An AI model is an evaluation block or computer program that uses machine learning methods, especially deep neural networks, to evaluate input data. Federated learning is a known concept outside of safety technology and is briefly described at the beginning. A general AI model is an integrated AI model generated by distributed training, different from the local AI models at different locations of the distributed training. The operating area includes at least one machine or other hazardous area or its surrounding environment. However, the operating area can be broader, for example, including production lines, aisles, and shelves, or even the so-called "factory level", i.e., the entire factory or logistics hall, where there are a large number of machines, hazardous points, sensors, and potentially people moving among them. The operating area is, in a safety-related sense, an area where accidents involving people can potentially occur and must therefore be prevented.

[0013] At multiple locations, at least one first sensor monitors the operating area there and generates corresponding sensor data. This corresponds to distributed training and is thus carried out decentralized at multiple locations. As part of federated learning, one location is assigned to a decentralized computing node. The AI model should work with the sensor data of the first sensor and thus preferably a large amount of sensor data, such as image data from cameras or point clouds from laser scanners or 3D cameras. Here, there can be multiple first sensors at one location, or even, depending on the complexity, especially at the "factory level", many first sensors with different sensor principles from each other.

[0014] Classic safety systems also monitor the operating area at multiple locations. To this end, the classic safety system has at least one second sensor, and thus the second sensor is assigned to the classic safety system rather than the AI model. Ultimately, the first sensor and the second sensor should be able to solve the same safety application problem, that is, to perform a safety assessment to prevent accidents at the required safety level, but they can be significantly different from each other in terms of the number, location, and type of sensors. The classic safety system obtains a safety assessment using a dedicated algorithm without machine learning or artificial intelligence. The safety assessment of the classic safety system is used to automatically annotate or label the sensor data of the first sensor for use as training examples. It should be emphasized that the first sensor and the second sensor respectively represent the roles belonging to the AI model and the classic safety system. Physically, they can be at least partially the same sensor, for example, a camera image evaluated in a classic manner and, for example, for a protection field is simultaneously used as sensor data for the AI model.

[0015] In this way, each local AI model at multiple locations has high-quality annotated training data available. Therefore, the purpose of performing local AI model training (preferably supervised learning) is its own safety assessment. Anyway, as long as the local AI model still needs to be trained and until the local AI model is certified suitable for safety applications, it is preferably not put into production temporarily. Alternatively, a simulation program can be used to retrain the local AI model that has already been put into production, as described in more detail later. Preferably, the basic architectures of the local AI models are the same as each other and are preferably initialized in the same way. Then, these local AI models will be different during the training process because they are trained using different sensor data at their respective locations.

[0016] So far, there are thus, decentralized at multiple locations, respective first sensors, local AI models, and classic safety systems with their second sensors, and the local AI models are trained asynchronously in the manner described. It should be emphasized that the second sensor can physically at least partially coincide with the first sensor. The multiple locations can be close to each other, such as at adjacent machines, but the training can also be distributed across different halls, companies, locations, regions, or even countries and continents.

[0017] The present invention is based on the following basic idea: collecting and integrating different local AI models at a central location to obtain a general AI model. This coordination and management is called orchestration. For this purpose, the training results of the local AI models are transmitted to a server, which is called an orchestration server according to its function. What is transmitted is not the training data, but the training results. For example, it can be weights, gradients or other information about the training results, or even a complete AI model or neural network. The only important thing is that the orchestration server receives enough information with which it can participate in the corresponding local training progress. For questions regarding the information to be transmitted and subsequently integrated into the general AI model, please refer to the literature on federated learning for supplementary reference. Preferably, the architecture of the general AI model corresponds to that of the local AI models. The general AI model can be generated for the first time from the local AI models and can also be improved, either by retraining the local AI models or by incorporating additional local AI models.

[0018] The method is a computer-implemented method that runs, for example, in the computing unit of a sensor and / or in a computing unit connected to the sensor, as well as in the orchestration server.

[0019] The advantages of the present invention are that distributed training creates the preconditions for AI-based safety functions. A technical ecosystem is created to implement AI functions in the context of functional safety. The system includes complex sensor data (such as 3D image data) and complex functions (such as safety object classification), and is scalable both in terms of the number of participating locations and the scale of the corresponding local safety applications, even for "factory-level" safety systems. Since only the training results are transmitted, and not, for example, the image data of a conventional training dataset, confidentiality interests, personal rights, and data protection are still safeguarded. All three issues discussed at the beginning are resolved in this context. First, high-quality, annotated training data is automatically obtained in a wide range of application scenarios, thus forming an extremely robust and powerful general AI model. Second, the general AI model can be verified using almost the same mechanisms according to strict safety technology standards (local verification, field testing), and then the correspondingly annotated sensor data is used as a ground-truth dataset. In this way, despite the black-box nature of the general AI model, the required safety level is still achieved. Third, continuous monitoring can be carried out to detect faults in order to counteract data drift, i.e., the gradual change of reality compared to past training data and the resulting related degradation of the general AI model.

[0020] Preferably, the orchestration server transmits the general AI model to at least one location where the general AI model is specifically used as a local AI model. Transmitting the AI model means transmitting sufficient information so that the AI model can be understood in whatever form. The general AI model generated from the training results of the local AI model is transmitted back to the location and preferably replaces the previous local AI model there. Thus, the location benefits from the training results of other locations. Alternatively, it is also conceivable to send the general AI model to a location that did not participate in the distributed training. It is assumed here that the locations for training are rich enough to allow generalization to other locations. Thus, after transmission, the local AI model at the location is the same as the general AI model. This situation may change again during subsequent processes due to local adaptation or local retraining.

[0021] Preferably, the local AI model evaluates the sensor data recorded by the first sensor at at least one location for safety assessment, in particular to initiate safety measures when a danger is recognized. The local AI model here is the general AI model obtained from the orchestration server or at least based on the general AI model obtained from the orchestration server. Thus, the general AI model can (also) take over the operations in production or at least tentatively, for example for verification. Then, in the operations in production, especially when the safety assessment recognizes a danger, safety measures are initiated. For example, the safety measures can be to stop or slow down the machine or initiate an avoidance movement. Thus, the method for distributed training is an initial or intermediate step of a method for using the general AI model or its derivatives or copies as local AI models to monitor the respective operating areas at each location.

[0022] Preferably, the classical safety system at least temporarily continues to perform safety assessment at at least one location. In particular, there may be a reference location where the classical safety system is installed and remains operational at least temporarily, while at other locations, the classical safety system may be removed or permanently deactivated, and there are also some locations where the classical safety system did not exist in the past and does not exist now. After the general AI model has been implemented as a local AI model, the classical safety system continues to perform safety assessment, which enables various implementation methods, which will now be described.

[0023] Preferably, the safety assessments of the classical safety system and the local AI model are compared with each other. The first conceivable purpose for this is to check whether the local AI model can still correctly process the current actual situation based on the sensor data. If the classical safety system and the local AI model do not match, data drift is suspected. This can be a trigger for retraining the local AI model or requesting an improved general local AI model from the orchestration server.

[0024] Preferably, when a classical safety system or a local AI model recognizes a danger, safety measures are initiated. Even if the classical safety system and the local AI model do not agree, it is irresponsible not to react to the recognized danger. Moreover, such situations are particularly valuable training examples. Additionally, if the local AI model is not solely responsible for safety (it may become solely responsible for safety after distributed training), diverse redundant monitoring increases the achievable safety level.

[0025] Preferably, the local AI model is verified based on a comparison. At this time, the local AI model has been generated from a general AI model, which means the latter has been practically verified, preferably at multiple locations. In other words, the field test of the general AI model is carried out at at least one location. As in training, the classical safety system generates a safety assessment for the sensor data of the first sensor. However, without excluding this possibility as part of retraining, the safety assessment is now generated not for training but sets the expected value ("ground truth") for the safety assessment of the AI model. Such a field test is carried out at least largely automatically, so that the operating area remains inaccessible to potentially unwanted personnel who want to conduct the field test (e.g., the operator of the orchestration server).

[0026] Preferably, the general AI model is optimized during an iterative process by: using the sensor data and the safety assessment of the corresponding location to retrain at least one local AI model; transmitting the training results of at least one local AI model to a coordination server; and the orchestration server improving the general AI model based on the training results. This describes a further iteration of the initial training, which can also be repeated multiple times to further optimize the general AI model, especially for updating against data drift. The corresponding safety assessment assigned as a label to the training dataset comes from at least temporarily parallelly used classical safety systems, local AI models, or other sources (e.g., manual annotation). The optimization can be carried out simultaneously with the operation of the already deployed local AI model. Preferably, an un-deployed copy of the local AI model is used for the corresponding local retraining. Once sufficient additional training data has been collected at sufficient locations, the orchestration server generates a new version of the general AI model and deploys it to these locations, preferably after re-verification and certification. For retraining, specific conditions can be set purposefully, such as in terms of the first sensor, the arrangement of the first sensor, the operating area, object properties, applications, environmental impacts, etc., to broaden the training basis or give an expected direction for retraining.

[0027] Preferably, the first sensor generates image data. Especially in the case of an AI model implemented as a neural network, there are a large number of powerful architectures and learning methods that are utilized in this way. In addition to conventional cameras, 3D cameras can also be envisaged. In this sense, the first sensor is preferably a safety sensor, i.e., its image data is reliably provided according to the required safety level. However, alternatively, if the first sensor itself is not safe, the first sensor and the AI model can also be combined to achieve the required safety level.

[0028] Preferably, the classical safety system has a radio positioning system (UWB, ultra-wideband). Such a system is described, for example, in EP 3 859 382 A1 mentioned in the introduction. However, the classical safety system is not limited to this. It is important to provide a safety assessment for the sensor data of the first sensor, such as a safety camera or a safety laser scanner, especially a sensor with protected field monitoring, and depending on the installation situation, simpler safety sensors such as safety gratings or door switches are also suitable for this.

[0029] Preferably, the safety assessment includes safety object classification, especially person recognition. There is no generally practical classical image assessment for this yet. So far, AI-based solutions have not been able to be compatible with functional safety, partly due to the lack of rich training data on relevant protection situations. The present invention can just make up for this defect. For example, a classical safety system with person recognition can be based on transponders carried by people or the existence of monitored access restrictions. If the AI model learns to safely recognize people, these auxiliary tools can be omitted later. Safety object classification is just an important example of a safety function. If the classical safety system provides another safety function, such as safety positioning or motion tracking, this can also be learned, and these safety functions can also be combined, such as safe person tracking. According to EP 3 859 382 A1, a combination as a tag-based radio positioning system is particularly preferred, and reliable person recognition can be achieved using this system. Here, an optical gate (i.e., a 3D camera system) is usually required to monitor the entry and exit of people not carrying tags, and its image data can serve a dual role as part of the classical system and as a provider of sensor data for the AI model.

[0030] The security system according to the present invention comprises an orchestration server and a plurality of computing nodes, each of which is located at one of a plurality of locations, wherein the orchestration server has a server computing unit and a first communication interface, and the computing nodes have respective AI computing units and a second communication interface so that an AI model or a training result of the AI model can be transmitted via the communication interface. At these locations, at least one respective first sensor is provided for generating sensor data by monitoring the respective operating area of the location, and a respective classic security system is provided for monitoring the respective operating area using at least one respective second sensor and for performing a security assessment. The respective AI computing unit is designed to train a respective local AI model using the sensor data and the security assessment of the respective location, and transmit the training result of the local AI model to the orchestration server, wherein the server computing unit is designed to generate or improve a general AI model based on the training result. In other words, the orchestration server and the computing nodes perform the method according to the present invention, and this is possible in all described embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Other features and advantages of the present invention will be described in more detail below based on exemplary embodiments and with reference to the accompanying drawings. In the accompanying drawings:

[0032] Figure 1 A schematic diagram is shown including locations of monitored operating areas, the locations representing computing nodes for distributed training;

[0033] Figure 2 An overview diagram of distributed training with one orchestration server and multiple compute nodes is shown;

[0034] Figure 3 An exemplary flow chart illustrating initial distributed training of a general AI model is shown;

[0035] Figure 4 An exemplary flow chart for dual safety assessment using classic safety systems and AI models for verification, data drift detection, and / or diversified redundancy monitoring is shown;

[0036] Figure 5 An exemplary flow chart illustrating security monitoring via a trained AI model; and

[0037] Figure 6 An exemplary flow chart for distributed retraining of a general AI model is shown.

[0038] Figure 1A schematic diagram showing the location 10 including the operation area 12 summarized by the factory logo is shown. At least one dangerous point or machine 14 to be monitored is located in the operation area 12, represented here by a robotic arm. The task of the safety monitoring described here is to prevent accidents between the machine 14 and the personnel 15 that may be located in the operation area 12. In the final effect, the first sensor 16 should be responsible for this or at least participate in it through the evaluation of the local AI model 18 (especially (deep) neural network) in the integrated or connected AI computing unit 20. However, the local AI model 18 must first be trained for this task, and since this local AI model is related to safety, it must be verified or certified according to the required safety level.

[0039] To obtain the annotated training data for the local AI model 18, a classical safety system including a second sensor 22 and a classical computing unit 24 is also provided. The classical safety system solves safety application problems in a known manner. Here, a safety assessment is performed. On the one hand, the machine 14 is identified to eliminate hazards, that is, to stop, decelerate, avoid, or take any appropriate measures to avoid accidents. On the other hand, the safety assessment is used as a label for the sensor data of the first sensor 16 recorded simultaneously, so that its sensor data automatically becomes the annotated training data of the local AI model 18.

[0040] Figure 1 The first sensor 16 and the second sensor 22 are shown as separate units respectively. In fact, in practical applications, they are often different devices. But first, only the functional roles are involved: the first sensor 16 is assigned to the AI model 18, and the second sensor 22 is assigned to the classical safety system. Physically, the same device can play both roles, such as a camera or a 3D camera, which performs classical evaluation within the safety system and simultaneously provides image data for the AI model.

[0041] The present invention is based on distributed training, especially federated learning. After the training is completed, such as after a process of a predetermined duration or after reaching a predetermined number of training data sets, the local AI model has not been set to be put into production. Instead, information about the training progress, that is, for example, the weights of the neural network or the gradients of the residuals, is output via the interface 26 to the orchestration server. At the orchestration server, the local AI models 18 of multiple locations 10 are collected and integrated into a general AI model, and then the general AI model is transmitted back to the location 10 and / or other locations. The distributed training will be elaborated in more detail later. In the terminology of federated learning, the location 10 is called a computing node, especially an AI computing unit 20.

[0042] The AI computing unit 20 and the classical computing unit 24 are not limited to specific hardware. On the contrary, even only one common hardware component or any number of hardware components can provide the required computing power, communication capabilities, and storage capabilities. Examples are digital computing components such as microprocessors or CPUs (Central Processing Units), FPGAs (Field Programmable Gate Arrays), DSPs (Digital Signal Processors), ASICs (Application-Specific Integrated Circuits), AI processors, NPUs (Neural Processing Units), GPUs (Graphics Processing Units), VPUs (Video Processing Units), etc., as well as any type of computer, including laptops, smartphones, tablets, (security) controllers, and local networks, edge devices, or the cloud. Additionally, there is a wide range of choices in terms of communication connections, such as Ethernet, I / O-Link, Bluetooth, WLAN, Wi-Fi, 3G / 4G / 5G, and standards that are in principle suitable for industry.

[0043] The illustration of position 10 is simplified and purely exemplary. The operating area 12 can be much broader and more complex and include multiple machines 14, such as processing machines, robots, AGVs, or other transportation systems, etc. Therefore, the required first sensors 16 and second sensors 22 are not just one but multiple. This extends to "factory-level" security systems, a cybersecurity system for an entire factory building or logistics building, where sensor information is collected in real time, evaluated with security techniques, and the sensor information is used for the control, optimization, and risk reduction of the operating area 12. The described basic principle, namely the annotation of the security assessment based on the classical security system and the corresponding distributed training of the local AI model 18, is also retained in this increased local complexity.

[0044] Accordingly, various first sensors 16 and second sensors 22 can be used. As an example, the first sensor 16 is a camera or a 3D camera so that the local AI model 18 can evaluate the image data. In 3D detection, it is advantageous to separate the detected objects from the background that does not change or changes very little by means of distance values. This is beneficial for further processing and generalization to other locations 10. For example, a classical security system can be a radio positioning system as in EP 3 859 382 A1. Thus, in particular, the classical security system can be used for an important security function of personnel recognition, which the classical security system provides based on transponders, and then the local AI model 18 learns to apply this function to the image data so that the transponder can preferably be dispensed with after training. Other classical security systems can be used, especially those that are also capable of personnel recognition. An example is an access control released by a unique identification mark. Here, it should be emphasized that although the first sensor 16 and the second sensor 22 solve the same security function problem, they do not necessarily have the same detection area. For example, the first sensor 16 can observe the machine 14, while the second sensor 22 can observe the only passage leading to the machine 14, because in this way (although indirectly) it can also be reliably determined whether there is a person near the machine 14.

[0045] Figure 2 Fig. shows an overview of distributed training with an orchestration server 28 and multiple computing nodes or locations 10. The orchestration server 28 has a communication interface 30 and a server computing unit (not shown separately), the hardware of which is similar to the above-mentioned computing units 20, 24. The location 10 has the structure described in Figure 1 Here, in Figure 2 the process of distributed training is described at the functional level. Now, first, an overall overview of distributed training is given, and then certain parts are described in further detail with reference to Figures 3 to 6

[0046] Initially, in the manner described in Figure 1 at the location 10, that is, Figure 2 at the lower left and lower right of

[0047] ​After local training is completed, the optimizations obtained thereby are transmitted to the orchestration server 28, which has a communication interface 30 for this purpose. Different from the training data itself, the information transmitted to the orchestration server 28 does not allow any conclusions to be drawn about the operating area 12 or even the specific personnel 15, so such information can be transmitted without any problems. The orchestration server 28 combines the training results from different locations 10 in a global or common AI model 32 (consensus). Then the common AI model 32 can be centrally inspected and certified. Subsequently, the common AI model is transmitted back to the location 10 and can be set there for production use. As an intermediate step before certification, the common AI model 32 can be used to conduct on-site testing of the location 10, and then the common AI model is certified and transmitted back for production operations.

[0048] It is conceivable that the common AI model 32 is also transmitted to locations 10 that did not participate in the distributed training, or vice versa, and only a part of the location 10 is used for training without transmitting the common AI model 32 back to these locations. The distributed training can be iteratively repeated to expand or improve the common AI model 32 or to account for data drift. Federated learning is suitable for security technology applications because it links centralized development and possible certification with decentralized and in any case partially different application areas or locations 10. Thereby, aspects such as a continuous back-and-forth flow of information and the monitoring of data drift or the large-scale execution of on-site testing can be achieved.

[0049] Figure 3 An exemplary flowchart is shown to observe the initial distributed training of the common AI model 32 in more detail. At this stage, the security responsibility is entirely borne by the classical security system. In step S1, the first sensor 16 observes the operating area 12 and generates sensor data. In parallel, in step S2, the classical security system also monitors the operating area 12 through the second sensor 22 and gives a security assessment. In step S3, the sensor data and the relevant security assessment of the classical security system are used as labels or annotations to train the local AI model 18. The training in steps S1 to S3 continues according to a regulation (such as a specific duration or the number of training steps). The local AI model 18 can be initialized with any value or a random value, or the local AI model 18 has already been pre-trained, for example, in the case of personnel recognition to be learned, pre-trained with images of personnel and other objects from any source. In step S4, the trained local AI model 18 or information that can track the training progress is transmitted to the orchestration server 28. Steps S1 to S4 are executed at multiple locations 10 or multiple distributed computing nodes.

[0050] In step S5, the orchestration server 28 collects the training progress at location 10 and generates the general AI model 32 therefrom. For example, the distributed computing nodes transmit the weights of the neural network or the gradients of the residuals. The weights can be averaged, possibly with different impacts depending on the importance of location 10 or its applications, or depending on the number of training data sets included in the respective local training. A similar method can be used to process the gradients, which are then calculated as the new weights of the general AI model 32, for example, in the backpropagation method in the orchestration server 28. One can consider the contributions of the distributed computing nodes as (mini) batches, and then the training of the general AI model 32 functions like the well-known batch learning, except that instead of artificially dividing a larger training data set into multiple batches, these batches are contributed by different locations 10. Further methods regarding integration in step S5 can be found in the literature on federated learning.

[0051] As described below with reference to Figure 4 In the optional step S6, the orchestration server 28 coordinates the validation of the general AI model 32 through field testing, as elaborated. To ensure a certain level of safety, the general AI model 32 is certified based on field testing or other specifications.

[0052] In step S7, the general AI model 32 is transmitted to the selected locations 10. These locations can be the same locations 10 that contributed the local AI models 18 in steps S1 to S4, but some of these locations 10 can be omitted or additional locations 10 can be added. Thus, the initial training is now complete, and various scenarios of using the obtained general AI model 32 as the new local AI model 18 at location 10 are now considered.

[0053] Figure 4 An exemplary flowchart showing a dual safety assessment using a classical safety system and the local AI model 18 is presented. In step S11, the first sensor 16 observes the operating area 12 and generates sensor data. In step S12, the local AI model 18 generated from the general AI model 32 gives a safety assessment of the sensor data. In parallel, in step S13, the classical safety system also monitors the operating area 12 through the second sensor 22 and gives a safety assessment. In step S14, the two safety assessments are compared with each other.

[0054] Now, this comparison can be used alternatively or cumulatively for three purposes. In step S15, the local AI model 18 is verified through field testing, and subsequently, the general AI model 32 is verified. In this case, the local AI model 18 is preferably not yet in production use. Based on the safety assessment of the classical safety system in step S13, it is known what results the local AI model 18 should achieve. Using the same as Figure 3The mechanism of the classical security system that automatically annotates training data now creates an expected value (“ground truth”) for verification in on-site tests. Preferably, the comparison results or the results of the on-site tests are transmitted to the orchestration server 28 and collected there from multiple locations 10. If successful, the general AI model 32 can be certified at the orchestration server.

[0055] In step S16, it is checked whether the local AI model 18 is still operating correctly and has not lost significant recognition performance, for example due to data drift. In this case, the local AI model 18 may already be in production use. As in the verification, the correctly operating AI model 18 should reproduce the security assessment of the classical security system. This online monitoring can also be carried out in an environment-specific manner for individual production facilities or logistics areas. If data drift is recognized, retraining can be carried out in particular, and appropriate security measures can be taken in response as long as security is no longer guaranteed.

[0056] In step S17, the local AI model 18 and the classical security system are used in parallel for diverse redundant monitoring. In this case, the local AI model 18 is in production, but it does not take sole responsibility for security and is only used in conjunction with the classical security system. Thereby, a higher security level can be achieved. Security measures are usually triggered in the form of a logical “OR”, i.e., if one of the two systems recognizes a danger, precautions are taken for safety. If the two systems are inconsistent, one of the systems can also be decisive, and in any case, preferably the cause of the inconsistency is checked, such as by retraining.

[0057] Figure 5 An exemplary flowchart of security monitoring by the trained general AI model 32 is shown, which is transmitted as a local AI model 18 to one of the locations 10. In step S21, the first sensor 16 observes the operating area 12 and generates sensor data. In step S22, the local AI model 18 generated from the general AI model 32 gives a security assessment of the sensor data. This security assessment determines whether security measures are taken. Thus, the local AI model 18 assumes security responsibility. The possibility of alternative monitoring together with the classical security system has already been mentioned in step S17.

[0058] Figure 6 An exemplary flowchart of the distributed retraining of the general AI model is shown. This flowchart largely corresponds to the process in Figure 3 where the starting point is an AI model that has already been trained in a distributed manner and that performs a security assessment on itself. Retraining can be repeated iteratively and as a process that runs concurrently with the continuous development and optimization of the AI model.

[0059] In step S31, the first sensor 16 observes the operation area 12 and generates sensor data. In step S32, the local AI model 18 generated from the general AI model 32 gives a security assessment of the sensor data. In parallel therewith, in step S33, the classical security system also monitors the operation area 12 via the second sensor 22 and gives a security assessment. Alternatively, only one of steps S32 or S33 is sufficient for the security operation and annotation of the sensor data of the first sensor 16. In step S34, the annotated sensor data obtained in this way is used to train the local AI model 18. Preferably, a copy of the local AI model 18 is trained so that the previous local AI model 18 can be used unchanged for production operations. In principle, it is conceivable to train the AI model 18 for production by itself, but this is not insignificant for the security level to be ensured. In step S35, the training progress is transmitted to the orchestration server 28. Then, the further steps S36 to S38 are analogous to Figure 3 steps S5 to S7 thereof, with the difference that the starting point of the new general AI model 32 is the previous general AI model 32 instead of a newly started or merely pre-trained general AI model 32.

[0060] Instead of retraining the general AI model 32, or as a supplement thereto, local adjustments can also be made only at certain locations 10 for its specific requirements. This is described in the literature under the keyword "Multi-Task-Learning" and can be achieved, for example, by having the front layers of the neural network remain unchanged from the general AI model 32 and personalizing the subsequent layers with the help of local training data. For example, the local AI model 18 can be adapted to special sensor arrangements, special environmental influences, special object properties or special tasks. Since local training data is available for this purpose, the adjusted functions can also be verified specifically for the relevant application cases.

Claims

1. A method for distributed training of a general AI model (32) for safety monitoring of an operating area (12), in particular for distributed training according to the principle of federated learning, wherein at a plurality of locations (10), at least one respective first sensor (16) generates sensor data by monitoring its respective operating area (12), and at a corresponding location (10) among the plurality of locations (10), a respective classical safety system monitors the corresponding operating area (12) using at least one respective second sensor (22) and performs a safety assessment, and uses the sensor data and the safety assessment of the corresponding location (10) to train a corresponding local AI model (18). It is characterized in that the training result of the local AI model (18) is transmitted to an orchestration server (28), and the orchestration server (28) generates or improves the general AI model (32) based on the training result.

2. The method according to claim 1, wherein The orchestration server (28) transmits the general AI model (32) to at least one location (10), where the general AI model is particularly used as a local AI model (18).

3. The method according to claim 2, wherein, The local AI model (18) evaluates the sensor data recorded using the first sensor (16) at at least one location (10) for safety assessment, and in particular activates safety measures when a danger is recognized.

4. The method according to claim 2 or 3, wherein, The classical safety system at least temporarily continues to perform safety assessment at the at least one location (10).

5. The method according to claim 4, wherein The safety assessments of the classical safety system and the local AI model (18) are compared with each other.

6. The method according to claim 5, wherein, When the classical safety system or the local AI model (18) recognizes a danger, safety measures are activated.

7. The method according to claim 5, wherein, The local AI model (18) is verified based on the comparison.

8. The method according to any one of the preceding claims, wherein, The general AI model (32) is optimized in an iterative process by: retraining at least one local AI model (18) using the sensor data and the safety assessment of the corresponding location (10); transmitting the training result of the at least one local AI model (18) to the orchestration server (28); and the orchestration server (28) improves the general AI model (32) based on the training result.

9. The method according to any one of the preceding claims, wherein, The first sensor (16) generates image data.

10. The method according to any one of the preceding claims, wherein, The classical safety system has a radio positioning system.

11. The method according to any one of the preceding claims, wherein, The safety assessment includes safety object classification, in particular personnel identification.

12. A security system for distributed training of a general AI model (32) for security monitoring of an operating area (12), in particular for distributed training according to the principle of federated learning, the security system comprising an orchestration server (28) and a plurality of computing nodes, each computing node being located at one of a plurality of locations (10), wherein the orchestration server (28) has a server computing unit and a first communication interface (30), and the computing nodes have respective AI computing units (20) and second communication interfaces (26) so that an AI model (18, 32) or a training result of the AI model (18, 32) can be transmitted via the communication interfaces, and at the location (10), at least one respective first sensor (16) is provided for generating sensor data by monitoring the respective operating area (16) of the location (10), and a respective classical security system is provided for monitoring the respective operating area (12) using at least one respective second sensor (22) and for performing a security assessment, wherein, The respective AI computing units (20) are designed to train their respective local AI models (18) using the sensor data and the safety assessment of the corresponding location (10), and transmit the training result of the local AI model (18) to the orchestration server (28), and wherein the server computing unit is designed to generate or improve the general AI model (18) based on the training result.

Images