Assistance system

Abstract

Assistance system (1) for configuring and / or operating an environment, in particular a factory environment, wherein the assistance system (1) comprises an environment model (2) of the environment, which includes configuration data records of structures (3) and / or facilities (4) of the environment that are configured or configurable by various configuration modules (5-7), wherein the assistance system (1) is configured to generate, based on the at least one configuration data record, a system output for a configuration query relating to the at least one structure (3) and / or facility (4) by means of the at least one configuration module (5-7), which includes an image output, in particular three-dimensional, of a part of the environment model (2) affected by the at least one configuration data record and a configuration message containing the at least one configuration data record of the structure (3) and / or facility (4).which is affected by the configuration query.

Description

The invention relates to an assistance system for the configuration and / or operation of an environment, in particular a factory environment. For configuring an environment and / or controlling the operation of an environment, i.e., in particular functional facilities operated within the environment, such as building equipment, logistics facilities, machinery, especially robots, and the like, individual programs or so-called "expert tools" are typically used in the prior art. In principle, there is no networking between the individual programs or "expert tools," so changes in one structure, area, or type of facility within the environment may have an impact on another structure, area, or type of facility within the environment, which, however, is not immediately apparent due to the lack of networking. This means, in particular, that individual users of the relevant programs or expert tools rely on seamless interdisciplinary communication. If a user makes changes in one of the programs, for example, concerning a building system, this change must be communicated to at least one other user who, for instance, is responsible for planning machinery or logistics facilities, or for controlling their operation. If, for example, a building system alters a utility line in the vicinity, this could result in the corresponding utility line no longer being available to a piece of machinery located nearby.Accordingly, changes made in one work area that affects the environment can result in related changes for at least one other work area, impacting the operation or configuration of that environment. If such changes are not communicated across the individual work areas, conflicts can arise, such as configuration or planning conflicts, operational conflicts, and the like. However, even within individual work areas, such as the configuration of building equipment in the surrounding area, checking a configuration against numerous different requirements and regulations can be time-consuming. Typically, such requirements and regulations, for example, safety zones, vehicle routes, protective devices, and the like, must be manually checked and addressed by a user. If subsequent changes occur, which may also be in a different work area, previously performed checks of the requirements may no longer be valid and must therefore be repeated. The invention is based on the objective of providing an assistance system for the configuration and / or operation of an environment, in particular a factory environment. The problem is solved by an assistance system according to claim 1. The dependent claims relate to possible embodiments. As described, the invention relates to an assistance system for configuring and / or operating an environment, in particular a factory environment. The factory environment can, for example, comprise the environment of a production site, such as a factory where components are manufactured. Specifically, motor vehicles or motor vehicle assemblies can be manufactured in the factory environment. The assistance system includes an environment model of the environment. The environment model can, for example, be designed as a digital twin of the environment. For example, structures and / or equipment that are actually present or intended to be present in the environment can be simulated or modeled in the environment model. In addition to purely three-dimensional modeling of such structures and / or equipment, it is also possible to simulate or model their function or operation.The assistance system can be applied to non-real-time environments in addition to real-time environments and digital twins. The environment model comprises configuration data records of structures and / or facilities within the environment, which are configured or configurable by various configuration modules. Such configuration modules can be understood as the programs, algorithms, or "expert tools" described previously. At least one such configuration module can therefore modify a configuration data record of a structure and / or facility within the environment. As previously explained, the configuration data record describes the respective structure and / or facility, in particular its design and / or operation. In other words, such a configuration module can be used to modify the planning, configuration, or operation of a structure and / or facility within the environment. The assistance system is designed to generate a system output in response to a configuration query concerning at least one structure and / or setup, using at least one configuration module and based on at least one configuration data record. In other words, a configuration query can be submitted to or transmitted to the assistance system. For example, the assistance system has a user interface through which a user can submit the configuration query to the assistance system. Based on the configuration query, the assistance system can then select at least one configuration module, and the assistance system can use the configuration module to generate a system output based on the at least one configuration data record. In one embodiment, a user can submit a configuration query to the assistance system for a specific structure and / or setup in the environment.The assistance system can receive such a configuration query. Depending on the affected structure and / or facility, for example, depending on the affected work area, the assistance system selects a suitable configuration module. Using the configuration module, the assistance system can access the configuration data record of the affected structure and / or facility and thus generate the system output. The system output comprises an image, in particular a three-dimensional one, of a part of the environment model affected by the at least one configuration data record, and a configuration message relating to the at least one configuration data record of the structure and / or facility affected by the configuration query. The user thus receives the described system output as a response to the configuration query. In the described implementation, the system output comprises an image and a configuration message. The image output can be displayed on an output device of the assistance system, for example, on a screen. The image output can, for example, display a three-dimensional output, in particular a three-dimensional part of the environment model. The image output specifically depicts the at least one structure and / or facility in the environment affected by the configuration query.In addition, the assistance system can output the configuration message, particularly in text form and / or as audio output. Specifically, the configuration query can be submitted by the user to the assistance system, for example, as text input, voice input, or user input within the 3D image output, such as by selecting areas and / or objects at a user interface. The assistance system can analyze the configuration query and, for example, identify at least one affected structure and / or facility. From this, the assistance system determines at least one configuration module that is required or suitable for processing the configuration query. This at least one specific configuration module is then applied to the environment model, in particular to the at least one configuration record of the environment's structure and / or facility. This generates system output that includes an image for the user and a configuration message for the user. In one embodiment, the user can use the contraction query to determine where specific safety zones are defined in the environment. The assistance system can call the at least one configuration module designed for configuring these safety zones. Based on the configuration data set for the affected structures and / or facilities in the environment, the assistance system uses the previously defined configuration module to determine the safety zones within the environment model. The identified safety zones can then be displayed as an image, for example, a three-dimensional environment model, along with a configuration message. The user therefore receives the answer to the configuration query directly from the assistance system, namely via the system output. Advantageously, the user does not need to determine the appropriate configuration module themselves and search through all configuration records of the environment model, as the assistance system, particularly through the selected configuration module, can identify only the relevant structures and / or facilities. Especially for interdisciplinary queries or queries where the user is not an expert, the assistance system offers the possibility of integrating all configuration modules, structures, and facilities into a higher-level environment model. The assistance system can thus be understood, in particular, as a so-called "human-machine interface." The assistance system can process configuration queries based on artificial intelligence, specifically generating and displaying system output using artificial intelligence. The assistance system can be trained to answer specific questions related to the environment. Specifically, configuration modules can be called via "function calling" and used by the assistance system to generate system output. In this context, the assistance system can also be trained to the specialized vocabulary of the work environment and / or the configuration modules. Specifically, the assistance system can be trained to generate the system output, particularly the configuration message and image output, based on artificial intelligence, specifically generative artificial intelligence and / or a large language model, for the configuration query. In one embodiment, the user can thus direct the configuration query to the assistance system. The assistance system can, for example, be implemented as a chatbot at the user interface. The configuration query is processed by the assistance system, and system output is generated as described above. This system output can be generated by artificial intelligence, specifically generative artificial intelligence and / or a large language model.As described, the assistance system, in particular the artificial intelligence algorithm used, can be trained on the language of the configuration query and / or the language for operating the at least one configuration module and / or the configuration data set. The assistance system can therefore support the user in their work. By consolidating the various configuration modules, as well as the facilities and / or structures, within the environment model, resources can be saved, since a separate model does not need to be developed, maintained, and operated for each configuration module. A further advantage is that changes a user makes to a structure and / or facility within the environment model via a configuration module have an impact on the same environment model that is directly visible to other configuration modules. For example, such impacts can also be determined for other configuration modules, thus eliminating the need for seamless interdisciplinary communication and preventing configuration errors. For example, if a configuration module intended for a building's infrastructure makes a change to that infrastructure—such as the routing of a utility line—another configuration module, intended for the operation of a machine, can determine whether the previously introduced change to the building's infrastructure will affect the machine's operation. Planning errors or conflicts in the planning, configuration, and operation of structures and / or facilities within the environment model can therefore be identified early or prevented entirely. In particular, it can be directly indicated that a planned change to a structure and / or facility would have a specific impact on at least one other structure and / or facility.For example, if the planned change to a supply line is included in the environment model, a potential resulting undersupply of at least one machine can be directly output, so that the change can be reviewed again and possibly implemented differently, or the machine can adapt to the changed supply line. In another embodiment of the assistance system, the environmental model can be designed as a digital twin of the environment, particularly a real-time digital twin. The environmental model can, for example, be a comprehensive 3D factory model of the environment. Here, all facilities and / or structures can be fully modeled or simulated in 3D within the environmental model. The digital twin of the environment thus forms a digital representation, particularly in real time, of the real environment. Based on this, it is possible to simulate the planning, configuration, and operation of the structures and / or facilities in the environment in real time. In addition to real-time environments and digital twins, the assistance system can also be applied to non-real-time environments. As described, the configuration modules can, in principle, be configured for any configuration of structures and / or facilities within the environment model. The configuration modules can thus be assigned to and suitable for any discipline or work area. In one embodiment of the assistance system, at least one configuration module can comprise at least one of the following modules: a building module, a plant module (in particular a robot module), a logistics module, and / or a safety module. By way of example, the building module allows for the configuration of at least one building component or structure within the environment model. For instance, structures and / or facilities within this area can be modified by the building module, or information about such structures and / or facilities can be determined and output by the building module. Such structures and / or facilities can include, for example, floors, walls, ceilings, columns, utility lines, and the like. Alternatively or additionally, at least one configuration module can include a plant module, in particular a robot module. Within the plant module, configuration data sets of plants, especially robot systems, machine tools, and the like, can be accessed and modified. The plant module can also influence the operation of such plants, for example, by setting their operating parameters. Furthermore, the configuration module can include a logistics module. This allows any logistics parameters within the environment model to be modified. For example, transport areas, routes, safety zones, and the like can be defined and / or changed by the logistics module. The safety module, on the other hand, concerns safety-critical parameters within the environment model. This includes, for example, safety zones, positions, and functions of safety devices such as sprinkler systems, fire extinguishers, and the like. It turns out that by accessing the individual configuration modules, the assistance system can also identify problems that span multiple work areas. This makes it possible to resolve or prevent planning conflicts. For example, by accessing the building module, plant module, logistics module, and security module, it can check whether media connections for a specific system are configured where needed, whether access routes for logistics facilities exist, and whether sufficient safety areas are available. Thus, the assistance system can act as an interface between different planners and / or operators of different subsystems. For example, users can conveniently submit configuration queries to the assistance system, which will then answer them.For example, it can be shown which building structures, such as hall columns, lack impact protection for logistics equipment, such as forklifts. It can also be checked whether safety equipment, such as fire extinguishers or sprinkler systems, are correctly positioned, adequately connected and supplied, and not obstructed by other structures. The assistance system can be further refined or trained based on prompt engineering. This allows for an abstracted or "high-level" description. For example, an expert can define or specify rules for the assistance system. This could involve assigning different configuration modules to various structures, facilities, or work areas. Furthermore, the assistance system can be continuously improved and expanded, for example, through further training. With further refinement, the assistance system can be configured to determine conflict information from the environment model that describes at least one conflict involving at least one structure and / or facility. As previously described, such conflict information can be determined based on a configuration query and output as a system query. For example, a user can specifically query whether at least one structure and / or facility is in conflict with another structure and / or facility, or whether the supply to at least one system or system type is ensured or not. However, the assistance system can also determine such conflict information independently or automatically. For example, if such conflicts exist between structures and / or facilities in the environment model, the assistance system can automatically output this conflict information as a system query.Advantageously, the user is therefore notified, either asked or unsolicited, about such conflict information that is present in the environment model. As described earlier, the system output, which includes an image, is generated in response to the configuration query. In one embodiment of the assistance system, the image output, particularly based on user input, can include a modifiable virtual camera within the environment model. In other words, the environment model can be implemented as a three-dimensional model of the surroundings, for example, a 3D factory model. Based on a configuration query, the image output can, for example, be a representation of the structure and / or equipment in question. The image output can be generated based on an image from a virtual camera within the environment model. The user can modify the virtual camera, particularly through user input. This allows the user to virtually navigate through the environment model and / or select the viewing direction within it.Advantageously, this also allows structures and / or facilities located in the vicinity of an affected structure and / or facility to be included. Furthermore, the assistance system can be configured, particularly based on the configuration query, to generate a system output that is set up to configure at least one structure and / or facility in the environment. The configuration query can, for example, also include commands for the assistance system to configure at least one configuration record of one of the structures and / or facilities in the environment. For example, operating parameters of structures and / or facilities can be set in this way, for instance, by modifying the associated configuration record. For example, a robot's behavior can be modified, particularly by changing its movement sequence. This allows the assistance system to make corresponding changes to the structure and / or setup of the environment, especially when there is a change in the environment, such as a change to a component being manufactured. As another example, at least one safety area can be modified or defined by the assistance system. The assistance system can automate such configurations, for example, to resolve planning conflicts. These configuration changes can be implemented automatically by the assistance system, suggested to a user, or executed only through targeted user input. In a further embodiment, the assistance system can be configured to index interconnected structures and / or facilities, particularly spatially adjacent structures and / or facilities, based on an octree. An "octree" is understood to be a tree structure that divides the space of the environment model into individual segments, which can in turn be subdivided into individual subsegments. By indexing the structures and / or facilities based on such a tree or tree structure, it is simplified for the assistance system to locate neighboring structures and / or facilities. Consequently, changes to one structure and / or facility can have effects that are also relevant to neighboring structures and / or facilities. Such connections can be easily checked, determined, and, if necessary, resolved based on the octree structure.In addition to the assistance system, the invention relates to a method for operating an assistance system, in particular an assistance system described above, for configuring and / or operating an environment, in particular a factory environment, wherein the assistance system comprises an environment model of the environment, which includes configuration data sets of structures and / or devices of the environment that are configured or configurable by various configuration modules, wherein, by means of the assistance system, a system output is generated for a configuration query relating to the at least one structure and / or device by means of the at least one configuration module based on the at least one configuration data set, which includes an image output, in particular three-dimensional, of a part of the environment model affected by the at least one configuration data set and a configuration message.which contains at least one configuration record of the structure and / or facility affected by the configuration query. The procedure can be carried out in particular with the assistance system described above, or the assistance system described above is designed to carry out the procedure in all its details. All advantages, details, specifications and / or features described in relation to the assistance system are fully transferable to the procedure and vice versa. The invention is explained with reference to exemplary embodiments and the figures. The figures are schematic representations and show: Fig. 1 a schematic representation of an assistance system according to an exemplary embodiment; Fig. 2 a schematic representation of a user interface of an assistance system according to an exemplary embodiment; and Fig. 3 a schematic representation of a flowchart of a method for operating an assistance system according to an exemplary embodiment. Figure 1 shows a purely schematic representation of an assistance system 1. The assistance system 1 is designed for the configuration and / or operation of an environment, specifically a factory environment. The assistance system 1 includes an environment model 2 of the environment. The environment model 2 contains, for example, structures 3 and / or equipment 4 of the environment. These can be stored or represented in the environment model 2 based on their configuration data sets. Specifically, the environment model 2 can be a model or simulation of the environment, particularly a real-time model or simulation, and in particular a so-called digital twin. Furthermore, the assistance system 1 has several configuration modules 5-7, which are designed to determine or modify configuration data sets for structures 3 and facilities 4. Configuration modules 5-7 differ from one another. They can be understood, for example, as programs or algorithms, specifically as so-called "expert tools." For instance, the different configuration modules 5-7 are assigned to different departments or work areas, or are designed within these departments to determine and modify configuration data sets. By way of example, the first configuration module 5 could comprise a building module. Configuration module 6 could, for example, comprise a plant module. Configuration module 7 could, for example, comprise a security module. The described modules and their allocation across configuration modules 5-7 are entirely arbitrary.Configuration modules 5-7 can be reassigned, replaced by other modules, or supplemented with additional modules. The assistance system 1 further comprises a user interface 8, which includes an input device 9 and an output device 10, in particular an image output device 11 and a message output device 12. The image output device 11 and the message output device 12 can be combined in the output device 10, as shown, for example, in Fig. 2. The image output device 11 and the message output device 12 can, for example, be displayed on a common screen. In particular, the input device 9, the output device 10, in particular the image output device 11, and the message output device 12 can be integrated into a common user interface 8. This interface can, for example, include a screen and at least one input device. Via the user interface 8, in particular the input device 9, a user can enter at least one configuration query and thus send it to the assistance system 1. This query is transmitted, for example, as indicated by an arrow, to a processing unit 13. The processing unit 13 is specifically designed to analyze the communication query. The processing unit 13 can, for example, execute at least one artificial intelligence algorithm. For instance, the processing unit 13 can comprise a chatbot, a large language model, or at least one generative artificial intelligence algorithm. The processing unit 13 can be trained on a vocabulary specific to the environment and / or at least one configuration data set and / or at least one configuration module 5-7. This training can be performed, in particular, by means of prompt engineering. Based on the analysis of the configuration query by the processing unit 13, the assistance system 1 can determine at least one structure 3 and / or facility 4 in the environment model 2 that is affected by the configuration query. Furthermore, the assistance system 1 can determine at least one of the configuration modules 5-7 that is suitable for processing the configuration query. If, for example, the configuration query concerns a building component in the environment model 2, such as a supply line, the assistance system 1 can select configuration module 5. Here, structure 3 can be configured as a supply line, allowing configuration module 5 to access the environment model 2, in particular the configuration data record of structure 3. Accordingly, the assistance system 1 can generate a system output for the configuration query and display it via the output device 10. The system output includes, in particular, a configuration message, especially speech-based and / or text-based. As illustrated by example in Fig. 2, this can be displayed on a screen and / or via loudspeakers, especially in the form of a chat log. Furthermore, the system output includes an image output of the selected structure 3 or configuration 4. If, for example, the configuration query concerns a system, in particular a robot 14, which is modeled as equipment 4 in the environment model 2, the assistance system 1 determines the configuration module 6, which is intended for systems in the environment model 2, by means of processing direction 13. Accordingly, a system output can be generated and displayed at output device 10. This can, in turn, be displayed as a text message and / or audio message at message output device 12. For example, it is shown that an image, in particular a three-dimensional image, of the environment or the environment model 2 is displayed at image output device 11, in which equipment 4 and structures 3, in particular the robot 14, can be depicted. The user thus receives an answer to the configuration query as both a message and an image.In other words, the user directly sees a display of the structure 3 and / or setup 4 affected by the configuration query. The user can optionally operate the image output device 11 via the user interface 8. The image output can, for example, be designed as a modifiable virtual camera in the environment model 2. In other words, the user can change the position and / or orientation of the virtual camera and thus move through the environment model 2. It is also possible to view the structures 3 and facilities 4 from different angles and at different distances in the environment model 2. System 1 can also be configured to determine conflict information from environment model 2. For example, if spatial and / or functional conflicts exist in environment model 2, assistance system 1 can determine these automatically or based on a configuration query. For instance, if a change to a supply line results in an insufficient supply to robot 14, or more generally to facility 4 or structure 3, assistance system 1 can determine and output this information. Furthermore, the assistance system 1 can be configured for structure 3 and / or facility 4, either initiated by a configuration query or automatically. For this purpose, the assistance system 1 can use the appropriate configuration module 5-7 to modify the configuration data record of structure 3 and / or facility 4. This usually results in changes to the behavior or operation of structure 3 and / or facility 4. For example, structure 3 and / or facility 4 can be replanned or their operating state can be changed. The procedure described herein, which can be executed in detail on the assistance system 1, is explained below in Fig. 3 with reference to blocks 15-19. The procedure can, for example, start in block 15, in which the assistance system 1 receives the configuration query via the user interface 8, in particular via the input device 9. In block 16, the configuration query is analyzed, for example, to determine which structure 3 and / or device 4 is affected, or which configuration module 5-7 is suitable for processing the configuration query. Subsequently, in block 17, the corresponding specific configuration module 5-7 is loaded, called, or executed. Combinations of different configuration modules 5-7 can also be called to process a configuration query. The at least one specific configuration module 5-7 can then determine and, if necessary, modify at least one configuration record of the affected structure 3 and / or facility 4 in block 18. Depending on the configuration query, a change to a configuration record can, for example, be made directly. Subsequently, the system output can be displayed in block 19, as described previously. This can, for example, be a simple information output to output device 10. Alternatively or additionally, at least one control command or a change to a configuration record can be displayed. This can be transmitted to at least one structure 3 and / or device 4 so that structure 3 and / or device 4 can be configured. As described, the assistance system 1 is designed to carry out the described procedure in all its details. Likewise, the procedure can be executed on the assistance system 1 in all its details. The foregoing description is therefore fully transferable from the assistance system 1 to the procedure and vice versa. The advantages, details and features described in relation to the individual embodiments can be combined, interchanged and transferred to one another as desired. REFERENCE MARK LIST 1 Assistance system 2 Environment model 3 Structure 4 Setup 5-7 Configuration module 8 User interface 9 Input device 10 Output device 11 Image output device 12 Message output device 13 Processing device 14 Robot 15-19 Block

Claims

Assistance system (1) for configuring and / or operating an environment, in particular a factory environment, wherein the assistance system (1) comprises an environment model (2) of the environment, which includes configuration data records of structures (3) and / or facilities (4) of the environment that are configured or configurable by various configuration modules (5-7), wherein the assistance system (1) is configured to generate, based on the at least one configuration data record, a system output for a configuration query relating to the at least one structure (3) and / or facility (4) by means of the at least one configuration module (5-7), which includes an image output, in particular three-dimensional, of a part of the environment model (2) affected by the at least one configuration data record and a configuration message containing the at least one configuration data record of the structure (3) and / or facility (4).which is affected by the configuration query. Assistance system (1) according to claim 1, characterized in that the assistance system (1) is configured to generate the system output, in particular the configuration message and the image output based on artificial intelligence, in particular generative artificial intelligence and / or a large language model, for the configuration query. Assistance system (1) according to claim 1 or 2, characterized in that the environment model (2) is designed as a, in particular real-time based, digital twin of the environment. Assistance system (1) according to one of the preceding claims, characterized in that at least one configuration module (5-7) comprises at least one of the following modules: - a building module and / or - a plant module, in particular a robot module, and / or - a logistics module and / or - a safety module. Assistance system (1) according to one of the preceding claims, characterized in that the assistance system (1) is learnable or trained based on Prompt Engineering. Assistance system (1) according to one of the preceding claims, characterized in that the assistance system (1) is configured to determine conflict information of the environment model (2) describing at least one conflict of at least one structure (3) and / or facility (4). Assistance system (1) according to one of the preceding claims, characterized in that the image output, in particular based on a user input, comprises a modifiable virtual camera in the environment model (2). Assistance system (1) according to one of the preceding claims, characterized in that the assistance system (1) is configured, in particular based on the configuration query, to generate a system output which is set up to configure the at least one structure (3) and / or device (4) in the environment. Assistance system (1) according to one of the preceding claims, characterized in that the assistance system (1) is configured to index interconnected structures (3) and / or facilities (4), in particular spatially adjacent structures (3) and / or facilities (4), based on an Octree. Method for operating an assistance system (1), in particular an assistance system (1) according to one of the preceding claims, for configuring and / or operating an environment, in particular a factory environment, wherein the assistance system (1) comprises an environment model (2) of the environment, which includes configuration data sets of structures (3) and / or devices (4) of the environment that are configured or configurable by various configuration modules (5-7), wherein, by means of the assistance system (1), a system output is generated for a configuration query relating to the at least one structure (3) and / or device (4) by means of the at least one configuration module (5-7) based on the at least one configuration data set, which includes an image output, in particular three-dimensional, of a part of the environment model (2) affected by the at least one configuration data set and a configuration message.which contains at least one configuration record of the structure (3) and / or facility (4) affected by the configuration query.

Images