Method for creating or updating a digital twin for an automation field device

Abstract

The invention comprises a method for creating or updating a digital twin (TW) for an automation field device (FG), the method comprising: - optically capturing an image (AN) of the field device (FG) using an operating unit (BE); - transmitting the image (AN) to a cloud-based platform (CL); - ascertaining a type (GT) of the field device (FG) and / or a description (BS) of a possible application of the field device (FG) on the basis of the image (AN), wherein, for the ascertaining step, a pretrained AI or machine learning algorithm (KI) analyzes the image (AN) using existing images of known types of field devices (FG) and / or design data; and - creating a digital twin (TW) of the field device (FG), wherein the type (GT) of the field device (FG) and / or the description (BS) of the possible application are added to the digital twin (TW) when it is created, or augmenting an existing digital twin (TW) of the field device (FG) to include the type (GT) of the field device (FG) and / or the description (BS) of the possible application of the field device (FG).

Description

[0001] Method for creating or updating a digital twin for an automation field device

[0002] The invention relates to a method for creating or updating a digital twin for a field device in automation technology.

[0003] Field devices used in industrial plants are already known from the state of the art. Field devices are widely used in process automation technology as well as in manufacturing automation technology. Field devices essentially refer to all devices used close to the process and that provide or process-relevant information. Field devices are used to record and / or influence process variables. Measuring devices or sensors are used to record process variables. These are used, for example, for pressure and temperature measurement, conductivity measurement, flow measurement, pH measurement, level measurement, etc. and record the corresponding process variables such as pressure, temperature, conductivity, pH value, level, flow, etc. Actuators are used to influence process variables.These include, for example, pumps or valves that can influence the flow of a fluid in a pipe or the fill level in a container. In addition to the previously mentioned measuring devices and actuators, field devices also include remote I / Os, wireless adapters, and generally devices located at the field level.

[0004] A large number of such field devices are produced and distributed by the Endress+Hauser Group.

[0005] In modern industrial plants, field devices are usually connected to higher-level units via communication networks such as fieldbuses (Profibus®, Foundation® Fieldbus, HART®, etc.). These higher-level units are usually control systems (DCS) or control units, such as a PLC (programmable logic controller). The higher-level units are used, among other things, for process control, process visualization, process monitoring and for commissioning the field devices. The measured values ​​recorded by the field devices, particularly sensors, are transmitted via the respective bus system to one (or possibly several) higher-level units. In addition, data transmission from the higher-level unit to the field devices via the bus system is also required, particularly for the configuration and parameterization of field devices and for controlling actuators.

[0006] In the context of Industry 4.0, or IoT ("Industrial Internet of Things"), the data generated by field devices is often collected directly from the field using so-called data conversion units, known as "edge devices" or "cloud gateways," and automatically transmitted to a central cloud-based platform. The terms cloud-based platform, cloud, and server are used synonymously for the purposes of this application.

[0007] One or more such cloud applications are located on the cloud-based platform. A cloud application is a program that runs on such a cloud-based platform or is integrated into it. A user can connect to the cloud-based platform via the Internet and make modifications to the corresponding cloud applications of the cloud-based platform and / or operate them, i.e., write data to the cloud applications, read data from the cloud applications, and / or edit this data.

[0008] A digital twin, also called a digital image, is a virtual representation of the field device that includes the identical configuration, parameter values, current device status, algorithms, etc. The digital twin thus exhibits all of the field device's properties that fully describe the field device for its intended purpose. The field device and the digital twin are intended to always be identical. A change in the properties of the field device leads to synchronization (via Industry 4.0 or HoT technologies) with the digital twin, so that the properties of the digital twin are updated accordingly. The digital twin is integrated, for example, in a cloud application. A digital twin is usually created after the order of a field device or during production.However, it often happens that properties of the field device change between the creation of the digital twin and the commissioning of the field device, so that the digital twin does not completely match the properties of the field device after the commissioning of the field device.

[0009] There are so-called "scanner apps" that run on mobile devices and allow the mobile devices to detect field devices in the field. It is possible to capture photos of a field device and enter metadata, such as a description of the application and / or the type of field device. The app can access the digital twin and supplement it with the photo and metadata.

[0010] This information must be entered separately and manually for each individual field device, even for identical or similar objects in comparable applications. To obtain a complete impression of the system and its surroundings, multiple photos must be taken.

[0011] A digital twin can also be created manually afterward and, for example, enriched with information captured by the scanner app. In addition to the disadvantage of this requiring a significant amount of manual effort, there is also no error control or plausibility check.

[0012] Based on this problem, the invention is based on the object of presenting a method which allows an incomplete digital twin of a field device to be completed in a simple manner or to create a complete digital twin for a field device.

[0013] The task is solved by a method for creating or updating a digital twin for an automation field device, comprising:

[0014] - Optically capturing a recording of the field device using a control unit; - Transmitting the recording to a cloud-based platform;

[0015] - Determining a type of field device and / or a description of a possible application of the field device based on the image, wherein a pre-trained AI or machine learning algorithm analyses the image with existing images of known types of field devices and / or design data for the step of determining; and

[0016] - Creating a digital twin of the field device, whereby the type of the field device and / or the description of the possible application are added to the digital twin during creation, or enriching an already existing digital twin of the field device with the type of the field device and / or the description of the possible application of the field device.

[0017] The method according to the invention enables the automated loading of data relating to field devices into a digital twin of the field device. If no digital twin is available for the field device, a new digital twin can be created on the cloud-based platform. For both variants, the user simply needs to visually capture an image of the field device using the control unit. The cloud-based platform then independently determines the device type of the field device and / or a description of a possible application of the field device with the aid of the AI ​​or machine learning algorithm. For this purpose, the AI ​​or machine learning algorithm was previously trained with training data. This training data includes images of field devices and / or design data as input data and defined device types or descriptions of applications as output data.An application description contains information on the task of the field device (e.g. measuring the temperature in a container, etc.), information on special configurations of the field device (e.g. a hygienic housing for applications in the food industry, etc.) and application-specific warnings and environmental information.

[0018] The digital twin is enriched with the device types and descriptions obtained in this way. To do this, the user selects the relevant digital twin or has it determined using the cloud-based platform. If no digital twin exists for this field device, the cloud-based platform creates a corresponding digital twin. For this, the user must provide identification information for the field device (i.e., a tag or serial number).

[0019] Field devices mentioned in connection with the method according to the invention have already been listed as examples in the introductory part of the description.

[0020] According to an advantageous embodiment of the method, if no clear type of field device and / or no clear description of a possible application of the field device can be determined, several possible types of field device or several possible descriptions are suggested to a user. For this purpose, the AI ​​or machine learning algorithm can assign a degree of possible agreement to the possible types of field device or the several possible descriptions. The possible types of field device or the several possible descriptions are presented to the user on the control unit, with the user then selecting the device type or the description via the control unit.

[0021] One embodiment of the method provides that, to create the description of a possible application, at least one additional piece of information is determined or provided, which is then analyzed by the AI ​​or machine learning algorithm. This could be, for example, a recording of the field device's surroundings, location information of the field device, and / or information regarding the orientation of the control unit when the recording was taken. When using this additional information, the AI ​​or machine learning algorithm must be trained with additional training data containing the aforementioned additional information.

[0022] According to an advantageous embodiment of the method, it is provided that if even using the at least one additional piece of information, no clear type of field device and / or no clear description of a possible application of the field device can be determined, additional information regarding the field device and / or the application is requested from the user, wherein in particular a questionnaire is used. The queries are generated by the operating unit or the cloud-based query, wherein the queries are presented to the user via the operating unit, and wherein the user enters the additional information via the operating unit. The additional information is requested, in particular, in a context-related manner.For example, the user might be asked which process medium flows through a pipeline in order to independently determine the correct value of the field device's density parameter for a flow measurement. If the user is unfamiliar with the measuring medium, they can also formulate its properties ("colorless," "odorless," etc.), allowing the cloud-based platform to independently deduce the missing information from these properties.

[0023] According to one embodiment of the method, it can further be provided that the recording includes one or more two-dimensional images of the field device and / or a video of the field device. For this purpose, the images preferably have different viewing angles or views of the field device. This allows, for example, field device types that have a similar top surface but significant differences from other viewing angles to be precisely differentiated from one another. The video should advantageously include a pan or change of perspective around the field device.

[0024] In a further embodiment, the design data is advantageously CAD data. This means that it does not have to be "real" data, but rather data that was generated during the design of the measuring point in which the field device is to be installed, or during the creation of the field device itself. This allows details that are not included in the training data photos to be taken into account during the training process of the AI ​​or machine learning algorithm, so that subsequent recognition is more precise. In a further embodiment of the method, after the type of field device or the description has been determined, a three-dimensional model of the field device is displayed on the control unit, which shows the same viewing angle of the field device as the image.This allows the user to correctly identify the device – they can confirm that it is the correct field device by comparing the field device in front of them with the displayed three-dimensional model. The three-dimensional model is also loaded into the digital twin or linked to it, so that the user can access it via the cloud-based platform. In addition to the field device, the three-dimensional model can also include other components of the measuring point (e.g., vessels, pipelines, network devices, network cabling, other field devices, etc.) that are located in close proximity to the field device and / or connected to it.

[0025] The three-dimensional model can be created using at least two images of field devices of the same type. Alternatively, the three-dimensional model can also be created from the design data.

[0026] According to an advantageous embodiment of the method, the user confirms or rejects the type of field device and / or the description for the possible application, with the confirmation or rejection being fed to the AI ​​or machine learning algorithm. This feedback further trains the AI ​​or machine learning algorithm. Through regular feedback, the AI ​​or machine learning algorithm becomes increasingly precise and delivers more accurate results. For example, this also trains the behavior at different times of day, when the brightness and color saturation of the images vary.

[0027] It can be provided that the AI ​​or machine learning algorithm is used "globally," i.e., that multiple users across systems access the AI ​​or machine learning algorithm, provide feedback, and thereby further train it. A further embodiment provides for a mobile device, in particular a smartphone or tablet, to be used as the control unit. A control unit such as the "Field Xperts" marketed by the applicant, or a wearable such as data glasses (e.g., Google Glasses) or a smartwatch, can also be used with the method according to the invention if it has a camera.

[0028] The invention is explained in more detail with reference to the following figure. It shows

[0029] Fig. 1 : an exemplary embodiment of the method according to the invention.

[0030] Fig. 1 shows a field device (FG) installed in a measuring point of a process plant. In this case, the field device (FG) is a level gauge designed to measure the level in a container.

[0031] The field device FG is in direct or indirect contact with one or more environmental components UK. In this case, the environmental component UK shown is the tank to which the field device FG is attached.

[0032] A so-called digital twin (TW) is to be created for the field device (FG) on a cloud-based platform (CL). A digital twin (TW) is a digital image of the field device (FG) and should have the same properties, in particular the same configuration and parameterization, as the field device (FG) itself.

[0033] For this purpose, an operating unit BE, in this case a mobile device in the form of a smartphone, is connected to the cloud-based platform CL via the Internet. The operating unit BE must authenticate itself to the cloud-based platform CL, for example using login data (such as a user name and password) and / or biometric authentication features (e.g. a fingerprint). After successfully connecting to the cloud-based platform CL, the user selects that a new digital twin TW should be created for an as yet unknown field device F. After entering identification information for the field device FG, such as a tag or serial number, the operating unit BE uses its camera to capture a photo AN of the field device FG. The photo consists of at least one, but preferably two or more photos of the field device FG, which should be taken from different angles.Alternatively, a video can be captured as recording AN, which should include a pan around the field device FG.

[0034] The recording AN is then transmitted to the cloud-based platform CL. A Kl or machine learning algorithm Kl implemented on the cloud-based platform CL subsequently processes the recording AN. The processing is carried out in such a way that the Kl or machine learning algorithm Kl compares the recording with existing images of known types of field devices FG and / or design data. This can be done by training the Kl or machine learning algorithm Kl in advance with training data containing these known types and design data (e.g., CAD data). In particular, the Kl or machine learning algorithm Kl recognizes distinctive geometries or geometric curves of the field device FG and / or externally recognizable components, such as a display unit, an electrical connection, a flange, etc.

[0035] As a result of the processing, a type GT of a field device FG and / or a description BS regarding a possible application of the field device FG is suggested to the user. In addition, a three-dimensional model MO of the field device FG is visualized on the control unit BE. Such a three-dimensional model MO can be created from photos of field devices of the same type and / or using design data.

[0036] After user confirmation, the type GT of the field device FG, or the description BS, and the three-dimensional model MO are added to the digital twin TW. If the captured image AN of the field device FG is not sufficient to accurately suggest the type GT or the description BS, additional information IN is requested, for example, an image of the field device FG's surroundings, location information of the field device FG, and / or information regarding the orientation of the control unit BE when the image AN was captured. The additional information IN is fed to the Kl or machine learning algorithm Kl, which again suggests the type GT of the field device FG and / or a description BS of a possible application.

[0037] If this additional information IN is also insufficient to reliably determine and propose the type GT of the field device FG and / or a description BS of a possible application, the user is requested to provide additional information regarding the field device FG and / or the application. For this purpose, a questionnaire contained on the cloud-based platform CL is used. The queries are carried out by the control unit BE or the cloud-based platform CL, with the queries being presented to the user via the control unit BE, and the user enters the additional information via the control unit.

[0038] After each step in which the user is presented with a suggestion, the user is asked to accept or reject it. This feedback is fed into the Kl or machine learning algorithm Kl, allowing it to be further trained and provide even more precise suggestions.

[0039] If a digital twin TW already exists for the field device FG, the user selects it after authentication. After confirming the suggestions, the digital twin TW is extended with the field device FG's type GT, the description BS, and / or the three-dimensional model MO. Reference symbols

[0040] ON recording

[0041] BE Control unit BS Description of a possible application of the field device

[0042] CL cloud-based platform

[0043] FG field device

[0044] GT types of field devices

[0045] IN further information(s) Kl Kl or Machine Learning Algorithm

[0046] MO three-dimensional model of the field device

[0047] TW digital twin

[0048] UK environmental component

Claims

Patent claims 1 . A method for creating or updating a digital twin (TW) for a field device (FG) of automation technology, comprising: - Optical recording of a recording (AN) of the field device (FG) by means of an operating unit (BE); - Transferring the recording (AN) to a cloud-based platform (CL); - Determining a type (GT) of the field device (FG) and / or a description (BS) of a possible application of the field device (FG) based on the image (AN), wherein a previously trained Kl or machine learning algorithm (Kl) analyses the image (AN) with existing images of known types of field devices (FG) and / or design data for the step of determining; and - Creating a digital twin (TW) of the field device (FG), whereby the type (GT) of the field device (FG) and / or the description (BS) of the possible application are added to the digital twin (TW) during creation, or enriching an already existing digital twin (TW) of the field device (FG) with the type (GT) of the field device (FG) and / or the description (BS) of the possible application of the field device (FG).

2. Method according to claim 1, wherein in the event that no unique type (GT) of the field device (FG) and / or no unique description (BS) of a possible application of the field device (FG) can be determined, a user is suggested several possible types of the field device (FG) or several possible descriptions.

3. Method according to claim 1 or 2, wherein for creating the description (BS) of a possible application at least one further piece of information (IN) is determined or made available, which is analyzed by the Kl or machine learning algorithm (Kl).

4. The method according to claim 3, wherein the at least one further piece of information (IN) comprises a recording of the surroundings of the field device (FG), location information of the field device (FG) and / or information regarding the orientation of the operating unit (BE) when capturing the recording (AN).

5. The method according to claim 3 or 4, wherein in the event that even using the at least one further piece of information no unique type (GT) of the field device (FG) and / or no unique description (BS) of a possible application of the field device (FG) can be determined, additional information regarding the field device (FG) and / or about the application is requested from the user, in particular using a questionnaire.

6. The method according to claim 5, wherein the creation of the queries is carried out by the operating unit (BE) or the cloud-based platform (CL), wherein the queries are presented to the user by means of the operating unit (BE) and wherein the user enters the additional information via the operating unit (BE).

7. Method according to one of the preceding claims, wherein the recording (AN) comprises one or more two-dimensional images of the field device (FG) and / or a video of the field device (FG).

8. Method according to one of the preceding claims, wherein the design data are CAD data.

9. Method according to one of the preceding claims, wherein after determining the type (GT) of the field device (FG) or the description (BS), a three-dimensional model (MO) of the field device (FG) is displayed on the operating unit (BE), which shows the same viewing angle of the field device (FG) as the recording (AN).

10. The method according to claim 7, wherein the three-dimensional model (MO) is created based on at least two images of field devices (FG) of the same type. 11 . Method according to one of the preceding claims, wherein the user selects the type (GT) of the field device (FG) and / or the description (BS) for the possible Application confirms or rejects, whereby the confirmation or rejection is fed to the Kl or machine learning algorithm (Kl), whereby the Kl or machine learning algorithm (Kl) is further trained with the confirmation or rejection.

12. Method according to one of the preceding claims, wherein a mobile terminal, in particular a smartphone or a tablet, is used as the operating unit (BE).

Images