Describing a device using a large language model

Abstract

The aim of the invention is to simplify the integration of devices into complex systems. For this purpose, a method is proposed for automatically describing a device (1) in a specified format. First, a device description in a proprietary format and an example description in the proprietary format are provided together with a transformed example description, the content of which corresponds to the example description in the specified format; the device description in the proprietary format, the example description in the proprietary format and the transformed example description are input into a large language model (9); and the large language model (9) outputs the device description (3) in the specified format.

Description

[0001] Description

[0002] Describing a device with a large language model

[0003] The present invention relates to a method for automatically writing a device in a predetermined format. Furthermore, the present invention also relates to a device for automatically writing a device and to a device arrangement comprising a device and an application device on which an application is implemented.

[0004] With an ever-growing number of IoT (Internet of Things) devices, for example, in industry or the energy sector, application developers face the challenge of creating interfaces and communicating with a heterogeneous landscape of different devices and device manufacturers using both open and proprietary protocols. Typical devices in industrial plants include sensors, electromechanical components, communication units, and drives. Common protocols used to facilitate communication between these components include HTTP, JSON, Modbus, XML, and so on.

[0005] In new projects, the effort required to implement protocol adapters for devices creates significant implementation costs and can undermine the overall goal of a solution. For example, an energy management system must read measured values ​​and send setpoints to a variety of devices, such as battery storage systems, inverters, charging stations, and so on.

[0006] The Web of Things approach (see FIG 1) enables streamlined connectivity for applications (apps) with various devices and their specific device-specific connectivity. The devices and their communication interfaces are abstracted by so-called thing descriptions (TDs), which define the properties, actions, and events for an individual device (e.g., reading a temperature measurement, setting an active power setpoint, etc.). The applications no longer have to directly handle proprietary formats or protocols, but can instead rely on predefined or standardized protocols and formats.

[0007] Although such an approach solves the connectivity challenge for application developers with respect to applications 2, each device 1 must first be described in a suitable format (thing description). This requires considerable work, as each new device type must be manually integrated into a system with all manufacturer-provided connectivity information (e.g., directly from the manufacturer's manual).

[0008] For solutions without a Web of Things approach, application developers would have to manually implement the connectivity stack (connection table for input and output ports) for each device in the project and configure each device according to the manual. With the Web of Things approach (specifically https: / / www.w3.org / WoT / ), a commissioning engineer would still have to manually fill out a Thing Description 3 from the device's manual, which sometimes contains several hundred properties.

[0009] Regardless of the grammatical gender of a particular term, persons with male, female or other gender identities are included.

[0010] The object of the present invention is therefore to simplify the integration of a device into a complex system.

[0011] According to the invention, this object is achieved by a method and a device according to the independent claims. Advantageous developments of the invention emerge from the subclaims.

[0012] According to the present invention, a (computer-implemented) method for automatically describing a device in a predetermined format is provided. As mentioned above, the device can be, for example, a measuring device (sensor), an electromechanical element (e.g., a robot arm), a communication device (e.g.,

[0013] This could involve a transmitter / receiver, a drive (e.g., electric motor), and the like. The device should be automatically described, particularly with regard to its connectivity (connection ports). Thus, the description of the device is not done manually, but rather with the help of an automated system (e.g., implemented on a computer).

[0014] First, a device description is provided in a proprietary format. The device description relates to the physical variables of the device, particularly those relating to input and output connections. For example, according to the device description, a current temperature in degrees Celsius is represented with two bytes. Similarly, according to a proprietary format, reactive energy in VAh can be specified with 4 bytes. In addition, an example description is provided in the proprietary format together with a transformed example description, the content of which corresponds to the example description in the specified format. An example description can relate to any example device, such as a temperature sensor. For the example description, as for the device description above, a description (e.g., "maximum current"), a unit (e.g., A), a byte length (e.g.,2) and, if necessary, other variables such as registers, device versions, and the like. A transformed example description with identical content in the specified format (e.g., JSON format) is provided for this example description. Thus, two example descriptions with identical content are provided in two different formats (proprietary format and specified format), from which the format transformation emerges.

[0015] In a next step, the device description in the proprietary format, the example description in the proprietary format, and the transformed example description in the specified format are entered into a large language model (LLM). In addition to these three descriptions as input variables, an instruction is also entered into the large language model as usual, according to which the device description is to be transformed from the proprietary format to the specified format, analogous to the example descriptions. If necessary, additional instructions can also be entered into the large language model.

[0016] The large language model outputs the device description in the specified format. Due to the format transformation defined by the example descriptions, the large language model is able to transform the device description in the proprietary format into a novel device description in the specified format. It is not necessary for the large language model to undergo a lengthy training phase. Instead, the large language model can be used instantly using the example input.

[0017] This advantageously results in a user-friendly approach that leverages a large language model to automatically generate valid object descriptions directly from plant manuals and the like. The proposed approach will significantly reduce commissioning effort. The main advantage of the proposed approach is that the connection of new devices (e.g., during commissioning of an industrial building) can be fully automated. Compared to conventional approaches, this drastically reduces the effort required to commission a new site. With this approach, untapped potential for data-driven use cases, e.g., energy management systems, can be easily tapped to increase energy efficiency, for example.

[0018] In one embodiment, it is provided that a raw device description is automatically recorded and the device description in the proprietary format is automatically extracted and provided from the raw device description. The raw device description is, for example, an image file scanned from a manual. The recording of the raw device description is therefore (semi-)automatic, e.g., using a scanner. The current device description is then extracted from the raw device description (e.g., image file) in the proprietary format, e.g., based on automatic text recognition. The provision of the device description is therefore preceded by automatic recording with two automatic steps, namely the recording of the raw device description and the extraction of the device description from it. A highly automated recording process can thus advantageously be provided.

[0019] According to another embodiment, the specified format is a standardized format. The specified format into which the device description is transformed is thus a standardized format. This means that, for example, during commissioning, the developers of corresponding applications can rely on standardized formats for the device descriptions.

[0020] In a further embodiment of the method according to the invention, the device description contains connection data and / or connection data specific to the device. As already indicated above, the device description can thus contain connection data, for example, relating to input or output or relating to registers, ports, and the like. In general, all known physical quantities can be described with the device description.

[0021] According to a further embodiment, the device description can be extracted from a manual by the large language model for provision. For example, the entire manual can be available in PDF format. Under certain circumstances, the entire manual is then made available to the large language model. If necessary, a suitable extraction algorithm recognizes a connection table in the manual, which is then extracted and transformed into the specified format. In a further embodiment, it is provided that the device description output by the large language model in the specified format is automatically validated with respect to the specified format. In this validation step, it is checked whether the specified or standardized format is also adhered to in the device description. This validation can be carried out automatically using a validation algorithm.

[0022] According to another embodiment, the device description output by the large language model is validated in the specified format in a validation dialog with a user. This means that a validation interface must be provided that allows the user to check the device description for format and, if necessary, supplement or modify it. This ensures that the automatically generated device description in the specified format is also error-free.

[0023] In a further embodiment, at least one further example description in the proprietary format and one further transformed example description for outputting the device description in the specified format are input into the large language model. This means that the language model receives not just a single example for the format transformation, but at least two examples. This can increase the security of the format transformation.

[0024] According to another embodiment, the device description is output in the specified format in a JSON format. The JSON format is a widely used format for describing connectivity. This allows the device description to be provided in a very common format, making the method more universally applicable.

[0025] In a further embodiment, it is provided that, in addition to the device description in the proprietary format, a user enters one or more instructions into the large language model for outputting the device description in the specified format. Thus, in addition to the device description and the at least two example descriptions, at least one user instruction is entered into the large language model. Such instructions can, for example, relate to the type of parameters that may appear in the device description to be output. Likewise, an instruction can relate to which parameters (names) may not be used. Furthermore, an instruction can also relate to inserting parameters such as maximum or minimum if necessary. Such instructions instruct the large language model precisely how the output should be carried out.

[0026] Thus, a method can be provided for communicating between an application and a device by describing the device in a predefined format according to a method as described above, and for the application to communicate with the device based on the device description in the predefined format. Communication between an application and a device can thus be realized based on a predefined connection protocol rather than a proprietary connection protocol.

[0027] The communication method can further provide for the application to communicate with the device to transmit a value of an operating parameter of the device from the device to the application and / or from the application to the device. In this way, values ​​can be exchanged between the application and the device. For example, an application can read a measured value from a device or an application can configure a device setting on a device.

[0028] The object formulated above can also be achieved according to the invention by a device for automatically describing a device in a predetermined format, comprising a memory device for providing a device description (which describes physical quantities of the device) in a proprietary format and for providing an example description in the proprietary format together with a transformed example description which corresponds in content to the example description in the predetermined format, a processor device on which a large language model is implemented, wherein the device description in the proprietary format, the example description in the proprietary format and the transformed example description can be input from the memory device into the processor device, and wherein the device description in the predetermined format can be output by the processor device using the large language model.

[0029] The advantages and further development possibilities listed above in connection with the method according to the invention also apply mutatis mutandis to the device according to the invention. Accordingly, the method features mentioned can be viewed as functional features of corresponding means or devices of the device. Furthermore, according to the invention, a device arrangement can be provided comprising a device, an application device on which an application is implemented, and a device just mentioned with which the device can be written to in the specified format, so that communication between the application and the device is possible based on the specified format. The application device can be a computer or a similar computing device.

[0030] The above-mentioned object is further achieved by a computer program product with program code means that cause an electronic computing device to perform a method of the above-mentioned type when the program code means are processed by the electronic computing device. Furthermore, a computer-readable storage medium with at least one computer program product of the above type can be provided.

[0031] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0032] The present invention will now be explained in more detail with reference to the accompanying drawings, in which:

[0033] FIG 1 is a schematic view of a Web of Things approach to simplifying device connectivity;

[0034] FIG 2 is a schematic view of a method example according to the present invention; and

[0035] FIG 3 an example of a graphical user interface for creating a device description in a given format.

[0036] The exemplary embodiments described in more detail below represent preferred embodiments of the present invention. The following examples are based on descriptions using the Web of Things standard (thing descriptions). However, the present invention can also be implemented using other IoT connectivity approaches.

[0037] According to FIG. 2, a device manual 5 is available for device 1. The device manual 5 can be available, for example, in a spreadsheet format, a PDF format, an HTML format, or the like. If necessary, only a section of the device manual 5 may be available in a corresponding format. Such a section is also referred to below as the device manual 5.

[0038] In an optional step of the embodiment of FIG 2, the extraction 6 of a connection specification from the device manual 5 takes place. Thus, the device manual 5 of a device 1 to be integrated into a system (e.g., a PDF file, an HTML page, an Excel spreadsheet) is used to extract the relevant information for the connectivity of the device 1, preferably in a table format. Alternatively, a large language model (hereinafter referred to as LLM) could be used to query the entire PDF file and find the corresponding table. However, this requires an additional call to the LLM, possibly with a predefined set of usable tokens / items.

[0039] Table 1 below shows an example of such a specification. Table 1 may be reproduced in the manual in the format shown.

[0040] Measurement and state variables

[0041] Table

[0042] The displayed table can be an image or already in a table format. Depending on this, conversion to a table format may or may not be necessary. Conversion to a table format can be done either manually, i.e., by copying the content and formatting it with a spreadsheet program such as Excel, or by automatically converting the table using an LLM (e.g., ChatGPT). In the latter case, the LLM converts text input into a table format. The connectivity information can now be extracted from this table and evaluated in the following steps.

[0043] First, a so-called “prompt” for the LLM must be created from the table containing the connectivity information.

[0044] For example, a raw device description with the connection data of device 1 was automatically captured from device manual 5, e.g., as a PDF file (e.g., by scanning). Extraction step 6 extracts the device description with the connection specification from the raw device description. The latter is thus available in a proprietary format.

[0045] Now, a prompt for an LLM (e.g., ChatGPT) is generated. For this purpose, a processing step 7 is provided in FIG. 2. This generates a part of the prompt for the LLM 8, which contains information about the device 1 to be integrated. Another part of the prompt consists of training examples 9.

[0046] For example, a standard prompt consists of a sequence of two training examples, each with the desired output for a corresponding input. The input is defined as a dictionary of the values ​​of a row of the documentation table or device description. No specific column names are required. If the respective name has a meaning, it is processed correctly by the LLM. The output is, for example, a property description in a JSON format.

[0047] In processing step 7, the user has the option of specifying additional instructions, such as general instructions (for example, “all integer variables should be described as floating-point numbers”) or an additional example.

[0048] In a further step, the LLM 9 is called. The prepared prompt, including the device description and one or more training examples 8, is fed to the LLM 9. There may be a restriction regarding the maximum number of input tokens for the LLM 9. Under certain circumstances, the prompt is split if the required device configuration or device description is longer than the maximum possible token size.

[0049] The LLM 9 generates a preliminary thing description 3' from the prompt.

[0050] In an optional validation step 10, the generated preliminary thing description 3' is validated against a standard (e.g., Web of Things), resulting in a validated thing description 3. This validation is performed to generally check the format of the generated thing description. If the LLM 9 does not recognize relevant information in the table, additional context can be added by the user and iterated with the LLM 9 in an optional validation dialog 11. The two optional validation steps 10 and 11 thus result in the validated thing description 3. In principle, validation could, of course, be omitted if the LLM 9 directly outputs the thing description 3.

[0051] In a further optional validation step, the actual connectivity to a real device could be tested. The generated thing description 3 of device 1 can be used directly by application 2 for the interface to device 1. For example, measured values ​​can be read from device 1 using application 2. Likewise, setting values ​​can now be written to device 1 using the application or application 2 via the thing description 3.

[0052] To further understand the inventive approach, the inventive method is illustrated using a process for onboarding a single property of a new battery. A standard prompt template for any device is used. The subsequent prompt consists of a multi-line instruction section, followed by an "input" section and an "output" section.

[0053] The only valid descriptions for the "type" parameter are "integer", "boolean", "number" and "string".

[0054] The only valid descriptions for the "modbus:type" parameter in the "forms" description are "integer", "boolean", "number", and "string".

[0055] The only valid names for the "modbus.entity" parameter in the "forms" description are "coil", "discreteinput", "inputregister", and "holdingregister".

[0056] It is absolutely forbidden to name a parameter 'enum'.

[0057] The parameters 'maximum' and 'minimum' can be added if required.

[0058] Input: {"Offset": 1, "Number of registers": 2, "Name": "Voltage L1-N", "Format": "Float", "Unit": "V", "Value range": "Access": "R"}

[0059] Output:

[0060] {

[0061] "VoltageL1-N": {

[0062] "type": "number",

[0063] "readonly": true,

[0064] "observable": true,

[0065] "forms": [

[0066] {

[0067] "href: " / 1?address= 1&quantity=2",

[0068] "op": [

[0069] "readproperty",

[0070] "observeproperty" ],

[0071] "modbus:type": "number",

[0072] "modbus:entity": "holding register", "modbus:pollingTime": 2000}

[0073] ]

[0074] }

[0075] The "input" part is generated from the device description, i.e., the table extracted from the device manual 5. The "output" part displays the data from the "input" part in a transformed format, e.g., JSON format. The example with the "input" part and "output" part is intended to demonstrate to LLM 9 how the format transformation should take place.

[0076] FIG 3 now shows how the method according to the invention can be used, for example, with the aid of a simple user interface (GUI).

[0077] The first step involves extracting connectivity information. To do this, a line from a documentation table in a device manual (e.g., copied from a PDF file) is inserted into a text area 12. This represents a raw device description. It is converted into a table 13 below it, which can be evaluated. This table 13 can be referred to as the device description.

[0078] Clicking the "Generate JSON" button 14 converts table 13 into suitable input for the LLM. The LLM 8 is started with the prompt shown above. This creates the object description. In the example shown in FIG. 3, a JSON file is created as output 15. It can be seen in the lower section of FIG. 3.

[0079] With the generated battery object description according to Edition 15, any application (e.g., an energy management system) can directly receive measurements and send setpoints to the device without having to worry about the underlying connection protocol.

[0080] 1 device

[0081] 2 Application 3 Description of the thing

[0082] 3' preliminary description of the thing

[0083] 4 Format

[0084] 5 Device manual

[0085] 6 Extraction 7 Processing

[0086] 8 LLM

[0087] 9 training examples

[0088] 10 Validation step

[0089] 11 Validation dialog 12 Text area

[0090] 13 Table

[0091] 14 Button

[0092] 15th issue

Claims

Patent claims 1. Method for automatically writing a device (1) in a predetermined format, characterized by - Providing a device description of the device (1) in a proprietary format, - Providing an example description in the proprietary format together with a transformed example description that corresponds in content to the example description in the specified format, - entering the device description in the proprietary format, the example description in the proprietary format and the transformed example description into a large language model (9), - Output of the device description in the specified format (3) by the large language model (9).

2. The method according to claim 1, wherein a raw device description is automatically acquired and the device description in the proprietary format is automatically extracted (6) and provided from the raw device description.

3. The method according to claim 1 or 2, wherein the predetermined format is a standardized format.

4. Method according to one of the preceding claims, wherein the device description contains connection data and / or connection data specific to the device (1).

5. The method according to claim 4, wherein the device description is extracted (6) from a manual (5) by the large language model (9) for providing.

6. Method according to one of the preceding claims, wherein the device description output by the large language model (9) in the predetermined format is automatically validated (10) with respect to the predetermined format.

7. Method according to one of the preceding claims, wherein the device description output by the large language model (9) is validated in the predetermined format in a validation dialogue (11) with a user.

8. Method according to one of the preceding claims, wherein at least one further example description in the proprietary format and a further transformed example description for outputting the device description in the predetermined format is input into the large language model (9).

9. Method according to one of the preceding claims, wherein the device description is output in the predetermined format in a JSON format.

10. Method according to one of the preceding claims, wherein, in addition to the device description in the proprietary format, a user enters one or more instructions into the large language model (9) for outputting the device description in the predetermined format (3).

11. Method for communicating an application (2) with a device (1) by describing the device (1) in a predetermined format according to a method according to one of the preceding claims and communicating the application (2) with the device (1) on the basis of the device description in the predetermined format (3).

12. The method according to claim 11, wherein communicating the application with the device (1) includes transmitting a value of an operating parameter of the device (1) from the device (1) to the application (2) and / or from the application (2) to the device (1).

13. Device for automatically writing a device (1) in a predetermined format, characterized by - a storage device for providing a device description in a proprietary format and for providing an example description in the proprietary format together with a transformed example description which corresponds in content to the example description in the predetermined format, - a processor device on which a large language model (9) is implemented, wherein - the device description in the proprietary format, the example description in the proprietary format and the transformed example description can be input from the storage device into the processor device, and wherein - the device description can be output by the processor device in the predetermined format (3) by means of the large language model (9).

14. Device arrangement with - a device (1), - an application device on which an application (2) is implemented, - a device according to claim 13, with which the device (1) can be written in the predetermined format, so that - communication between the application (2) and the device (1) is possible on the basis of the specified format.

15. A computer program product comprising program code means which cause an electronic computing device to carry out a method according to one of claims 1 to 12 when the program code means are processed by the electronic computing device.

Images