Method and apparatus for generating a program code set

Abstract

The invention relates to a computer-implemented method for generating a program code set for the operation of a technical device by an operating apparatus using operating data, wherein the following steps are carried out: a) providing the operating data, including labels of the operating data with respect to predefined features, which describe the operating data structurally and / or functionally, b) providing an input request, comprising definitions for analysis in a domain-specific language and instructions for generating the program code set, c) generating the program code set by means of the large language model using the operating data and the input request, and providing the program code set, d) providing reference program code sets for operating a general technical device, e) classifying the reference program code sets by means of a provided program-code-set model on the basis of artificial intelligence, and assigning to groups in view of the features, f) generating semantic program-code-set descriptions by means of the program-code-set model for the features and / or transfer parameters of the reference program code sets, g) parsing the generated program code sets with the reference program code sets using the features, wherein the program-code-set descriptions are taken into account in the reference program code sets, as a result of which a target program code set is obtained which is intended for the operation of the technical device by the operating apparatus using operating data.

Description

[0001] Method and device for generating program code

[0002] The invention relates to a computer-implemented method and a system for generating program code for operating a technical device with operating data.

[0003] The invention further relates to a computer program product, a computer-readable storage medium or a computer-readable data carrier, and a data carrier signal.

[0004] In the current industrial manufacturing landscape, there is a significant need for image processing know-how and technologies to address challenges related to quality assurance, process optimization, automation, and efficiency improvement.

[0005] As the industry continues to drive digital transformation, the role of computer vision in manufacturing is expected to increase further, driving innovation and competitiveness in the industry.

[0006] In the current manufacturing landscape, there is a notable shortage of people with advanced visual and programming skills.

[0007] The interdisciplinary nature of implementing computer vision solutions in manufacturing further exacerbates this deficiency, as it requires a deep understanding of both technological principles and domain-specific knowledge.

[0008] Classic camera systems with basic image processing functions require operators who are qualified and responsible for basic visual inspection tasks in manufacturing.

[0009] This requires a basic understanding of camera operation and image acquisition techniques, as well as familiarity with simple image processing tools and software, and the ability to perform routine maintenance and troubleshooting on camera systems.

[0010] However, it offers limited functionality compared to more advanced image processing systems, as well as a lack of flexibility and scalability for complex applications that require advanced image analysis.

[0011] Furthermore, the ability to adapt to changing manufacturing requirements and environments is rather limited.

[0012] Generic AI cloud services like "Visual Inspection AI" require operators with production and quality assurance knowledge who want to implement automated inspection solutions without extensive AI expertise. This requires a basic understanding of manufacturing processes and quality control principles, as well as familiarity with cloud-based platforms and services for data processing and analysis. Furthermore, it requires the ability to configure, deploy, and manage AI models using pre-built tools and APIs provided by cloud service providers.

[0013] This enables access to advanced AI and machine learning algorithms without requiring extensive expertise, and supports scalable and flexible solutions that can adapt to diverse manufacturing needs and requirements. Furthermore, integration with cloud infrastructure offers accessibility, reliability, and easy deployment across multiple locations.

[0014] However, this creates a dependency on the internet connection, which can lead to latency and reliability issues in real-time applications, and there are concerns regarding data privacy, security, and regulatory compliance when processing sensitive manufacturing data in the cloud. Furthermore, potential costs associated with usage-based pricing models and ongoing subscription fees can negatively impact the application.

[0015] Deep learning solutions for local manufacturing require data scientists, machine learning engineers, or AI specialists tasked with developing customized image processing solutions tailored to specific manufacturing requirements.

[0016] Advanced knowledge of deep learning algorithms, computer vision techniques and machine learning frameworks (e.g. TensorFlow, PyTorch) is expected, as well as knowledge of programming languages ​​such as Python for model development and deployment, and also expertise in data preprocessing, feature engineering, model training and optimization for manufacturing applications, and a deep understanding of hardware and software integration for deploying deep learning models on edge devices or local servers.

[0017] This provides powerful, real-time processing capabilities suitable for demanding manufacturing environments, enhanced data protection and security through local processing of sensitive manufacturing data without reliance on external services, and improved control and customization in the development, deployment, and optimization of deep learning models tailored to specific manufacturing requirements. However, development and deployment require expertise in deep learning, computer vision, and hardware / software integration from an operator, as well as higher upfront costs associated with hardware infrastructure, software licenses, and ongoing maintenance.

[0018] Furthermore, only limited scalability and flexibility are achieved compared to cloud-based solutions, especially for distributed or geographically distributed production facilities.

[0019] In the prior art, it is known to obtain program codes by applying large language models, such as CoPilot or ChatGPT, but the reliability and quality of the generated program codes are usually insufficient.

[0020] Microsoft Copilot is an assistant function based on artificial intelligence, and a copilot can be understood as a software system that performs a task in interaction with a user, in contrast to software systems that perform a task without user interaction and are called autopilot.

[0021] ChatGPT is an AI-powered voice assistant, developed by OpenAI, based on the GPT-4 architecture and is capable of understanding and generating human speech.

[0022] The publication Garcia-Gonzalez Daniel ET AL: "DSL-Xpert: LLM-driven Generic DSL Code Generation", September 22, 2024 (2024-09-22), XP093250118, DOI: https: / / doi.org / 10.1145 / 3652620.3687782 describes a method in which a large language model (LLM) with a very specific vocabulary, for which these models have not been previously trained, is used, and with the help of so-called semantic parsing, a pre-trained LLM translates an instruction in natural language into the vocabulary of the established domain-specific DSL language. By using the DSL grammar as context (grammar prompt) and providing application examples, such as few-shot learning, the LLM can quickly generate reliable domain-specific code.

[0023] Publication US 2021 / 096543 A1 describes an industrial integrated development environment (IDE) that supports a virtual design environment, enabling the developer of automation systems to carry out project development by interacting with a virtual reality presentation of the plant, with the IDE system interpreting the developer's interactions as design specifications for the automation system to be developed and translating these interactions into control code, visualizations, device configurations and other system aspects that conform to the design specifications.Publication US 2023 / 096325 A1 provides operations for deep parameter learning for code synthesis, including receiving a source code file and generating an abstract syntax tree (AST), generating a dataset with code features and corresponding metadata features that establishes a deep connection between parameters and their use based on the updated computer executable code and the metadata generation task, and training a language model for a sequence-to-sequence generation task based on the dataset.

[0024] The object of the invention is to provide a solution in which complex tasks in the control of devices or the data analysis of operating data of a device, such as optical quality control, can be carried out flexibly without requiring a high level of expertise from an operator.

[0025] In other words, a simple and efficient method for optical quality control is to be proposed.

[0026] The task is accomplished by a computer-implemented method for generating program code to operate a technical device using an operating unit with operating data, comprising the following steps: a) providing the operating data, including markers of the operating data with respect to predefined features that structurally and / or functionally describe the operating data, to a large language model; b) providing an input prompt, including definitions for analysis in a domain-specific language and instructions for generating the program code, to the large language model; c) generating the program code using the large language model with the operating data and the input prompt, and providing the program code to a computing device; d) providing reference program codes for operating a general technical device, which serve as a reference basis for permissible program codes.in particular for the operation of the technical device, form, to the computing device, e) classify the reference program codes using a provided program code model based on artificial intelligence, and assign them into groups taking into account the characteristics, by the computing device, f) generate semantic program code descriptions using the program code model for the characteristics and / or transfer parameters of the reference program codes, by the computing device, g) parse the generated program codes with the reference program codes using the characteristics, whereby the program code descriptions in the reference program codes are taken into account, thereby obtaining a target program code which is intended to operate the technical device with operating data by the operating device.

[0027] This enables an operator to perform quality assurance tasks in a user-friendly and task-specific manner.

[0028] By abstracting implementation details at a low level, an operator is enabled to efficiently navigate quality assurance processes and ensure accurate and timely results, as well as easily access and utilize advanced image processing and analysis functions tailored to specific requirements, thereby optimizing the quality control workflow and increasing productivity.

[0029] Furthermore, the effort required for marking images, the so-called "labeling", is minimized through the use of foundation models and "few-shot learning".

[0030] Operators can use the embedded basic model to quickly adapt to new product types or manufacturing scenarios, thereby reducing the need for extensive marking and training data.

[0031] Furthermore, operators can easily generate accurate segmentation results for quality assurance tasks, significantly reducing the time and effort required for data annotation and model training.

[0032] In the publication Jeong Jongheon ET AL: "WinCLIP: Zero- / Few-Shot Anomaly Classification and Segmentation", March 26, 2023 (2023-03-26), XP093202218, arXiv, DOI:

[0033] 10.48550 / arXiv.2303.14814, https: / / arxiv.org / pdf / 2303.14814 describes an example of a zero-shot classification, which is hereby incorporated into the present invention by reference in order to present an embodiment of zero-shot learning or classification.

[0034] The operational data can be control data for controlling a technical device, or analysis data, which is recorded, for example, by sensors during the operation of a technical device.

[0035] The markings of the operational data can indicate predefined characteristics that describe the operational data structurally and / or functionally, for example object descriptions for the analysis of operational data, or specific control information for controlling a device.

[0036] The input prompt contains definitions for analysis in a domain-specific language and instructions for generating the program code, and is directed to the large language model.

[0037] The program code is generated by applying a large language model, such as CoPilot or OpenAI.

[0038] The reference program codes are program codes intended for the operation of a general technical device and are typically already tried and tested in practice, or already implemented in a product, and are therefore recognized as reliable.

[0039] The reference program codes do not need to be directed to exactly the same technical device as specified in the claim. However, a high degree of technical similarity between the application of a reference program code and the technical device of the claim further increases reliability.

[0040] Thus, the reference program codes exhibit a predetermined minimum reliability that exceeds a specified limit, where the limit is, for example, determined by a degree of testing of the reference program code or by a high degree of technical conformity between the application platform of the technical device for which the reference program code is intended and the technical device of the claim.

[0041] The provided program code model based on artificial intelligence can, for example, be generated and trained using a large number of provided program codes, whereby the program code model is parameterized according to characteristics such as function, structure or parameters passed in the program code.

[0042] The semantic program code descriptions of the reference program codes can, for example, describe the function or the parameters in the program code in more detail.

[0043] Operating a technical device can be understood as the evaluation of analytical data from a technical device or the generation of control data for controlling a technical device, whereby the analytical data or the control data constitute the operating data of the technical device.

[0044] A parser is a program, usually part of a compiler, which ensures that program code is correctly translated into machine-executable language. A parser performs a syntax analysis of the provided program code, examining its structure by breaking it down into its constituent parts and checking or correcting them.

[0045] In addition to program code, the parser receives a reference program code which includes examples of correct syntax, thus representing a reference library for valid and performant program code.

[0046] In other words, parsing is the process of analyzing and interpreting a string or data stream in order to understand its structure and put it into a form suitable for further processing.

[0047] This involves a transformation or translation of an input program code to be parsed into a form of reference program code with a permissible reference structure, which is defined or determined, for example, by a subsequent system such as a technical device with an operating device.

[0048] The reference structure of the reference program codes includes permissible, i.e., valid and performant, program codes, especially for the operation of the technical device.

[0049] The task of the parser is to decompose and convert an input into a usable format for further processing, whereby such a format can be defined by a provided reference format.

[0050] For example, a command sequence formulated in a programming language is syntactically analyzed and then broken down into its individual components.

[0051] In process step g), the generated program codes are accordingly interpreted during parsing using the reference program codes, which form a basis for permissible, i.e., valid and performant program codes.

[0052] The reference program codes are used in particular for the operation of the technical device.

[0053] Thus, the previously generated program codes are analyzed and checked using parsing, and if necessary adapted accordingly, and translated into a format compatible or permissible with the reference program codes, which is verified, for example, in terms of both syntax and efficiency.

[0054] The interpretation is performed using the characteristics that structurally and / or functionally describe the operational data, taking into account the program code descriptions in the reference program codes. An abstract syntax tree (AST) is a data structure used in computer science to represent the structure of a program or code segment.

[0055] It is a tree representation of the abstract syntactic structure of a text written in a formal language (often source code), where each node of the tree represents a construction that occurs in the text.

[0056] The syntax is “abstract” in that it does not represent every detail that occurs in the real syntax, but only the structural or content-related details.

[0057] For example, grouping brackets are implicitly included in the tree structure, so they do not need to be represented as separate nodes.

[0058] Similarly, a syntactic construction such as an if-condition-then statement can be represented by a single node with three branches.

[0059] This distinguishes abstract syntax trees from concrete syntax trees, which are traditionally called parse trees.

[0060] Parse trees are normally created by a parser during the translation and compilation of the source code.

[0061] After creation, additional information is added to the AST through subsequent processing, e.g., context analysis.

[0062] Abstract syntax trees are also used in program analysis and program transformation systems.

[0063] The operation of a technical device is carried out by an operating device, usually in the form of an electronic control device, with operating data, usually in the form of control commands, which are executed by the control device in order to operate the connected technical device accordingly.

[0064] The technical device can be, for example, a production device such as an injection molding machine, a processing device such as a CNC milling machine, an operating device such as a pump or a fan, or a transport device such as a conveyor belt, or similar.

[0065] In a further development of the invention, the target program code is transferred from the computing device to an operating device, and the operating device then operates the technical device with the target program code. This allows the generated program code to be executed efficiently on a target device, such as an edge device.

[0066] In a further development of the invention, it is provided that the operation of the technical device relates to the evaluation of analysis data of the technical device, and the operating data are formed by the analysis data which are acquired by a corresponding acquisition device which is encompassed by the operating device.

[0067] This allows for the simple and efficient creation of program code for the analysis of captured images, for example by a camera during the visual inspection of manufactured products by a corresponding production machine.

[0068] In a further development of the invention, it is provided that the operation of the technical device relates to the generation of control data for controlling the technical device, and the operating data are formed by the control data.

[0069] This allows for the simple and efficient creation of program code to control a device, such as a production machine for manufacturing products, or operating equipment such as pumps, motors, drives, etc.

[0070] In a further development of the invention, it is provided that the markings of the provided operational data are generated using a provided analysis model based on artificial intelligence, and the analysis data are image data, and in the image data the markings are defined using the analysis model for predefined objects or structures.

[0071] This allows, for example, image identification markers, so-called "text tokens", to be generated for the markings of the operational data, which describe the image context of the image data.

[0072] These text tokens can then be injected into the input prompt of the large language model to enable the suggestion or generation of even more suitable program codes or building blocks of / for program codes.

[0073] This allows for the simple and efficient generation of program code for the analysis of captured images, for example in object recognition during the visual inspection of manufactured products by a corresponding production machine.

[0074] The problem according to the invention is also solved by a system for generating program code for operating a technical device by means of an operating device with operating data, comprising a computing device with a processor and a memory, wherein the system is configured to execute the method according to the invention.

[0075] In a further development of the invention, it is provided that the system also has an operating device which is set up to operate the technical device with the program code.

[0076] The problem according to the invention is also solved by a computer program product with machine-readable instructions stored therein, wherein the instructions, when executed by the system according to the invention, cause it to execute the method according to the invention.

[0077] The problem according to the invention is also solved by a computer-readable storage medium or a computer-readable data carrier comprising instructions which, when executed by a computer, cause it to execute the method or the method steps according to the invention.

[0078] The problem according to the invention is also solved by a computer-readable data carrier on which the computer program product according to the invention is stored.

[0079] The problem according to the invention is also solved by a data carrier signal that transmits the computer program product according to the invention.

[0080] The invention is explained in more detail below with reference to an embodiment illustrated in the accompanying drawings. The drawings show:

[0081] Fig. 1 shows a first embodiment of a system according to the invention,

[0082] Fig. 2 shows a second embodiment of a system according to the invention,

[0083] Fig. 3 shows an embodiment of a method according to the invention,

[0084] Fig. 4 shows a further embodiment of a flowchart of the method according to the invention,

[0085] Fig. 5 shows a further embodiment of a flowchart of the method according to the invention,

[0086] Fig. 6 shows an example of a captured image for processing by the method according to the invention.

[0087] Fig. 7 shows an example of an automatically performed segmentation,

[0088] Fig. 8 shows an example of DSL instructions for parsing.

[0089] Figures 1 and 2 each show exemplary embodiments of a system according to the invention. The system for generating program code for operating a technical device TD with operating data comprises a computing device (RV) with a processor and a memory.

[0090] The system is set up to carry out the method according to the invention.

[0091] The system also includes an operating device BV, which is designed to operate the technical device TD with the program code.

[0092] In the system according to Fig. 1, the operation of the technical device is carried out using control data for controlling the technical device, for example in the form of a program code.

[0093] Control data is generated, which forms the basis for the operation of the technical device TD.

[0094] In the system according to Fig. 2, the operation of the technical device is carried out using analysis data from the technical device, for example image data from the production of a product by a corresponding production device.

[0095] The operational data is formed from the analysis data, which is captured by a suitable recording device, such as a camera.

[0096] Fig. 3 shows an embodiment of a method according to the invention in the form of a flowchart.

[0097] The procedure for generating a target program code for operating a technical device TD by an operating device BV with operating data is computer-implemented in at least one step.

[0098] The following steps are performed: a) Providing the operational data, comprising markers L_IMG of the operational data with respect to predefined features that structurally and / or functionally describe the operational data, to a large language model, LLM; b) Providing an input prompt T_PR, comprising definitions for analysis in a domain-specific language, DSL, and instructions for generating the program code, to the large language model; c) Generating the program code using the large language model with the operational data and the input prompt, and providing the program code to a computing device; d) Providing reference program codes for the operation of a general technical device, which form a reference basis for permissible program codes, in particular for the operation of the technical device, to the computing device.e) Classifying the reference program codes using a provided program code model based on artificial intelligence, and assigning them to groups taking into account the characteristics, by the computing device; f) Generating semantic program code descriptions using the program code model for the characteristics and / or transfer parameters of the reference program codes, by the computing device; g) Parsing the generated program codes with the reference program codes using the characteristics, taking into account the program code descriptions in the reference program codes, thereby obtaining a target program code which is intended to operate the technical device TD with operating data via the operating device BV, by the computing device.

[0099] The target program code can be transferred from the computing device RV to an operating device BV.

[0100] The technical device TD can be operated by the operating device BV with the target program code.

[0101] The operation of the technical device TD may involve the evaluation of analysis data from the technical device TD.

[0102] The operational data can be formed from the analysis data, which are recorded by a corresponding recording device, which is included in the operating device BV.

[0103] The operation of the technical device TD may involve the generation of control data for controlling the technical device TD.

[0104] The operational data can be formed from the tax data.

[0105] The L_IMG markings of the provided operational data can be generated using a provided analysis model based on artificial intelligence.

[0106] The analysis data can be image data.

[0107] In the image data, the L_IMG markers can be defined using the analysis model for predefined objects or structures.

[0108] Fig. 4 and Fig. 5 show a further embodiment of a flowchart of the method according to the invention.

[0109] The process begins with the acquisition of unlabeled images, which are then segmented using a large vision transformer model to identify, for example, different visual features. So-called "zero-shot learning" or "one-shot learning" techniques enable rapid model adaptation.

[0110] Feeding a domain-specific language (DSL) into the CoPilot context enables seamless code generation and post-processing.

[0111] Templates (TEMPL) can be stored in an artifact register (REG) for reproducibility before the model is deployed to a target environment for real-time inference.

[0112] First, an image without labels is captured (engl. “ingest unlabeled image”).

[0113] In this process, raw images without associated labels or annotations are entered into the system, enabling further processing and analysis to extract meaningful insights or patterns for quality control in manufacturing.

[0114] This is followed by segmentation of the image using a large vision transformer model, which is a foundational model used to segment images into different areas or objects based on their visual properties, analysis, and post-processing for downstream tasks such as quality assurance.

[0115] Subsequently, automated marking of data is carried out using machine learning through "zero-shot learning" or "one-shot learning".

[0116] This learning approach allows models to generalize from a single or minimal number of examples per class, enabling rapid adaptation to new or unseen data without extensive labeled training datasets.

[0117] Subsequently, an input of DSL information, l_DSL, is made to the CoPilot context.

[0118] Integrating a domain-specific language (DSL) into the CoPilot context involves embedding domain-specific commands or instructions into the CoPilot environment to enable seamless collaboration between manufacturing professionals and AI models to generate code snippets or automate tasks.

[0119] This can be achieved through "prompt engineering" or "retrieve, augment, generate", or RAG for short, whereby the DSL can be maintained in a knowledge base.

[0120] The aforementioned manufacturing professionals lack extensive programming knowledge, such as in Python programming. The following describes the generation of post-processing program code using CoPilot.

[0121] CoPilot uses AI support to automatically generate post-processing code and refine or improve the output of the primary AI models or algorithms, thereby streamlining the development process and improving efficiency.

[0122] Then, a template TEMPL is saved to an artifact registry REG.

[0123] Storing templates containing pre- / post-processing code and model code for runtime inferences / predictions in an artifact registry enables the storage and versioning of reusable and deployable components.

[0124] Finally, the DEPL of the model is distributed to the target environment of the technical device TD.

[0125] This includes deploying trained templates in the target production environment, such as edge devices or cloud servers, enabling real-time inference and integration into operational workflows to create added value at scale.

[0126] Fig. 5 shows a further embodiment of a flowchart of the method according to the invention, based on the representation according to the preceding figure.

[0127] The illustration shows image recognition using image data captured by a camera CAM during the production of a product by a corresponding production device, whereby produced products are transported past the camera CAM on a conveyor belt CON.

[0128] A captured image IMG is initially unlabeled and is fed to an image encoder l_ENC, generating the so-called "image encodings".

[0129] Subsequently, a mask decoder M_DEC performs the masking with provided input prompts, which were processed by an input prompt encoder P_ENC, and labeled images L_IMG (English "labeled images") are created and stored in a memory.

[0130] A subsequent input prompt provides the CoPilot with one or more text input prompts T_PR.

[0131] Information regarding “pre-processing” program codes PREP_C, “post-processing” program codes POSTP_C and segmentation codes SEG_C, which represent inference, is provided from a knowledge base database KB.

[0132] The text input prompts T_PR, as well as the “pre-processing” program codes PREP_C, the “post-processing” program codes POSTP_C and the segmentation codes SEG_C are combined into a template TEMPL, which is then used to perform parsing, the generated target program codes of which are stored in an artifact registration REG, or are used to operate a technical device.

[0133] The technical device is preferably connected to a client of a client-server system.

[0134] Fig. 6 shows an example of a captured image for processing by the method according to the invention,

[0135] The image shown is IMG, which was captured by the camera CAM on the conveyor belt CON.

[0136] The image IMG shows a product, namely a plate P with holes H1-H4 in the plate P.

[0137] The plate P is automatically segmented in the process, as previously discussed, for example according to predefined quality criteria.

[0138] Such quality criteria for plate P could be that four holes must be present, and the distance between the holes must always be the same, such as 5 cm.

[0139] Fig. 7 shows an example of an automatically performed segmentation, in which the program code shown was generated using DSL with the CoPilot.

[0140] The first instruction in the program code CountObjects(segmentedJmage) == 4 means that the instruction checks whether the number of segmented objects in the image is four.

[0141] The first instruction in the program code MeasureDistance(segmented_objects) == 5cm means that the instruction checks whether the distance between each pair of segmented objects, i.e. the holes, is exactly 5 cm.

[0142] Fig. 8 shows an example of DSL instructions for parsing.

[0143] DSL instructions are parsed into an abstract syntax tree AST, resulting in optimized program code.

[0144] The code snippet shown analyzes the DSL commands in an AST and then iterates over the AST nodes to extract and execute each command along with its associated constraint.

[0145] The following functional signatures (implementation of the DSL commands) are provided:

[0146] The function `count_objects(segmented_image)` takes an image containing segmented objects as input and returns the number of segmented objects found in the image. The function `measure_distance(segmented_objects, distance_threshold)` takes a list of segmented objects and a distance threshold as input, measures the distance between each pair of segmented objects, and returns a list of the distances between the objects. Simple examples of a DSL for measurement applications are shown here.

[0147] CountObjects(segmentedJmage): Count the number of segmented objects in the given image,

[0148] MeasureDistance(segmented_objects): Measure the distance between a pair of segmented objects, - MeasureLength(segment): Measure the length of a given segment,

[0149] MeasureArea(region): Measure the area within a specified region of the image.

[0150] Reference symbol list

[0151] BV Operating Device

[0152] CAM camera

[0153] CON conveyor belt

[0154] DEPL Distributing the model to the target environment

[0155] DSL domain-specific language (engl. “domain specific language”)

[0156] Generating "post-processing" program code with CoPilot

[0157] H1-H4 Hole l_DSL Input of DSL information to CoPilot context l_EMB Image “embedding” l_ENC Image encoder l_ULI Capture of an image without markers (engl. “ingest unlabeled image”)

[0158] IMG image

[0159] KB database (English: "knowledge base")

[0160] L_IMG Labeled images

[0161] LVTM Large Vision Transformator Model

[0162] M_DEC Mask Decoder

[0163] ML Machine Learning with “zero-shot learning” or “one-shot learning”

[0164] P plate

[0165] P_ENC Input Prompt Encoder

[0166] POSTP_C “post-processing” program code

[0167] PREP_C “pre-processing” program code

[0168] REG artifact registration

[0169] RV rake device

[0170] SEG_C Segmentation Code, Inference

[0171] Sl Segmenting the image with LVTM

[0172] ST Saving a template TEMPL in the artifact registry REG

[0173] T_PR Text input prompt

[0174] TD technical device

[0175] TEMPL Template

Claims

Patent claims# 1. Computer-implemented method for generating target program code for operating a technical device (TD) by an operating device (BV) with operating data, comprising the following steps: a) providing the operating data, comprising markers (L_IMG) of the operating data with respect to predefined features that structurally and / or functionally describe the operating data, to a large language model (LLM); b) providing an input prompt (T_PR), comprising definitions for analysis in a domain-specific language (DSL) and instructions for generating the program code, to the large language model; c) generating the program code using the large language model with the operating data and the input prompt, and providing the program code to a computing device; d) providing reference program code for operating a general technical device.which form a reference basis for permissible program codes, in particular for the operation of the technical device, to the computing device; e) classifying the reference program codes using a provided program code model based on artificial intelligence, and assigning them into groups taking into account the characteristics, by the computing device; f) generating semantic program code descriptions using the program code model for the characteristics and / or transfer parameters of the reference program codes, by the computing device; g) parsing the generated program codes with the reference program codes using the characteristics, whereby the program code descriptions in the reference program codes are taken into account, thereby obtaining a target program code which is intended to operate the technical device (TD) with operating data by the operating device (BV), by the computing device.

2. Method according to claim 1, wherein the target program code is transferred from the computing device (RV) to an operating device (BV), and the technical device (TD) is operated by the operating device (BV) with the target program code.

3. Method according to claim 1 or 2, wherein the operation of the technical device (TD) relates to the evaluation of analysis data of the technical device (TD), and the operating data are formed by the analysis data which are acquired by a corresponding acquisition device which is encompassed by the operating device (BV).

4. Method of claim 1 or 2, wherein the operation of the technical device (TD) relates to the generation of control data for controlling the technical device (TD), and the operating data are formed by the control data.

5. Method according to claim 3, wherein the markings (LJ MG) of the provided operational data are generated using a provided analysis model based on artificial intelligence, and the analysis data are image data, and in the image data the markings (L_IMG) are defined using the analysis model for predefined objects or structures.

6. System for generating program code for operating a technical device (TD) by means of an operating device (BV) with operating data, comprising a computing device (RV) with a processor and a memory, wherein the system is configured to execute the method according to one of the preceding claims.

7. Computer program product comprising machine-readable instructions stored therein which, when executed by the system according to one of the preceding claims, cause the system to execute the method according to one of the preceding claims.

Images