Method and apparatus for generating project-specific network architecture

Abstract

The invention relates to a method and apparatus for generating a project-specific network architecture. The invention relates to a method for generating a project-specific network architecture, said method comprising the steps of:-providing (S1) a base model, in particular a large language model, with LoRa network adaptation; providing (S2) a model library having training data pairs, each of which consists of input data having a model application and / or model and / or hardware and / or software specification, and output data having at least one network architecture belonging to the respective input data; -selecting (S3) item-specific training pairs on the basis of said library of models; and-training (S4) the LoRa network of the base model on the basis of the item-specific training pair for generating the item-specific network architecture.

Description

Technical Field

[0001] This invention relates to a method and apparatus for generating project-specific network architectures. Background Technology

[0002] Scaling is one of the biggest challenges in the (mass production) development of neural networks. In particular, the scaling laws governing neural networks come into play in this situation, and these laws are the subject of empirical research. Upscaling networks, as illustrated by current developments using large language models, appears relatively simple, while downscaling them to production-ready target hardware involves significantly more complex issues.

[0003] In recent years, significant progress has been made in scaling neural networks upwards. This scaling significantly improves generalization ability, enhancing the performance and applicability of machine learning models. This development is particularly illustrated by examples such as ChatGPT, which achieves high accuracy and flexibility in language processing due to its large size and complexity.

[0004] The limitations of scaling up are primarily defined by the availability and performance of training and inference hardware, as well as by the quantity and quality of available (training) data. As long as these resources are available to a sufficient degree, there seem to be few limitations on the growth and improvement of the model.

[0005] In comparison, scaling neural networks down to production-ready target hardware becomes significantly more complex. Different network architectures behave differently on different hardware, making hardware-agnostic development of such neural networks virtually impossible. A common solution is to co-develop the hardware and network architectures to ensure optimal coordination. Alternatively, a model library with different architectures can be created, and the most efficient architecture can be selected from this library based on available hardware.

[0006] The joint development of hardware and software is a costly but promising solution to maximize the efficiency and performance of neural networks. A modular approach—where multiple architectures are prepared for different hardware requirements—is also feasible. This strategy enables flexible matching and optimal utilization of existing hardware by selecting the architecture that operates most efficiently.

[0007] In view of the above, the objective of this invention is to describe a further improved method and / or a corresponding device.

[0008] This task is solved by the method according to the features of claim 1. This task is solved by the apparatus according to the features of claim 10. Summary of the Invention

[0009] Based on the first aspect, a method for generating project-specific network architectures is proposed. This method comprises the following steps:

[0010] - Provides a basic model with LoRa network adaptation, especially a large language model;

[0011] - Provide a model library with training data pairs, wherein each training data pair consists of input data and output data, the input data having model application and / or model and / or hardware and / or software specifications, and the output data having at least one network architecture belonging to the corresponding input data;

[0012] - Select project-specific training pairs based on the model library; and

[0013] - The LoRa network, trained on the base model using project-specific training pairs, is used to generate project-specific network architectures.

[0014] It should be understood that the steps and other optional steps according to the invention do not necessarily have to be performed in the order shown, but may be performed in other orders. Furthermore, additional intermediate steps may be provided. Moreover, each step may include one or more sub-steps without departing from the scope of the method according to the invention.

[0015] According to the second aspect, a device for generating project-specific network architectures is proposed. This device has evaluation and computing facilities configured to perform the following steps:

[0016] - Provides a basic model with LoRa network adaptation, especially a large language model;

[0017] - Provide a model library with training data pairs, wherein each training data pair consists of input data and output data, the input data having model application and / or model and / or hardware and / or software specifications, and the output data having at least one network architecture belonging to the corresponding input data;

[0018] - Select project-specific training pairs based on the model library; and

[0019] - The LoRa network, trained on the base model using project-specific training pairs, is used to generate project-specific network architectures.

[0020] The arguments made for this method apply accordingly to this device. It should be understood that linguistic variations characteristic of the method description can be restated for the device according to linguistic conventions, without needing to explicitly list such descriptions here.

[0021] This invention makes it possible to train a dynamic model library based on a base model. This base model can also be LoRa adapted. Therefore, this method enables the simpler and faster generation of network architectures with predetermined specifications for a given hardware. This is made possible by the adapted base model. Therefore, the adapted base model is preferably a generator for matching network models to predetermined hardware specifications.

[0022] This method offers several advantages. Firstly, it allows for the generation of (first) network architectures without lengthy development time. The network architectures generated in this way can then be used as a basis for further manual development and / or further network architecture searches. Secondly, it is possible to generate networks for previously unknown hardware by providing new hardware specifications. This method assists in making decisions when designing hardware on which neural networks with hardware-optimized architectures should be implemented. Furthermore, this method preferably enables automatic optimization of the model library using self-supervised learning. Moreover, with this method, particularly based on LoRa technology, it is possible to add new models and / or new model types and / or network architectures at low cost.

[0023] Particularly preferred is that the neural network is trained based on training data for a project-specific application, such as the classification and / or segmentation of objects in image data, the neural network having a resulting or further optimized project-specific network architecture, the training data being selected depending on the project or task to be performed by the neural network.

[0024] Therefore, to generate project-specific network architectures, a base model is established as a starting point for code generation. This base model can be, for example, a Large Language Model (LLM), such as ChatGPT. Furthermore, it is currently preferable to start with a library of models with known network architectures that possess their own data specifications. This utilizes the accumulated knowledge of Large Language Models (LLMs) regarding known and implemented neural network model descriptions. It is known why a neural network is used (task), on what hardware the neural network is run (hardware specification), and how good the results are (performance), etc. All this known problem-specific, input data-specific, quality-specific, and / or hardware-specific information and / or other information is preferably used as the corresponding input data for the base model during training. In addition to the known output data corresponding to the labeled output data, the base model is also provided with a known and often precisely described network architecture for training. For example, the output data can be provided via PyTorch code, which describes the network architecture.

[0025] In the initial training step of the base model, a training pair with the configuration (project-specific network description + additional standards, network architecture, if necessary) = (prompt, architecture) can be generated from the description of the elements in the model library. Then, LoRa training of the base model is performed based on this training pair. As a result, preferably, a neural network with the network architecture generated from the base model is obtained.

[0026] In another aspect, it is proposed that input data and / or output data be provided as text prompts and / or as statistical descriptions and / or as code descriptions in a programming language manner.

[0027] Since not all input data can be provided as input text or in text format, it is preferable to allow and add statistical and / or other descriptions of the input data as training input data. The input data may also contain information about the training data, on which network architectures were trained to be used as output data. Preferably, the original training data from the input data can also be used to train the base model.

[0028] In another aspect, it is proposed that the method further features: optimizing the resulting project-specific network architecture by means of prompt engineering, in particular by matching and / or expanding and / or curating the input data of the selected training pairs.

[0029] To obtain the best possible network architecture, it is preferable, especially after initial training, to determine, through "hint engineering," which will (retrospectively) be the best model or network description for the neural network with the resulting architecture. This is possible, for example, through gradient descent. Here, the gradients on the input data, rather than the trainable LoRa weights, are preferably defined. Then, given a fixed network architecture output, the best possible input (hint) can be determined from the base model.

[0030] In the case of prompt engineering, it is preferable to generate modified training pairs (prompt*, architecture) for multiple or all training data pairs (prompt, architecture). For this purpose, a network G* is preferably generated through which the optimal prompt can be found. Then, it is preferable to generate new, manually managed training pairs (prompt*, architecture) for multiple or all network architectures, wherein it is preferable to ensure that the architecture information is not included in the separately modified prompt*. Furthermore, additional information not present in the (initial) prompt can be added to the modified prompt*. The LoRa adaptation can then be retrained based on the separately modified training pairs. In this way, an optimized or managed network can be obtained.

[0031] In another aspect, the method further includes optimizing an already optimized project-specific network architecture through self-supervised learning, specifically by supplementing the input data of the selected training pairs with the required network performance criteria.

[0032] Subsequently, or after the prompting process, the network architecture can be further optimized in another step through self-supervised learning. Since the resulting metrics of the network architecture can preferably also be input into the prompts, or rather, into the input data, it is also possible to demand a better neural network. If the neural network is described at training time with, for example, a Mean Intersection over Union (mIOU) of 0.6 in the (initial) prompts, a higher mIOU can be demanded in the modified prompts when generating a new network with a new network architecture.

[0033] Preferably, a neural network with the obtained and engineered network architecture is trained based on training data, and then the trained neural network is evaluated to obtain results for the network under the target metric. Based on this, a continuously modified Prompt** can now be generated, in which the target metric a can be replaced, for example, by a better target metric a*. This can be done randomly by requiring a*>a or by specifying a maximum value. A modified network architecture can then be generated from the base model using the modified Prompt**. The modified network architecture can then be retrained and evaluated. Thus, a modified target metric a** can be obtained.

[0034] The network optimized in this way can then be retrained based on the new training data for the specific application of the project.

[0035] On the other hand, it is proposed that self-supervised learning be carried out continuously until network performance standards are met or other termination criteria are reached.

[0036] In another aspect, it is proposed that the input data contains information about hardware specifications, data specifications, task specifications, result specifications, training data used, and / or output format.

[0037] Additional information can also be included in the input data.

[0038] In another aspect, it is proposed to further optimize the resulting or optimized project-specific network architecture through neural architecture search.

[0039] In another aspect, protection is also claimed for a control device included in a vehicle and / or robotic system and / or industrial machine with autonomous driving capabilities, on which a neural network having a network architecture found according to one of the aspects of the method can be implemented and carried out. The control device can be implemented as an embedded system. The network architecture can be optimized based on the hardware specifications of the control device.

[0040] In another aspect, a computer program having program code is claimed for implementing at least a portion of the method in one of its aspects when the computer program is executed on a computer. In other words, a computer program (product) is proposed that includes instructions, which, when executed by a computer, cause the computer to implement the method / steps of the method in one of its aspects.

[0041] In another aspect, a computer-readable data carrier having program code of a computer program is proposed for implementing at least a portion of the method in one of its aspects when the computer program is executed on a computer. In other words, the present invention relates to a computer-readable (storage) medium comprising instructions that, when executed by a computer, cause the computer to implement the method / steps of the method in one of its aspects.

[0042] The described design and improvement schemes can be combined with each other arbitrarily.

[0043] Other possible designs, improvements, and implementations of the present invention include combinations of features of the invention not explicitly mentioned in the embodiments described above or below. Attached Figure Description

[0044] The accompanying drawings should facilitate a further understanding of the embodiments of the invention. These drawings illustrate the embodiments and, in conjunction with the description, are used to explain the principles and solutions of the invention.

[0045] Many advantages, among others, are derived from the accompanying drawings. The elements shown in the drawings are not necessarily depicted to scale.

[0046] Figure 1 A schematic flowchart illustrating an embodiment of the method; and

[0047] Figure 2 A schematic block diagram of the method in the case of reasoning is shown.

[0048] In the figures described herein, unless otherwise indicated, the same reference numerals denote the same or functionally identical elements, components or assemblies. Detailed Implementation

[0049] Figure 1 A schematic flowchart illustrating the method for generating a project-specific network architecture is shown.

[0050] In any embodiment, the method may be implemented at least in part by device 100, which may include a plurality of components not shown in more detail, such as one or more providing means and / or at least one evaluation and calculation means. It should be understood that the providing means may be constructed together with the evaluation and calculation means, or may be different from the evaluation and calculation means. Furthermore, device 100, which may be part of a system, may include storage means and / or output means and / or display means and / or input means.

[0051] The computer-implemented method includes at least the following steps:

[0052] In step S1, a basic model with LoRa network adaptation, particularly a large language model, is provided.

[0053] In step S2, a model library with training data pairs is provided, wherein each training data pair consists of input data and output data, the input data having model application and / or model and / or hardware and / or software specifications, and the output data having at least one network architecture belonging to the corresponding input data.

[0054] In step S3, project-specific training pairs are selected based on the model library.

[0055] In step S4, the LoRa network of the training base model is used to generate the project-specific network architecture based on the project-specific training pair.

[0056] Figure 2This illustrates the inference state of the currently trained base model 200 used to generate a project-specific network architecture. Here, based on input data 202, a neural network 204 with a preset network architecture 206 satisfying the input data 202 can be generated by the base model 200 with LoRa adaptation, the input data including, for example, information about hardware specifications, data specifications, task specifications, result specifications, training data used, and / or output format.

Claims

1. A method for generating a project-specific network architecture, the method comprising the steps of: - Provides (S1) a basic model with LoRa network adaptation, especially a large language model; - Provide (S2) a model library with training data pairs, wherein each training data pair consists of input data and output data, the input data having model application and / or model and / or hardware and / or software specifications, and the output data having at least one network architecture belonging to the corresponding input data; - Select project-specific training pairs based on the model library (S3); and - The LoRa network based on the project-specific training pair is used to train (S4) the base model to generate the project-specific network architecture.

2. The method of claim 1, wherein the input data and / or the output data are provided as text prompts and / or as statistical descriptions and / or as code descriptions in a programming language.

3. The method according to claim 1 or 2, wherein the method further comprises: The resulting project-specific network architecture is optimized by prompting engineering, particularly by matching and / or expanding and / or managing the input data of the selected training pairs.

4. The method of claim 3, wherein the method further comprises: Self-supervised learning can be used to optimize already optimized project-specific network architectures by supplementing the input data of the selected training pairs with the required network performance standards.

5. The method of claim 4, wherein the self-supervised learning is performed continuously until the network performance criterion is met or other termination criteria are reached.

6. The method according to any one of the preceding claims, wherein the input data has information regarding hardware specifications, data specifications, task specifications, result specifications, training data used, and / or output format.

7. The method according to any one of the preceding claims, wherein the resulting or optimized project-specific network architecture is further optimized by neural architecture search.

8. A computer program having program code for performing at least a portion of the method according to any one of claims 1 to 7 when the computer program is executed on a computer.

9. A computer-readable data carrier having program code of a computer program for performing at least a portion of the method according to any one of claims 1 to 7 when the computer program is executed on a computer.

10. An apparatus (100) for generating a project-specific network architecture, wherein the apparatus (100) has evaluation and computing means configured to perform the following steps: - Provides a basic model with LoRa network adaptation, especially a large language model; - Provide a model library with training data pairs, wherein each training data pair consists of input data and output data, the input data having model application and / or model and / or hardware and / or software specifications, and the output data having at least one network architecture belonging to the corresponding input data; - Select project-specific training pairs based on the model library; and - The LoRa network used to train the base model based on the project-specific training pair is used to generate the project-specific network architecture.

Images