Method and computer program product for transforming an electrical-electronic architecture from a visual graphics data format into a formal description language

Abstract

The invention relates to a computer-implemented method for transforming a representation of an electrical / electronic architecture (E / E architecture) of a vehicle, comprising at least one electronic component, provided in a machine-readable visual graphic data format (10), into a machine-readable representation according to a predetermined formal description language (FDL). The representation provided in the visual graphic data format (10) comprises at least one text element and / or one graphic element.A context is extracted from the visual graphic data format (10) by extracting text from at least one text element using a computer-implemented character recognition method. This text at least partially specifies at least one electronic component, and / or a computer-implemented shape recognition method assigns at least one graphic element to an electronic component and / or a connection between at least two electronic components. The context is analyzed, and a structure of the arrangement of electronic components in an E / E architecture is identified. From this structure, the FDL-compliant representation is generated using a computer-implemented generative artificial intelligence method and provided in a machine-readable FDL data format (20). The invention further relates to a computer program product for carrying out the method.

Description

[0001] The invention relates to a method for transforming a representation of a vehicle's electrical-electronic architecture, provided in a machine-readable visual graphics data format, into a machine-readable representation according to a predetermined formal description language. The invention further relates to a computer program product for carrying out such a method.

[0002] An electrical / electronic architecture (hereinafter referred to as E / E architecture) comprises at least one electronic component, typically multiple electronic components. Such electronic components can be electronic control units (ECUs), but also other types, such as analog-to-digital converters, digital-to-analog converters, electronic storage devices, or bus controllers. Typically, an E / E architecture includes at least one ECU.

[0003] E / E architectures for vehicles are created at least partially using generic graphically oriented drawing tools according to the state of the art and are available at least partially in a graphically oriented output format.

[0004] For example, especially in the early stages of architectural design, electronic circuits, overview block diagrams, diagrams in a UML (Unified Modeling Language) format, and similar design descriptions are provided as vector or pixel graphics data in standardized data formats such as Portable Network Graphics (PNG), Portable Document Format (PDF), and Scalable Vector Graphics (SVG). Furthermore, the use of drawing programs for creating diagrams that represent at least one aspect of an E / E architecture and provide a proprietary, graphically oriented output format is widespread. Examples of such drawing programs include Microsoft Visio, Microsoft PowerPoint, Sketch, diagrams.net, and Lucidchart.

[0005] Such graphically oriented output formats, which may contain text elements (e.g., annotations, labels, comment fields, or similar), vector-oriented drawing elements (circles, boxes, diamonds, connecting lines and arrows, or similar), and / or purely graphic elements (images, icons, symbols, or similar), are referred to below as visual graphic data formats. Vector-oriented drawing elements and purely graphic elements are collectively referred to below as graphic elements.

[0006] Results stored in a visual graphics data format often require considerable manual effort to transform into a machine-readable form that can be further processed by specialized design tools developed for automotive system design. Typically, such design tools, like Vector's PREEvision product, provide an interface for machine-readable data that conforms to the syntax and semantics of a formal description language (FDL).

[0007] Therefore, there is a need for a method and system for transforming visual graphic data formats into a representation that meets the requirements of an FDL.

[0008] Document EP 1 457 917 B1 describes a device and a method for converting network drawings from a raster format to a vector format. In a raster image comprising a network drawing, text is recognized using optical character recognition (OCR) and a character set belonging to the raster image. The recognized text is extracted to generate a text-only raster image and a text-free raster image. Graphic objects contained in the text-free raster image are recognized using pattern recognition and parameters specific to the raster image. Vector graphic elements corresponding to the recognized graphic objects are used to generate a text-free vector image. Vector text elements corresponding to the extracted text are added to the text-free vector image.Detecting graphic objects within a raster image without text involves creating a list of two-dimensional pixel gradients, where a pixel gradient is a group of pixels that are horizontally adjacent and share a common y-coordinate. Pixel gradients with identical x-coordinates and adjacent y-coordinates are merged to create the two-dimensional pixel gradient. Lines are then identified from this list of two-dimensional pixel gradients, and the two-dimensional pixel gradients identified as lines are removed. Large binary objects are identified from the list. Graphic objects are then identified from these large binary objects by matching each binary object with its corresponding vector graphic object.

[0009] The invention is based on the objective of providing an improved method for transforming a vehicle's E / E architecture from a representation present in a machine-readable visual graphics data format into a machine-readable representation according to a predetermined formal description language (FDL). This objective is achieved according to the invention with a method having the features of claim 1.

[0010] Furthermore, the invention is based on the objective of providing a computer program product for carrying out such a method. This objective is achieved according to the invention with a computer program product having the features of claim 5.

[0011] Advantageous embodiments of the invention are the subject of the dependent claims.

[0012] According to a first aspect of the invention, a computer-implemented method for transforming a representation of the E / E architecture of a vehicle, comprising at least one electronic component, provided in a machine-readable visual graphics data format, into a machine-readable representation according to a predetermined FDL involves reading the representation in the machine-readable visual graphics data format. The representation in the machine-readable visual graphics data format comprises at least one text element and / or one graphic element.

[0013] The representation in the machine-readable visual graphics data format can be provided and read as a file in a proprietary format, which depends on the drawing tool used. The representation can also be provided and read in a machine-readable data format with generic syntax, such as JSON (JavaScript Object Notation) or XML (Extensible Markup Language). Alternatively, instead of reading a file, the representation can also be read programmatically via an application programming interface (API) provided by the respective drawing tool.

[0014] From the imported representation in the machine-readable visual graphics data format, a context is extracted that at least partially describes the E / E architecture depicted therein. For this purpose, a text is extracted from at least one text element using a computer-implemented optical character recognition (OCR) method, and this text specifies at least one electronic component, at least partially.

[0015] Examples of computer programs for OCR include Tesseract, Google Cloud Vision OCR, AWS Textract, and similar software libraries. OCR extracts annotations, comments, or similar descriptions from visual graphic data formats, identifying an ECU or other electronic component of the E / E architecture (e.g., by name) and / or capturing its parameters.

[0016] Additionally or alternatively, a computer-implemented method of graphic shape recognition, also known as shape recognition, is used to assign at least one graphic element to an electronic component and / or a connection between at least two electronic components.

[0017] Examples of computer programs for shape recognition include software libraries such as OpenCV. Shape recognition can also be performed or supported by deep learning methods, such as the prior art known as YOLO (J. Redmon, S. Divvala, R. Girshick, A. Farhadi: You Only Look One: Unified, Real-Time Object Detection, https: / / doi.org / 10.48550 / arXiv.1506.02640).

[0018] Graphic shape recognition is used to identify both vector-oriented drawing elements (such as drawn rectangular boxes, drawn connecting lines between boxes, drawn circles, and the like) and graphic elements (such as symbols, icons, and the like) in terms of their type, size, position, and / or orientation. For example, in an E / E architecture drawn in the style of a block diagram, rectangular boxes are each assigned to an ECU. Lines between rectangular boxes are assigned electrical and / or signal connections between ECUs. In this way, the purely visually interpreted representation of the visual graphic format is translated into a set of context objects, which have a semantic relationship to an E / E architecture.

[0019] In a subsequent analysis step, at least one structure of the arrangement of electronic components is recognized in the context using a computer-implemented pattern recognition method and assigned to the context.

[0020] Such a pattern recognition method can be implemented, for example, using a convolutional neural network (CNN) that has been trained in a prior learning step using a multitude of typical E / E architectures. Each of these E / E architectures is characterized by a spatial arrangement of its electronic components and a layout of the connections between these electronic components. By comparing the training data with the context objects of the analyzed context (each assigned to an electronic component or a connection between electronic components), a context-appropriate E / E architecture is identified and adjusted as needed.

[0021] Additionally or alternatively, a computer-implemented method is used to determine at least one relationship between at least two electronic components from the extracted text and / or graphic element. Such methods can be implemented, for example, as natural language processing (NLP) techniques using computer programs like TensorFlow or PyTorch, or similar software. These methods extract contextual information from text elements and graphic elements labeled with text. From this, dependencies and connections between electronic components are determined (for example, between ECUs, each represented as a box and connected by lines).

[0022] From the identified and assigned structure of the arrangement of the at least one electronic component and / or from the at least one relationship between at least two electronic components, the FDL-compliant representation is generated using a computer-implemented generative artificial intelligence method. Such methods are known and available, for example, as generative pre-trained transformers (GPTs) based on large language models (LLMs).

[0023] Using such a generative method, the graphic and text elements recognized as context objects, as well as the parameters and boundary conditions extracted from them, are transformed into a formal language that meets the syntax requirements of the predetermined FDL.

[0024] In a subsequent step, a computer-implemented validation procedure is used to validate the resulting FDL-compliant representation for completeness, consistency, and adherence to the FDL syntax. This ensures that only correct (well-formed) FDL-compliant representations are generated that are conflict-free and complete. Computer-implemented validation procedures, formal verification methods, and software tools for their execution are known from the prior art.

[0025] The FDL-compliant representation is provided in a machine-readable FDL data format, for example as a file or as a data record that is stored in a database or transferred to another computer-implemented process.

[0026] The method according to the invention enables a particularly efficient automated transformation of a visual graphics data format into a machine-readable data format, which can subsequently be read by design tools specializing in the design of E / E architectures. This saves time in the development of E / E architectures. The manual effort required to transfer design results from one or more drawing programs into a specialized design tool for E / E architectures can be reduced.

[0027] Furthermore, the accuracy of transferring intermediate results generated with generic drawing programs into a formally structured description according to an FDL suitable for a specialized design tool is improved. Ease of use can be enhanced by allowing development engineers to connect to specialized development tools while maintaining their familiar toolchain. In particular, this enables the rapid and efficient creation of preliminary designs and discussions at an early stage of development using lightweight, easy-to-learn drawing tools.

[0028] In one embodiment of the invention, the FDL-compliant representation of the E / E architecture is provided in a user interaction tool, which is configured for displaying and / or interactively customizing the FDL-compliant representation for / by a user. Preferably, the user interaction tool is configured as a design tool for the interactive design of E / E architectures in the automotive sector. Such design tools, for example, Vector's PREEvision product, typically have data interfaces and / or APIs through which an FDL-compliant representation can be provided.

[0029] This implementation enables the seamless and lossless integration of preliminary results developed and / or documented using generic drawing programs. Generic drawing programs are easy to use and accessible to a wide range of people. By incorporating these preliminary results, which are in a visual graphics data format, into a subsequent development workflow—typically based on specialized design tools for modeling and simulating E / E architectures—a standardized, integrated, and seamless workflow can be ensured across various disciplines.

[0030] In one embodiment of the method, at least one graphic element and / or text element is assigned to an electronic component and / or to a connection between at least two electronic components based on its color representation in the visual graphic data format. This embodiment enables the particularly easy and reliable assignment of text and / or graphic elements based on visually easily distinguishable and therefore particularly widespread features.

[0031] In one embodiment of the method, at least one graphic element and / or text element is assigned to an electronic component and / or to a connection between at least two electronic components based on its position, size, and / or orientation in the visual graphic data format. This embodiment allows the use of visually easily distinguishable graphic elements (for example: rectangular box – ECU, rectangular box with rounded corners – component with subcomponents) that can be assigned to context objects corresponding to an E / E architecture particularly easily and reliably. Similarly, an assignment can also be made based on the position, size, or orientation of a graphic or text element.

[0032] According to a second aspect of the invention, a computer program product comprises an input interface, a context extraction module, a context analysis module, a translation module, a syntax generation module, and an output interface.

[0033] The input interface is set up to read in a representation of an E / E architecture in a visual graphics data format.

[0034] The context extraction module is designed to extract, using a computer-implemented character recognition method, a text from at least one text element that at least partially specifies at least one electronic component, and / or to assign, using a computer-implemented shape recognition method, at least one graphic element to an electronic component and / or a connection between at least two electronic components.

[0035] The context analysis module is designed to use a computer-implemented pattern recognition method to recognize at least one known pattern of the arrangement of electronic components and assign it to a context, and / or to use a computer-implemented method to determine at least one relationship between at least two electronic components from the at least one extracted text and / or graphic element.

[0036] The translation module is designed to generate an FDL-compliant representation from a pattern of the arrangement of at least one electronic component and / or from the at least one relationship between at least two electronic components using a computer-implemented generative artificial intelligence method, and to validate it for completeness, consistency, and conformity with the syntax of the FDL using a computer-implemented validation procedure.

[0037] The output interface is set up to provide the FDL-compliant representation in a machine-readable data format.

[0038] The advantages of the computer program product correspond to the advantages of the computer-implemented method according to the first aspect of the invention.

[0039] In one embodiment, the output interface for data coupling is set up with a data input interface of a design tool for the computer-aided design of E / E architectures of a vehicle.

[0040] This implementation enables the seamless and lossless integration of preliminary results developed and / or documented using generic drawing programs. Generic drawing programs are easy to use and accessible to a wide range of people. By incorporating these preliminary results, which are in a visual graphics data format, into a subsequent development workflow—typically based on specialized design tools for modeling and simulating E / E architectures—a standardized, integrated, and seamless workflow can be ensured across various disciplines.

[0041] Exemplary embodiments of the invention are explained in more detail below with reference to a drawing.

[0042] This shows: Fig. 1. A flowchart for the process of transforming a visual graphics data format into a machine-readable FDL-compliant data format.

[0043] Fig. Figure 1 shows a schematic flowchart with steps 1 to 5, arranged between a start point S and an end point E, and executed sequentially. In the first step, step 1, a representation of an E / E architecture is read in as a machine-readable visual data format 10, for example, as a file or as a data stream. The visual data format 10 can be read in as a file created by a drawing tool. Alternatively, the visual data format 10 can be provided as a data stream captured via an API of the respective drawing tool.

[0044] The visual graphic data format 10 comprises text and graphic elements, not shown in detail here, for the visual representation of an E / E architecture. For example, the visual graphic data format 10 includes rectangular boxes, each representing an ECU, and lines arranged between them, representing electrical and / or data connections between ECUs.

[0045] Graphical elements can be visually distinguished based on their shape, size, position, and / or orientation, and thus assigned to different electronic components of the E / E architecture. For example, rectangular boxes can each represent an ECU, rectangular boxes with rounded corners an electronic component with multiple subcomponents, lines an electrical or data line, and filled circles of a certain size an electrical connection between electrical lines.

[0046] Similarly, different assignments in the visual representation (and thus in the visual graphic data format) can also be encoded by different colors. For example, blue lines can represent a connection via a CAN (Controller Area Network) bus, while red lines can represent a connection via a LIN (Local Interconnect Network) bus. Likewise, and analogously, text elements can be assigned to electronic components based on their size, color, position, orientation, font, and / or other characteristics.

[0047] Optionally, the first step 1 can be in Fig. One unspecified sub-step involves preprocessing the visual graphics data format. Such preprocessing might include, for example, cropping one or more visible areas, enhancing contrast, or transforming grayscale or color values.

[0048] In the second step, a context with context objects is extracted. For example, text is extracted from text elements using an OCR tool such as Tesseract, Google Cloud Vision, or AWS Textract. Shape and color characteristics of graphic elements are extracted using image processing methods and tools, such as OpenCV or YOLO. The context objects (each associated with at least one text element and / or graphic element) are determined in such a way that they can be assigned to elements of the E / E architecture (an electronic component, such as an ECU, or a connection between electronic components, or parameters of such an electronic component or connection).

[0049] In the third step, the context is analyzed. Using artificial intelligence pattern recognition methods, such as CNNs or rule-based inference, and / or context understanding methods (e.g., NLP techniques and / or tools like TensorFlow or PyTorch), a structure for the arrangement of elements within an E / E architecture is derived from the set of extracted context objects (taking into account their arrangement topology, size, orientation, and other characteristics). Furthermore, parameters (e.g., supply voltage, operating temperature range, voltage level, bus line bit depth, or similar) of electronic components or connections are determined and assigned.

[0050] In the fourth step, an FDL-compliant representation of the E / E architecture is formed from the structure thus obtained using a computer-implemented generative method of artificial intelligence, for example with a GPT method based on an LLM and / or a rule-based inference method.

[0051] In the fifth step, the FDL-compliant representation is converted into a machine-readable FDL data format 20, which can be provided as a file or data stream. The FDL data format 20 can be accessed, for example, by an interactive design tool that is set up and specialized for the detailed design of E / E architectures for vehicles (by opening a file or via an API). Reference symbol list 1 to 5 first to fifth step 10 Visual graphics data format 20 Formal Description Language (FDL) - Data Format E Endpoint S Starting point QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] EP 1 457 917 B1

[0008] Cited non-patent literature

[0000] J. Redmon, S. Divvala, R. Girshick, A. Farhadi: You Only Look One: Unified, Real-Time Object Detection, https: / / doi.org / 10.48550 / arXiv.1506.02640

[0017]

Claims

[1] Computer-implemented method for transforming a representation of an electrical-electronic architecture (E / E architecture) of a vehicle comprising at least one electronic component, provided in a machine-readable visual graphic data format (10), into a machine-readable representation according to a predetermined formal description language (FDL), wherein the representation provided in the visual graphic data format (10) comprises at least one text element and / or one graphic element, characterized by , that - the representation is read into the machine-readable visual graphics data format (10), - a context is extracted, whereby - using a computer-implemented character recognition method, a text is extracted from at least one text element, which specifies at least one electronic component at least partially, and / or - using a computer-implemented shape recognition method, at least one graphic element is assigned to an electronic component and / or a connection between at least two electronic components, - the context is analyzed, whereby - using a computer-implemented pattern recognition method, at least one structure of the arrangement of electronic components of an E / E architecture is recognized and assigned to the context and / or - using a computer-implemented method from which at least one extracted text and / or graphic element is extracted, at least one relationship between at least two electronic components is determined, - from the structure of the arrangement of the at least one electronic component and / or from the at least one relationship between at least two electronic components, the FDL-compliant representation is formed using a computer-implemented generative method of artificial intelligence and validated for completeness, consistency and conformity with the syntax of the FDL using a computer-implemented validation procedure and - the FDL-compliant representation is provided in a machine-readable FDL data format (20). [2] Method according to claim 1, characterized by , that the FDL-compliant representation is provided in a user interaction tool that is set up to display and / or interactively customize the FDL-compliant representation for / by a user. [3] Method according to any one of the preceding claims, characterized by, that at least one graphic element and / or text element is assigned to an electronic component and / or to a connection between at least two electronic components based on its color representation in the visual graphic data format. [4] Method according to any one of the preceding claims, characterized by , that at least one graphic element and / or text element is assigned to an electronic component and / or to a connection between at least two electronic components based on its position and / or size and / or orientation in the visual graphic data format. [5] Computer program product comprising an input interface, a context extraction module, a context analysis module, a translation module, a syntax generation module and an output interface, characterized by , that - the input interface for reading in a representation of an E / E architecture in a visual graphics data format (10) is set up, - the context extraction module is set up for this purpose, - to extract, using a computer-implemented character recognition method, from at least one text element a text that at least partially specifies at least one electronic component, and / or - to assign at least one graphic element to an electronic component and / or a connection between at least two electronic components using a computer-implemented shape recognition method, - the context analysis module is set up for this purpose, - to recognize at least one known pattern of the arrangement of electronic components using a computer-implemented pattern recognition method and to assign it to a context and / or - to determine, using a computer-implemented method, at least one relationship between at least two electronic components from the at least one extracted text and / or graphic element, - the translation module is designed to create an FDL-compliant representation from a pattern of the arrangement of at least one electronic component and / or from the at least one relationship between at least two electronic components using a computer-implemented generative artificial intelligence method, and to validate its completeness, consistency, and conformity with the syntax of the FDL using a computer-implemented validation method and - the output interface is set up to provide the FDL-compliant representation in a machine-readable FDL data format (20). [6] Computer program product according to claim 5, characterized by, that the output interface for data coupling with a data input interface of a design tool for the computer-aided design of E / E architectures of a vehicle is set up.

Images