Method and system for determining the object structure of an object, as well as a control device for such a system

Abstract

The invention relates to a method for determining the object structure of an object. By using at least one neural network, edge endpoints (7), intermediate edge points (5), and segments (9) are determined, wherein edge endpoints (7) mark a predetermined end region of the object, intermediate edge points (5) mark line geometric shapes of the object, and segments (9) each positionally represent a partial segment position in an instance (14, 15, 17) of the object structure. The determined object structure is then provided for a computer vision function, in particular for a lane detection system. Subsequently, longitudinal and / or lateral guidance of a motor vehicle is enabled by means of a control device comprising this computer vision function.

Description

[0001] The invention relates to a method for determining an object structure of an object, comprising the steps of: providing image data that describe an image of an environment with the object located therein and that was received from at least one sensor device; and feeding the image data into at least one neural network that is trained to determine feature data, wherein the feature data comprise predetermined features (characteristics) with respect to basic geometric shapes (reference shapes) and / or colors of the object.

[0002] Object detection is a subfield of image processing that focuses on identifying individual objects in images. The term "object" refers, for example, to traffic islands and / or road markings and / or lane lines and / or another vehicle when it comes to object detection for driver assistance systems. However, conventional methods, such as the use of bounding boxes or image recognition, have limitations because they can only detect objects of a certain, fixed shape as a whole. Providing the correct assignment and / or identification and / or ordering of pixels or pixel groups, especially when detecting connected lines, therefore represents a challenging task.

[0003] To ensure an advantageous design of such object recognition systems, it is therefore desirable to determine an object, an object structure, or a centerline of an object of any shape. "Object structure" here refers to the basic shape (e.g., a straight line segment) and / or the composition of the object's optical appearance from such basic shapes or characteristics, e.g., a polyline shape.

[0004] An object detection system specifically designed for lane detection typically only considers highway scenarios where lane markings and road edges can be described by straight lines. However, such an object detection system fails in urban scenarios, for example, where objects such as road edges and / or traffic islands can have any geometric shape, such as the aforementioned polyline shape. A polyline shape, or polyline, refers to an open or closed sequence of connected lines and / or arc segments and / or line segments that, taken together, no longer form a single straight line.

[0005] The paper entitled “Spatial Ray Features for Real-Time Ego-Lane Extraction,” written by Tobias Kühnl, Franz Kummert, and Jannik Fritsch and published in 2013, discloses a method for extracting lanes from bird’s-eye view images using spatial ray features within a bottom-up approach.

[0006] The publication entitled “Rethinking efficient lane detection via curve modeling”, authored by FENG, Zhengyang [et al.], discloses a method for lane detection based on parametric Bézier curves.

[0007] Also well known is the Wikipedia article “Attention (machine learning)” from October 13, 2023, available at: https: / / en.wikipedia.org / w / index.php?title=Attention_(machine_learning)&oldid=11798 79370 [accessed on December 5, 2024]. It reveals the functionality, significance, and applications of attention mechanisms in machine learning.

[0008] The Wikipedia article “Small object detection” from August 9, 2023, available at: https: / / en.wikipedia.org / w / index.php?title=Small_object_detection&oldid=116941971 4 [accessed on December 5, 2024], discusses methods and challenges in detecting small objects in computer vision applications.

[0009] The Wikipedia article "Backpropagation" from September 9, 2023, available at: https: / / en.wikipedia.org / w / index.php?title=Backpropagation&oldid=1174621109 [accessed on December 5, 2024], reveals the basic principles and applications of the backpropagation algorithm in the optimization of neural networks.

[0010] The invention is based on the object of recognizing an object structure of an object in an environment, in particular a road traffic environment, on the basis of image data.

[0011] The problem is solved by the subject matter of the independent patent claims. Advantageous developments of the invention are described by the dependent patent claims, the following description, and the figures.

[0012] The invention provides a method for determining an object structure of an object. The method can be used, for example, as part of a computer vision functionality to recognize an object in a motor vehicle's surroundings with regard to its object structure. Based on the recognized object structure, a driving trajectory for avoiding the object (e.g. a traffic island) or for aligning the motor vehicle with regard to the object (e.g. a lane marking) can be calculated. The method comprises the following steps: Image data describing an image of an environment with the object located therein is received or provided from at least one sensor device.The image data is fed into at least one artificial neural network trained to determine feature data. The feature data comprises predefined features or characteristics with respect to basic geometric shapes (reference shapes) and / or colors of the object. Such basic shapes can be, for example, horizontal and / or vertical lines and / or arcs, to name just a few examples.

[0013] The feature data can be determined using a so-called backbone, wherein the backbone is designed as at least one artificial neural network, wherein the backbone can, for example, comprise at least one residual neural network and / or densely connected convolutional network pre-trained on predetermined features with regard to basic geometric shapes and / or colors of the object. The at least one neural network can additionally or alternatively be trained on data describing polygonal shapes and / or structures. The feature data can, for example, be comprised or provided as a feature vector. In other words, the at least one neural network can be designed to receive image data or visual data from at least one sensor device and to determine and / or extract feature data therefrom.The at least one neural network can comprise a convolutional neural network (CNN) and / or a graph neural network (GNN). Additionally or alternatively, the at least one neural network can be configured as an encoder-decoder network. Overall, the at least one neural network can therefore perform semantic segmentation.

[0014] A so-called feature map can be generated from the at least one neural network. The at least one neural network can comprise a plurality of convolutional layers, whereby the feature map comprises a receptive field (e.g., the detection of 3 x 3 to 200 x 200 image points or pixels) that can, for example, detect objects such as traffic islands and / or road edges and / or lane markings as a whole.

[0015] In at least one neural network, three different image processing functionalities can now be connected downstream, which can independently process the features from the feature map on a pixel-by-pixel basis.

[0016] The at least one neural network comprises, as a first image processing functionality, a so-called edge endpoint detector, which is designed to determine and / or mark edge endpoints or endpoints of the object structure of the object in a step a). The edge endpoints each mark a predetermined object end region (in particular a line end) of the object. The feature data are subjected to a binary classification by means of the edge endpoint detector, wherein the binary classification can include that feature vectors of the feature data are normalized and a normalized value, i.e. a classification value, is representative of having or representing an edge endpoint of an object structure above a predetermined threshold value. An edge endpoint can be recognized by the fact that an image region is identified in the image data as a component of a line shape (e.g.a curb) is detected and, starting from this edge endpoint, a remainder of the line leads away in only one direction (instead of in two different directions). For example, an edge endpoint can be detected and / or set at an expiring lane line and / or lane marking. In other words, for each pixel of an image, it can be checked whether an edge endpoint is present and / or represented.

[0017] Using the edge endpoint detector, a so-called seed point map can be generated, whereby the seed point map has or indicates marked edge endpoints or so-called seed points. For example, if the image data shows a white lane arrow on a gray road surface, the two ends of the lane arrow (tip and opposite end) can be identified as edge endpoints. The edge endpoint detector can be trained to identify shapes with a size ranging from 20x20cm to 75x75cm and that end in one spatial direction or do not extend any further in this direction. In this case, the object ends in this direction, and an edge endpoint can be set there. Using so-called seed point detection, one-dimensional points (edge ​​endpoints or seed points) are detected.Therefore, start and / or end points can be learned, which can be recognized as edge end points.

[0018] The at least one neural network comprises, as a second image processing functionality, an edge intermediate point detector, which is configured to determine and / or mark, in a step b), edge intermediate points and / or key points and / or connection points of the object structure of the object to the feature data. The edge intermediate points mark linear geometric shapes of the object, wherein a linear geometric shape is a part of the object structure of the object. The linear geometric shapes can be formed from straight lines and / or arcs and / or a mixture of both and / or polygonal surfaces.

[0019] The feature data are subjected to a further binary classification, different from the first, by means of the edge intermediate point determiner, wherein this second binary classification may comprise that feature vectors of the feature data are normalized and a normalized value, i.e. a classification value, is representative of having line-geometric shapes of an object structure above a predetermined threshold value.

[0020] The feature vectors are therefore preferably each subjected to a binary classification, whereby the feature vectors can comprise parameterizable (numerical) properties of a geometric shape and / or a pattern of the object in a vectorial manner. Different features characteristic of the pattern can form different dimensions of the feature vectors. Using the feature vectors, the subsequent binary classification can be facilitated, as they greatly reduce the number of properties to be classified (for example, instead of an entire image, only a feature vector consisting of N numbers needs to be considered, where the number N is smaller than the number of pixels in the image).

[0021] In the above example of the lane arrow, for example, individual line segments or line sections of the boundary line formed by the transition between the white lane arrow and the gray road surface can each be identified as an edge intermediate point. The edge intermediate point detector can be trained to identify sections with a length in a range of 20 cm to 75 cm that belong to a continuous line running in two directions in the surrounding area as an edge intermediate point.

[0022] Additionally or alternatively, it can be provided that by means of non-maximum suppression (NMS) only edge end points and / or edge intermediate points are determined and / or imaged which have an intensity or brightness value above a predetermined threshold value.

[0023] According to the invention, the third image processing functionality, in a step c), includes determining route shares of individual route points of the object structure by applying a regression criterion or regression network of the at least one neural network to the feature data. Route shares each represent a proportional route position in an instance of the object structure. An instance can mean a continuous partial object. If, for example, the object structure is characterized or represented by several lane lines, one of these lane lines can be described as an instance. A route point that is located exactly in the middle of this instance, for example, can accordingly contain a route share of 50% measured in percentage points. A route point in the lower third of the instance can therefore have, for example, 33%.In other words, the at least one neural network can estimate and / or map an arc length of at least one instance of the object structure by means of the regression network, which is designed, for example, as a multi-layer perceptron (MLP) and / or as a convolutional neural network.

[0024] To train the at least one neural network for this image processing functionality, training data comprising, for example, (labeled) images of lines and / or line segments of different lengths and / or shapes can be used. One or more points can be marked on each line, whereby the label must be known in advance as the percentage of an arc length of this line that reaches a respective marked point. In particular, a convolutional neural network can be used for this purpose, which has linear activation since a continuous value is predicted. The at least one neural network can thus have a pixel-based regression model. A regression loss function, such as mean squared error (MSE), can be used for this purpose. Additionally or alternatively, the Adam optimizer, for example, can be used.

[0025] These three image processing functionalities therefore determine whether a pixel in the feature map has or represents an edge endpoint and / or an edge intermediate point and / or a specific route portion. The three aforementioned image processing functionalities can be performed independently of one another, so that the aforementioned corresponding information or estimates are obtained independently of one another. In a step d), the object structure is determined using the edge endpoints and / or the edge intermediate points and / or the route portions. The object structure is then made available for a computer vision function, wherein longitudinal and / or lateral guidance of a motor vehicle is carried out by means of a control device having this computer vision function, for which purpose the aforementioned driving trajectory can be used.

[0026] Overall, the goal is to perform semantic segmentation, whereby edge endpoints, intermediate edge points, and / or segments can be quantified as segments. A segmented image depicting these segments can thus be generated as the output of the at least one neural network.

[0027] The computer vision function can be incorporated, for example, in a motor vehicle as a parking aid and / or reversing system and / or lane detection system and / or lane change assistance system. Furthermore, the invention can be used for autonomous and / or semi-autonomous driving, i.e., levels 1 to 2 and / or 3, by performing longitudinal and lateral guidance of the motor vehicle using the determined object structure. The invention offers the advantage that the object structure to be determined can comprise any geometric shape.

[0028] The invention provides that step d) includes the following steps: In a step a.1), edge end points and / or edge intermediate points are targeted by means of a filter area by applying a filter, wherein this filter area delimits at least one object end area of ​​at least one instance of the object structure at the beginning of filtering. This filter can in particular be a so-called "window-based" filter that systematically slides over the output of the at least one neural network or an intermediate layer thereof, i.e. a processed image. The object end area refers to an area or a location at which at least one instance of an object structure ends, for example a line end if the instance represents a lane line. In a step b.1), distance portions of the targeted edge end points and / or edge intermediate points are used to determine an extension direction.The expansion direction refers to the direction in which at least one instance expands and / or extends. The filter area can have a surface area of ​​the overall image in a range of 2% to 20%. It can be rectangular or square, for example.

[0029] In step c.1), a raycasting method is applied to the instance so that individual instances of the object structure are marked as such and / or separated from one another and / or sub-areas of the instances are assigned to the corresponding instances. More precisely, the raycasting method involves firing a ray from an edge endpoint or edge intermediate point in different directions within an instance delimited by the filter area, so that further edge endpoints and / or edge intermediate points in this delimited instance are visible or determined. This makes it possible to determine a further edge endpoint and / or edge intermediate point that has the next largest or next smallest distance component on the same instance according to the raycasting method.

[0030] The filter is thus centrally aligned on an (initial) edge endpoint or edge intermediate point and then, in a step d.1), connects this edge endpoint and / or edge intermediate point with another edge endpoint and / or edge intermediate point within this filter area that has the next largest or next smallest path component on the same instance according to the raycasting method. In other words, the filter can follow the path of an instance.

[0031] For example, if an edge endpoint contains a distance component of 0% to 1%, measured in percentage points, the filter can search for an edge endpoint and / or edge intermediate point with a distance component of, for example, 2%, and connect them together and / or mark them as belonging to a connection. Applying the raycasting method prevents an edge endpoint and / or edge intermediate point located on one instance from being connected to another edge endpoint and / or edge intermediate point located on a different instance.

[0032] In step e.1), the filter is shifted along the instance by a specified length, resulting in a new filter region. This length can, for example, include a shift of one to 30 pixels. In step f.1), edge endpoints and / or intermediate edge points in this new filter region are again targeted.

[0033] In a step g.1), steps b.1) to f.1) are repeated until a predetermined threshold value with respect to a number of interconnected edge endpoints and / or edge intermediate points and / or a predetermined maximum value of a route share and / or a total number of instances is reached and / or no further instances are present and / or no further edge endpoints and / or edge intermediate points to be connected are present in the filter area.

[0034] The filtering and / or the application of the raycasting method can take place within post-processing, meaning that the window-based filtering is no longer necessarily part of the at least one neural network. So-called post-processing is performed, whereby the post-processing automatically processes the outputs or predictions of the at least one neural network.

[0035] The invention also includes further developments which result in additional advantages.

[0036] A further development provides that the predetermined maximum value of a distance component lies in a range of 70 to 90 percentage points, or preferably in a range of 80 to 90 percentage points. Thus, not all edge endpoints and / or edge intermediate points of at least one instance need to be connected to each other for that instance to be mapped. This can save computing power and / or memory.

[0037] A further development provides that one instance represents a line of a lane marking. Multiple instances can therefore represent multiple lines, thus representing an entire lane marking.

[0038] The edge-intermediate point detector and the regression network are designed to estimate an object structure as a whole, with meaningful and / or high-quality results being achieved, in particular, by combining all three image processing functionalities and applying post-processing. "Meaningful" and / or "high-quality" means that the (ground truth) object structure can be accurately and / or clearly and / or expressively and / or sharply represented.

[0039] A further development provides for the feature data to be determined using a self-attention technique. Using the self-attention technique, the computational complexity of the at least one neural network used to determine the feature data can be reduced.

[0040] A further development proposes that the feature data be determined using a feature pyramid network technique. Using the feature pyramid network technique, low-level (local) and high-level (semantic) features can be more closely linked.

[0041] An advantageous embodiment provides that the determination of the object structure of the object is trained by means of error feedback (backpropagation), wherein training data is used that has a label of the object structure. As a label of a line or polyline, for example, it can be provided that a line position and / or line thickness and / or line length and / or line orientation and / or line color and / or line type are known. Additionally or alternatively, as already described, one or more points can be marked on a line, wherein the label previously known is what percentage of the arc length of this line has been reached up to a respective marked point.

[0042] The at least one neural network is therefore trained using supervised learning. During backpropagation, an input, such as image data, which is transformed into input vectors, is propagated through the at least one neural network. The resulting output of the at least one neural network is compared with the desired output. The difference between the two values ​​is considered the error of the neural network, and the error is then propagated back via the output layer to the input layer. In the process, the weights of the neuron connections of the at least one neural network are changed depending on their influence on the error. This guarantees an approximation of the desired output when the input is applied again. Using backpropagation, the at least one neural network can be corrected. The parameters of the at least one neural network can thus be optimized or improved.With the thus improved parameters of the at least one neural network, the at least one neural network is suitable in the application phase to determine meaningful output vectors (outputs) from input vectors (inputs) that deviate from the originally learned input vectors of the training cases.

[0043] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0044] The invention also includes the control device. The control device can have a data processing device or a processor device that is configured to carry out an embodiment of the method according to the invention. For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can have program code that is configured to carry out the embodiment of the method according to the invention when executed by the processor device. The program code can be stored in a data memory of the processor device. The processor device can, for example,based on at least one circuit board and / or on at least one SoC (System on Chip).

[0045] The invention also includes further developments of the control device according to the invention that have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the control device according to the invention are not described again here.

[0046] The control device can be part of a motor vehicle, which is preferably designed as a motor vehicle, in particular as a passenger car or a truck, or as a passenger bus or a motorcycle.

[0047] As a further solution, the invention also encompasses a computer-readable storage medium comprising program code which, when executed by a computer or computer network, causes the computer to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated on the Internet, for example, as a so-called app store server and / or cloud server. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code may be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).

[0048] The invention also encompasses combinations of the features of the described embodiments. The invention therefore also encompasses implementations that each comprise a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.

[0049] Exemplary embodiments of the invention are described below. Shown are: Fig. 1 a schematic representation of an architecture of at least one artificial neural network for realizing an embodiment according to the invention; Fig. 2 a sketch illustrating a filtering on an output of the at least one neural network according to the invention and Fig. 3 a sketch illustrating a raycasting method on an instance according to the invention.

[0050] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0051] In the figures, the same reference symbols designate elements with the same function.

[0052] Fig. Figure 1 shows a schematic representation of an architecture of at least one artificial neural network for implementing an embodiment of the disclosure. First, image data 1 can be provided, which describes at least one image of an environment with an object located therein and which was received from at least one sensor device, e.g., during a journey in a motor vehicle.

[0053] For this purpose, it can be provided that sensor data from at least one sensor device, e.g., from a camera system and a radar system, are previously weighted (multiplied) by a respective sensor-specific weighting factor, and a sum or a square sum can be calculated from the thus weighted sensor data, thereby generating a sum value. Thus, a sensor fusion can be performed. If the sum value is greater than a threshold value, image data 1 can be generated that signals that an object is within the detection range of the sensors. If the sum value is less than the threshold value, image data 1 can be generated that signals that no object is within the detection range. In this case, it can be provided that a message is output indicating that no object has been detected. If an object has been detected, the method can continue as follows: The image data 1 can then be fed into at least one neural network, in particular into a backbone 2 of the at least one neural network, which is trained to determine feature data, wherein the feature data comprises predetermined features regarding basic geometric shapes and / or colors of the object. This backbone 2 can have a self-attention technique 11 and / or a feature pyramid network technique 12 and / or be configured to implement such a technique. This feature data can then be represented on a feature map 3.

[0054] In a step a), edge endpoints 7 of the object structure of the object can be determined by applying an edge endpoint determiner 6 of the at least one neural network to the feature data, wherein edge endpoints 7 each mark a predetermined object end region of the object.

[0055] In a step b), intermediate edge points 5 of the object structure can be determined by applying an intermediate edge point determiner 4 of the at least one neural network to the feature data, wherein intermediate edge points 5 mark line-geometric shapes of the object, wherein a line-geometric shape is part of the object structure of the object.

[0056] In a step c), distance components 9 of individual distance points of the object structure can be determined by applying a regression network 8 of the at least one neural network to the feature data, wherein distance components 9 each positionally represent a proportional distance position in an instance 14, 15, 17 of the object structure.

[0057] Steps a) and b) and c) can be carried out independently.

[0058] In a step d), the object structure can be determined by means of the edge end points 7 and / or the edge intermediate points 5 and / or the line segments 9, after the filtering of Fig. 2 was applied.

[0059] Fig. Figure 2 shows a sketch illustrating filtering on an output of at least one neural network. Shown are a first instance 14, a second instance 15, and a third instance 17 of an object structure, e.g., a lane marking. These instances 14, 15, 17 can represent a total of three lane lines. Each of these instances 14, 15, 17 has an edge end point 7. By way of example, it is shown how a filter area 13 targets intermediate edge points 5 by applying a filter. For this purpose, it can be provided that this filter area, at the beginning of a filtering, delimits at least one object end area of ​​at least one instance 14, 15, 17 of an object structure. This filter can be a window-based filter that systematically filters across the Fig. 2 represents the output of the at least one neural network. In a further step, segments 9 of the intermediate edge points 5 that lie on the instance 14 can be used to determine an extension direction.

[0060] Within this filtering, as in Fig. 3, a raycasting method can be applied so that the instances 14, 15, 17 of the object structure are marked as such and / or separated from each other and / or sub-areas of the instances 14, 15, 17 are assigned to the corresponding instances 14, 15, 17.

[0061] Here, the raycasting method is applied to instance 14 as an example. In doing so, a raycasting beam 16 can be fired in different directions within instance 14 delimited by filter area 13, so that further edge endpoints 7 and / or intermediate edge points 5 can be identified or determined within this delimited instance 14. This makes it possible to identify a further edge endpoint 7 and / or intermediate edge point 5 and / or enable a connection to this further edge endpoint 7 and / or intermediate edge point 5, which has the next largest or next smallest distance portion 9 on the same instance 14 according to the raycasting method. By applying the raycasting method to instance 14, instance 15 can be disregarded, so that a connection of edge endpoints 7 and / or intermediate edge points 5 is only enabled and / or performed within instance 14.

[0062] For example, if an edge endpoint 7 contains a distance share of 1% in percentage points, the filter can search for an edge endpoint 7 and / or an edge intermediate point 5 that has a distance share of, for example, 2%, and connect them together and / or mark them as belonging to a connection. By applying the raycasting method, it is possible to prevent an edge endpoint 7 and / or an edge intermediate point 5 located on instance 14 from being connected to another edge endpoint 7 and / or an edge intermediate point 5 located on another instance 15, 17.

[0063] The filter can be shifted along an instance 14, 15, 17 by at least one length section, resulting in a new filter area 13. This length section can, for example, include a shift of one pixel to 30 pixels. Edge endpoints 7 and / or intermediate edge points 5 in this new filter area 13 can then be targeted again. The steps can be repeated until a predetermined threshold value with respect to a number of interconnected edge endpoints 7 and / or intermediate edge points 5 and / or a predetermined maximum value of a distance portion 9 and / or a total number of instances 14, 15, 17 is reached and / or no further instances 14, 15, 17 are present and / or no further edge endpoints 7 and / or intermediate edge points 5 to be connected are present in the filter area 13.

[0064] The object structure can be provided for a computer vision function, such as lane detection. Longitudinal and / or lateral guidance of a motor vehicle can then be enabled and / or performed by a control device having this computer vision function.

[0065] Overall, the examples show how the at least one neural network and a control resulting from the output of the at least one neural network can be provided. List of reference symbols 1 Image data 2 Backbone 3 Feature map 4 edge intermediate point detectors 5 Edge intermediate point 6 edge endpoint detectors 7 Edge endpoint 8 Regression network 9 Route share 11 Self-Attention Technique 12 Feature Pyramid Network Technique 13 Filter area 14 first instance 15 second instance 16 Raycasting beam 17 third instance

Claims

[1] Method for determining an object structure of an object, comprising the steps: - providing image data (1) which describe at least one image of an environment with the object located therein and which was received from at least one sensor device; and - feeding the image data (1) into at least one neural network which is trained to determine feature data, wherein the feature data comprise predetermined features with regard to basic geometric shapes and / or colors of the object; a) determining edge endpoints (7) of the object structure of the object by applying an edge endpoint detector (6) of the at least one neural network to the feature data, wherein edge endpoints (7) each mark a predetermined object end region of the object, b) determining intermediate edge points (5) of the object structure by applying an intermediate edge point determiner (4) of the at least one neural network to the feature data, wherein intermediate edge points (5) mark line-geometric shapes of the object, wherein a line-geometric shape is part of the object structure of the object, characterized by c) determining distance components (9) of individual distance points of the object structure by applying a regression network (8) of the at least one neural network to the feature data, wherein distance components (9) each positionally represent a proportional distance position in an instance (14, 15, 17) of the object structure, wherein a) and b) and c) are carried out independently of one another, d) wherein the object structure is determined by means of the edge end points (7) and / or the edge intermediate points (5) and / or the line segments (9), by the steps: a.1) Targeting edge end points (7) and / or edge intermediate points (5) by means of a filter area (13) by applying a filter, wherein this filter area (13) delimits at least one object end area of ​​at least one instance (14, 15, 17) of the object structure at the beginning of a filtering, b.1) Using stretching components of the targeted edge end points (7) and / or edge intermediate points (5) to determine an extension direction, c.1) applying a raycasting method to the instance (14, 15, 17) so that individual instances (14, 15, 17) of the object structure are marked as such and / or separated from each other and / or sub-areas of the instances (14, 15, 17) are assigned to the corresponding instances (14, 15, 17), d.1) connecting at least one targeted edge end point (7) and / or edge intermediate point (5) with another edge end point (7) and / or edge intermediate point (5) in this filter area (13) which has the next larger or next smaller distance component (9) on the same instance (14, 15, 17) according to the raycasting method, e.1) Moving the filter along the instance (14, 15, 17) by at least one length section, so that a new filter area (13) is obtained, f.1) re-targeting edge endpoints (7) and / or edge intermediate points (5) in this new filter area (13), g.1) Repeating steps b.1) to f.1) until a predetermined threshold value with respect to a number of interconnected edge end points (7) and / or edge intermediate points (5) and / or a predetermined maximum value of a distance portion (9) and / or a total number of instances (14, 15, 17) is reached and / or no further instances (14, 15, 17) are present and / or no further edge end points (7) and / or edge intermediate points (5) to be connected are present in the filter area (13), - Providing the object structure for a computer vision function, - Carrying out longitudinal and / or transverse guidance of a motor vehicle by means of a control device having this computer vision function. [2] Method according to claim 1, wherein the predetermined maximum value of a route component (9) is in a range of 70 to 80 percentage points, preferably in a range of 80 to 90 percentage points. [3] Method according to one of the preceding claims, wherein an instance (14, 15, 17) represents a line of a lane marking. [4] Method according to one of the preceding claims, wherein the feature data are determined by means of a self-attention technique (11). [5] Method according to one of the preceding claims, wherein the feature data are determined by means of a feature pyramid network technique (12). [6] Method according to one of the preceding claims, wherein the at least one neural network is trained to determine the object structure by means of error feedback, using training data which has a characterisation of the object structure. [7] Control device for a motor vehicle, wherein the control device comprises a processor device having program instructions which, when executed by the processor device, cause it to carry out a method according to one of the preceding method claims. [8] Motor vehicle comprising a control device according to claim 7 and at least one sensor device for determining image data (1), which is coupled to the control device, wherein the image data (1) describe at least one image of an environment with an object located therein and which were received from the at least one sensor device during a journey in the motor vehicle, are provided, wherein by means of the control device a control for carrying out a longitudinal and / or transverse guidance of the motor vehicle is carried out.

Images