Method for detecting changes in a track section and computer program product

Abstract

A method for detecting changes to a road segment is described, wherein map data of the road segment is loaded into an algorithm, wherein at least one satellite image of the road segment is loaded into the algorithm, wherein the map data and the satellite image are compared by the algorithm, wherein the road segment has approval for semi-autonomous driving or autonomous driving at level 3 or higher, wherein the at least one satellite image is acquired using a geostationary satellite or a satellite cluster flying in a low orbit, wherein at least one deviation between the map data and the at least one satellite image is detected, wherein the road segment is classified as invalid, and wherein the approval is revoked in the event of invalidation.

Description

A method for detecting changes in a section of track and a computer program product are described. Methods for detecting changes in a section of track and computer program products of the type mentioned above are known in the prior art. The SAE J3016 standard defines six levels of autonomous driving, from Level 0 (no automation) to Level 5 (full automation). Starting at Level 3 (Conditional Automation), a vehicle's system completely takes over the driving task under certain conditions, such as on highways, while the driver must be able to take over again at any time upon request. At Level 4 (High Automation), the system is capable of driving without human intervention in defined areas (e.g., geofencing), even if the driver cannot or does not want to intervene. Level 5 (Full Automation) describes vehicles that can drive under all conditions and without human assistance, regardless of road type, weather, or traffic conditions. Human responsibility decreases significantly from Level 3 onward, while the vehicle increasingly operates autonomously. In the context of partially and fully autonomous driving at Level 3 or higher according to SAE J3016, precise and reliable route planning by an autonomous driving system is crucial. Road sections designated for autonomous operation are typically defined under the assumption that they offer stable and predictable traffic flow. However, unforeseen changes to these routes, such as construction sites, temporary lane markings, altered traffic patterns, or new obstacles, can pose significant challenges to the route planning and decision-making of the autonomous system. Such changes can lead to the system either relying on insufficient or outdated map data or struggling to correctly interpret the altered conditions in real time. This poses the risk of incorrect decisions, such as failing to recognize a change in the road layout or blocking traffic flow through unnecessary braking or evasive maneuvers. Particularly in construction zones, often marked by temporary and difficult-to-see traffic signs or guide lines, or in the case of weather-related restrictions such as a snow-covered road, the autonomous vehicle's ability to reliably interpret and react to the new situation is put to the test. Furthermore, short-term temporary disruptions, such as a change in route due to a short-term recovery operation, pose a particular challenge. From DE 10 2019 215 522 A1, a method for updating attributes in at least one digital map by a control unit is known, wherein a digital map of an area divided into a plurality of sections is received, at least one aerial image and / or satellite image of the area is received, the at least one aerial image and / or satellite image of the area is divided into a plurality of segments which are geographically congruent with the map sections of the digital map, the segments of the at least one aerial image and / or satellite image are fed as input to at least one neural network and are evaluated by the neural network to determine attributes in the respective segments, the attributes determined by the neural network are stored to update or to complete attributes already stored in the digital map.Furthermore, a control unit, a computer program and a machine-readable storage medium are disclosed. The task therefore arises to further develop methods for detecting changes in a section of road as well as computer program products of the type mentioned above in such a way that earlier warnings to semi-autonomous or autonomous vehicles about obstructions on a section of road are possible. The problem is solved by a method for detecting changes in a section of track according to claim 1 and a computer program product according to dependent claim 10. Further embodiments and developments are the subject of the dependent claims. A method for detecting changes to a road segment is described, wherein map data of the road segment is loaded into an algorithm, wherein at least one satellite image of the road segment is loaded into the algorithm, wherein the map data and the satellite image are compared by the algorithm, wherein the road segment has approval for semi-autonomous driving or autonomous driving at level 3 or higher, wherein the at least one satellite image is acquired using a geostationary satellite or a satellite cluster flying in a low orbit, wherein at least one deviation between the map data and the at least one satellite image is detected, wherein the road segment is classified as invalid, and wherein the approval is revoked in the event of invalidation. Geostationary satellites orbit approximately 35 to 36 kilometers above the equator and move with the Earth's rotation. This allows them to remain stationary relative to a fixed point on the Earth's surface. This characteristic makes them ideally suited for route monitoring, as they enable continuous coverage of a specific area. This reduces the complexity of the system, since only a single satellite is needed to continuously monitor the required road area. Additionally, satellite images from a geostationary satellite are consistent over time, meaning they are not affected by the current angle from which the satellite is observing the area, as this angle remains constant. Low Earth orbit (LEO) satellite clusters operate at altitudes of approximately 300 to 2,000 kilometers. These satellites move quickly, orbiting the Earth several times a day. Such clusters often consist of hundreds or thousands of satellites working together to provide wide-area or near-global coverage, for example, for internet services or Earth observation. Their low altitude allows for lower latency but requires complex networks for continuous data transmission. Although the construction of a LEO satellite cluster is more complex, LEO satellites often achieve higher image resolution than geostationary satellites for optical reasons. The corresponding classification provides a clear indication of whether a specific road segment has changed sufficiently compared to the map data for that segment to revoke its authorization for autonomous driving at Level 3 or higher. Level 3 is defined according to SAE standard J3016. Level 3 permits autonomous driving or semi-autonomous procedures under certain conditions known as the Operational Design Domain (ODD). The ODD describes the specific framework within which an autonomous driving system can operate safely and reliably. This includes clear restrictions regarding geographical areas, such as specific roads or regions, as well as the types of roads the vehicle is permitted to use, for example, highways or urban streets.Furthermore, the ODD often includes speed limits, suitable weather conditions such as clear visibility or dry roads, and even time restrictions, such as daylight hours only. The complexity of traffic situations, such as construction sites or busy intersections, can also influence the ODD. These boundaries are essential because the vehicle in question can only operate safely and as intended within the ODD. Outside this defined area, operation is either impossible or requires human intervention, especially for systems with automation levels up to and including Level 4. The corresponding procedure ensures that, in the event of actual changes to the section of track, the ODD can be adapted accordingly at short notice in order to avoid incorrect reactions of the corresponding system and thus increase road safety. According to further training, the classification of a section of track may also include a reason for an invalid declaration of a corresponding release, for example, a construction site or recovery work in the section of track. In a first further development, it is provided that at least one satellite image has a resolution of 1 m or less. The use of high-resolution images offers significant advantages in determining whether a road section remains suitable for Level 3 semi-autonomous driving. High-resolution images enable a precise and detailed analysis of current road conditions, including lane markings, traffic signs, temporary construction sites, or other changes such as obstacles or altered traffic patterns. The high image quality allows even small details to be recognized, which are relevant for deciding whether the section of the route meets the requirements of semi-autonomous driving from level 3 onwards. In a further, more advanced version, it is envisaged that the section of track will have a length of one kilometer or more. This way, switching between driving modes too frequently, e.g. from autonomous driving to manual driving and vice versa, can be avoided. In a further, more advanced version, it is provided that the revocation of a release is transmitted to at least one motor vehicle that is equipped for semi-autonomous or autonomous driving at level 3 or higher. This allows the vehicle to react immediately if the conditions in the relevant section of the route have changed sufficiently compared to the map data. In a further refinement, the algorithm includes a machine learning model trained using supervised learning, wherein the machine learning model extracts information from the map data and the data of at least one satellite image using a convolutional neural network or a transformer model. A convolutional neural network (CNN) is a specialized architecture of artificial neural networks that is particularly effective for processing and analyzing image and video data. It uses convolution layers to extract features from input data, such as edges, shapes, or textures. This stepwise processing allows simple patterns to be recognized in early layers, while later layers identify more complex features and relationships. In addition to the convolution layers, a CNN includes pooling layers, which reduce the amount of data and thus optimize computing power, as well as fully connected layers, which are used for final classification or decision-making. A Transformer model is an advanced architecture in artificial intelligence specifically designed for processing sequential data such as text or images. It is based on the mechanism of attention, which allows for the effective weighting of relevant parts of a sequence, regardless of their position. Unlike earlier models such as recurrent neural networks (RNNs), the Transformer processes the entire sequence simultaneously, enabling parallel training and significantly higher efficiency. The architecture consists of repeated layers of encoder and decoder modules that extract features and generate context-dependent output. During the training of the machine learning model, a large dataset of both valid and invalid samples must be used, including satellite images that contain at least one deviation from the map data. In a further refinement, it is envisaged that the classification is carried out using a flattening layer, a forward-directed neural network, and a loss function. A flattening layer is a component in neural networks that transforms multidimensional input data, such as that from a convolutional neural network architecture, into a one-dimensional vector. This transformation allows the extracted features from previous layers to be passed to a feedforward neural network, which performs the final classification. The feedforward network uses a loss function to calculate the error between predicted and actual outputs. If the task, as in this case, is classification, the cross-entropy loss can be used. This function measures the difference between the predicted probability distribution of the network and the actual target distribution. Optimizing the cross-entropy loss function iteratively improves the model. In a further, more advanced version, it is envisaged that the machine learning model will be able to recognize attributes that lead to the invalidation of a release. Invalid attributes may include, but are not limited to: snow on the road, roadworks on one or more sides, approaching emergency vehicles, accident ahead. In a further, more advanced version, it is envisaged that the procedure will be repeated at regular intervals. This allows for continuous monitoring of the relevant section of the route. In a further, more detailed version, it is provided that each scenario in which a permit for the road section has to be declared invalid will result in a separate issue. A first independent subject matter relates to a computer program product comprising a computer-readable storage medium on which instructions are embedded which, when executed by at least one computing unit, cause that at least one computing unit to be equipped to execute the procedure of the aforementioned type. The process can be executed on one or more computing units, so that certain process steps are executed on one computing unit and other process steps on at least one other computing unit, whereby calculated data can be transmitted between the computing units if necessary. Further advantages, features, and details will become apparent from the following description, in which – possibly with reference to the drawing – at least one embodiment is described in detail. Identical, similar, and / or functionally equivalent parts are marked with the same reference numerals. The figures schematically show: Fig. 1 a training procedure for training an algorithm to detect changes in a section of a path, and Fig. 2 a procedure for detecting changes in a section of a path. Fig. 1 shows a training procedure for training an ALG algorithm to detect changes in a section of a route. For training, a training dataset TD is used, containing both map data KD and satellite imagery SB. The satellite imagery SB covers at least part of the same route segment as the map data. The satellite imagery SB contains discrepancies compared to the map data, for example, due to construction sites, road blockages, accidents, or similar issues. The training data is used to train a machine learning model MLM, which is part of the algorithm ALG. This algorithm classifies the route segment. The classification has two different categories: valid, represented by a plus sign, and invalid, represented by a minus sign. First, a comparison is made between the map data and at least one satellite image to align them. This is necessary because the satellite images may depict a different area than the map data. Scaling and image transformations may be required in this process. Then, within the mutually aligned map data and satellite images, a section of the route is selected that has a length of 500 meters or longer, in particular one kilometer. Within the route segment, a search is then conducted for discrepancies between the map data and the route segment. The algorithm, which utilizes a CNN, is trained using supervised learning. The training dataset TD contains labels, in this case, satellite image data SB. During the training process, the algorithm ALG learns to create a function that maps the input data KD, SB to the labels L. To achieve this, the input data KD, SB are processed by the machine learning model MLM, which generates a prediction. This prediction is then compared to the actual label L, and any potential error is calculated using a loss function. This error is then used to adjust the model parameters using optimization methods such as gradient descent. This process is repeated iteratively until the machine learning model MLM has learned the relationship between inputs and target values ​​with high accuracy. The trained machine learning model MLM can then be used to make predictions for unknown map data KD and satellite images SB. Fig. 2 shows the application of the trained algorithm ALG in reality. Unknown map data (KD) and satellite imagery (SB) are used to classify routes as valid (+) and invalid (-). If a route segment is declared invalid, a corresponding revocation (WR) is issued, which can be sent to at least one vehicle capable of Level 3 or higher autonomous driving. A handover request is then triggered shortly before reaching the affected route segment, allowing the driver to navigate the segment manually. The driver can be given information explaining the reasons for the revocation. In the event of a declaration of invalidity, a human supervisor may review the case, forwarding it to the responsible person. This person can then acknowledge the classification as invalid or revoke it, thereby reopening the section of road for autonomous driving at Level 3 or higher. Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as a further explanation in the description. Reference symbol list ALG Algorithm KD Map Data KL Classification L Label MLM Machine Learning Model TD Training Data SB Satellite Image + valid - invalid WR Revocation QUOTES INCLUDED IN THE DESCRIPTION This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature DE 10 2019 215 522 A1

[0007]

Claims

Method for detecting changes to a road segment, wherein map data (KD) of the road segment is loaded into an algorithm, wherein at least one satellite image (SB) of the road segment is loaded into the algorithm (ALG), wherein the map data (KD) and the satellite image (SB) are compared by the algorithm (ALG), characterized in that the road segment has approval for semi-autonomous driving or autonomous driving at level 3 or higher, wherein the at least one satellite image (SB) is acquired using a geostationary satellite or a satellite cluster flying in a low orbit, wherein at least one deviation between the map data (KD) and the at least one satellite image (SB) is detected, wherein the road segment is classified as invalid (-), and wherein the approval is revoked in the event of invalidation. Method according to claim 1 characterized in that the at least one satellite image (SB) has a resolution of 1 m or less. Method according to claim 1 or 2, characterized in that the route segment has a length of one kilometer or longer. Method according to one of the preceding claims, characterized in that the revocation of a release is transmitted to at least one motor vehicle equipped for semi-autonomous or autonomous driving at level 3 or higher. Method according to one of the preceding claims, characterized in that the algorithm (ALG) includes a machine learning model (MLM) trained by supervised learning, wherein the machine learning model (MLM) extracts information from the map data and the data of the at least one satellite image using a convolutional neural network or a transformer model. Method according to claim 5, characterized in that the classification is carried out by means of a flattening layer comprising a forward-directed neural network and a loss function. Method according to claim 5 or 6, characterized in that the machine learning model (MLM) is recognized to identify attributes that lead to the invalidation of a release. Method according to one of the preceding claims, characterized in that the method is repeated at regular intervals. Method according to one of the preceding claims, characterized in that each scenario in which a release for the road section must be declared invalid causes a separate output. Computer program product comprising a computer-readable storage medium on which instructions are embedded which, when executed by at least one computing unit, cause the at least one computing unit to be configured to execute the method according to one of the preceding claims.

Images