Method and system for improving the recognition of traffic signs with supplementary signs for a driver assistance system

The method improves ADAS systems by integrating image and map data fusion to reliably recognize additional traffic signs, addressing recognition challenges and enhancing safety and navigation accuracy.

DE102023115404A1Pending Publication Date: 2024-12-19DR ING H C F PORSCHE AG
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
DE102023115404
Authority / Receiving Office
DE · DE
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-06-13
Publication Date
2024-12-19

AI Technical Summary

Technical Problem

Existing driver assistance systems (ADAS) struggle to reliably recognize additional traffic signs such as time-dependent speed limits and signs applicable to specific vehicle types, leading to incorrect speed adjustments and potential rule violations.

Method used

A method involving image data capture, feature extraction, classification, and data fusion with map data to enhance the recognition of main traffic signs and additional signs, using a data fusion algorithm to determine the reliability of the information source (image or map data) for accurate sign recognition.

Benefits of technology

Enhances the reliability and safety of ADAS systems by ensuring accurate recognition of additional signs, allowing for appropriate vehicle responses and improved navigation, even in adverse conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000012_0000
    Figure 00000012_0000
  • Figure 00000013_0000
    Figure 00000013_0000
  • Figure 00000013_0001
    Figure 00000013_0001
Patent Text Reader

Abstract

The invention relates to a method for improving the recognition of traffic signs with supplementary signs for a driver assistance system, comprising: - Recording (S10) image data (270) by at least one camera (210) of a camera module (200) of a vehicle (250) while driving along a route; - Extracting (S20) a main traffic sign (273) and / or a supplementary sign (275) from the image data (270) using a software application (450) of an evaluation module (400); - Classifying (S30) the main traffic sign (273) and / or the supplementary sign (275) by the software application (450) with regard to the information contained therein; - Comparing (S40) the extracted and classified image data (470) with map data (370) using a data fusion algorithm; - Evaluate (S50) the data fusion to extract the information about the additional sign (275) either from the extracted and classified image data (470) or from the map data (370) and transfer it to a driver assistance module (500).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The invention relates to a method, a system and a computer program product for improving the recognition of traffic signs with supplementary signs for a driver assistance system.

[0002] Modern vehicles are equipped with a variety of advanced driver assistance systems (ADAS) and highly automated driving functions (ADS). ADAS systems assist the driver in controlling the vehicle, increasing safety and enhancing the driving experience. There are various types of ADAS, such as adaptive cruise control, which automatically maintains a constant distance to the vehicle ahead and adjusts the vehicle's speed; lane keeping assist, which warns the driver before leaving their lane; emergency braking, which automatically applies the brakes if a critical situation is detected; and parking assist, which helps the driver with parking maneuvers. ADAS systems are constantly being developed and contribute to increasing road safety and preventing accidents.

[0003] A traffic sign algorithm is a crucial component of many driver assistance systems and autonomous vehicles, as it enables the vehicle to recognize and react to traffic signs such as speed limit signs, stop signs, and pedestrian crossings. Advanced driver assistance systems (ADAS) and traffic sign algorithms are therefore closely linked, with the latter being a key component of ADAS systems. For example, as an ADAS-equipped vehicle approaches a stop sign, the traffic sign algorithm recognizes the sign and transmits the relevant information to the vehicle's control system. The control system can then use this information to automatically apply the brakes and bring the vehicle to a stop.Similarly, when a speed limit sign is detected, the control system can adjust the vehicle's speed to ensure that the prescribed maximum speed is observed.

[0004] The traffic sign algorithm is a machine vision algorithm for recognizing and classifying traffic signs in images or video recordings. The algorithm uses a combination of image processing techniques and machine learning algorithms to recognize the shape, color, and content of traffic signs. The algorithm essentially consists of the following steps: a) Preprocessing: The image or video image is first preprocessed to improve image quality, reduce noise, and increase the contrast and brightness of the image. b) Feature extraction: The algorithm extracts relevant features from the image, e.g., color, shape, and texture, which can be used to distinguish between different types of traffic signs. c) Classification: The extracted features are fed to a machine learning algorithm, e.g. a neural network, which is trained with a dataset of traffic sign images to recognize and classify the traffic signs in the input image. d) Post-processing: After the recognition and classification of the traffic signs, post-processing techniques can be used to remove false positive results, track the traffic signs over time, and improve recognition accuracy.

[0005] The traffic sign algorithm is therefore an essential component of a driver assistance system, as it is capable of recognizing a wide variety of traffic signs and triggering a response from the driver assistance system, particularly for warning, regulatory, and informational signs. The algorithm must be accurate and robust, and function under varying light and weather conditions to ensure the correct operation of the driver assistance system. A well-functioning traffic sign algorithm is thus crucial for improving vehicle safety and efficiency on the road. By employing advanced image processing for traffic sign recognition and response, driver assistance systems can help prevent accidents, reduce congestion, and enhance the overall driving experience.

[0006] To ensure the quality of the traffic sign algorithm, test drives are conducted regularly. During these drives, a test driver evaluates the algorithm from a customer perspective. Based on the driver's evaluation, a statistical analysis is performed, providing insights into the algorithm's performance and the nature of any errors. This analysis revealed a significant weakness in the recognition of supplementary signs. Examples of supplementary signs include time-dependent speed limits, speed limits applicable only to trucks, speed limits for turning lanes, and signs indicating "noise protection," etc. In particular, supplementary signs applicable only to trucks are recognized very unreliably and are therefore not processed correctly by an ADAS system. While the supplementary sign itself is generally recognized for time-dependent speed limits, the time indication, e.g.,Between 8 a.m. and 6 p.m., the speed limit is often not reliably detected. This results in a valid speed limit being displayed at the wrong time. Alternatively, a 30 km / h speed limit sign with the additional "Noise Protection" sign is only recognized for turning lanes and therefore not applied by the ADAS system when driving straight ahead. This can lead to drivers not complying with the rules, for example, by maintaining the previously permitted maximum speed of 50 km / h. While high-performance cameras can improve perception, this is associated with high costs and is therefore not a practical solution.

[0007] EP 4 072 173 A1 discloses a method for comparing map information recorded by a test vehicle with stored map data in order to update and improve the map data.

[0008] DE 10 2018 005 865 A1 discloses a method for testing a driver assistance system designed as a map-based speed limit system, in which traffic signs are recorded using image and map data.

[0009] The invention is based on the objective of demonstrating ways in which the recognition of traffic signs with supplementary signs for a driver assistance system, in particular an ADAS system, can be improved in order to increase the safety and reliability of the driver assistance system.

[0010] This problem is solved according to the invention with respect to a method by the features of claim 1, with respect to a system by the features of claim 12, with respect to an ADAS system by the features of claim 14, and with respect to a computer program product by the features of claim 15. The further claims relate to preferred embodiments of the invention.

[0011] The present invention enhances the performance of a driver assistance system, particularly an ADAS system, by reliably detecting a main traffic sign with a supplementary sign. Supplementary signs contain information relevant to compliance with the traffic rule specified by the main traffic sign. The reliable and error-free detection of a main traffic sign with a supplementary sign enables an appropriate response from the driver assistance system, which is crucial for driver safety. Supplementary signs are no longer overlooked or misinterpreted. This allows the ADAS system, for example, to automatically adjust the vehicle's speed to the speed limit. A further advantage of improved supplementary sign recognition is more reliable navigation, as the ADAS system can also display directional signs and other traffic signs to the driver when needed.

[0012] According to a first aspect, the invention provides a method for improving the recognition of traffic signs with supplementary signs for a driver assistance system. The method comprises the following steps: - Recording image data by at least one camera of a vehicle's camera module while driving along a route; - Extracting a main traffic sign and / or a supplementary sign from the image data using a software application of an evaluation module; - Classification of the main traffic sign and / or the supplementary sign by the software application with regard to the information contained therein; - Comparing the extracted and classified image data with map data using a data fusion algorithm; - Evaluating the data fusion to extract the information about the additional sign either from the extracted and classified image data or from the map data and transferring it to a driver assistance module.

[0013] The training program stipulates that the driver assistance module will include an ADAS system.

[0014] In an advantageous embodiment, it is provided that if the image data and the map data match with respect to a recognized additional character, the recognized additional character is extracted from the image data and made available to the driver assistance module.

[0015] In a further embodiment, it is provided that in the event of a discrepancy between the information in the image data and the map data, the quality of the image data is evaluated and the information about the additional sign is taken either from the image data or from the map data and provided to the driver assistance module.

[0016] In an advantageous embodiment, it is provided that, in the case of low image data quality, the information about the additional sign is extracted from the map data and made available to the driver assistance module.

[0017] In particular, it is intended that in the event of an incorrect classification, the information about the additional sign will be extracted from the map data and provided to the driver assistance module.

[0018] Advantageously, if the image data is of high quality, the information about the additional sign is extracted from the image data and provided to the driver assistance module.

[0019] In a further embodiment, it is provided that, in the event of a systematic camera error being detected despite high image data quality, the information about the additional sign is extracted from the map data and made available to the driver assistance module.

[0020] In particular, if map data is missing, the information about the additional sign is taken from the image data and provided to the driver assistance module.

[0021] Ideally, the camera is a digital camera with Full HD resolution.

[0022] In another embodiment, the database and / or the evaluation module is / are integrated into a cloud computing infrastructure.

[0023] According to a second aspect, the invention provides a system for improving the recognition of traffic signs with supplementary signs for a driver assistance system. The system comprises a camera module, a database, an evaluation module, and a driver assistance module. The camera module is configured to record image data using at least one camera of a vehicle while driving along a route.The evaluation module comprises a software application designed to extract a main traffic sign and / or a supplementary sign using a software application, to classify the main traffic sign and / or the supplementary sign with regard to the information contained therein, to compare the extracted and classified image data with map data using a data fusion algorithm, and to evaluate the data fusion in order to transmit the information about the supplementary sign either from the extracted and classified image data or from the map data to a driver assistance module.

[0024] In a further training course, it is provided that if the image data and the map data match with regard to a recognized additional sign, the recognized additional sign is extracted from the image data and provided to the driver assistance module, or that if there is a discrepancy between the information in the image data and the map data, the quality of the image data is assessed and the information about the additional sign is extracted either from the image data or from the map data and provided to the driver assistance module, or that if map data is missing, the information about the additional sign is extracted from the image data and provided to the driver assistance module.

[0025] According to a third aspect, the invention provides an ADAS system for a motor vehicle that uses the method according to the first aspect for the recognition of traffic signs with a main traffic sign and a supplementary sign.

[0026] According to a fourth aspect, the invention provides a computer program product comprising an executable program code configured to perform the method according to the first aspect when executed.

[0027] The invention will now be explained in more detail with reference to exemplary embodiments shown in the drawing.

[0028] This shows: Fig. 1 a block diagram to illustrate an embodiment of a system according to the invention; Fig. 2 a flowchart to explain the individual process steps of a process according to the invention; Fig. 3 a block diagram of a computer program product according to an embodiment of the third aspect of the invention.

[0029] Further features, aspects and advantages of the invention or its embodiments are explained in the following description in conjunction with the patent claims.

[0030] Fig. Figure 1 shows a system 100 according to the invention for improving the recognition of traffic signs with supplementary signs for a driver assistance system (ADAS system).

[0031] The system 100 according to the invention comprises a camera module 200, a database 300, an evaluation module 400 and a driver assistance module 500. The camera module 200, the database 300, the evaluation module 400 and the driver assistance module 500 can each be equipped with a processor and / or a memory unit, but can also be integrated in a common computing unit, in particular in a vehicle 250.

[0032] In particular, the database 300 and the evaluation module 400 can be integrated into a cloud computing infrastructure. However, the database 300 can also be housed, for example, in a navigation system and the evaluation system in the vehicle's on-board computer.

[0033] In the context of the invention, a "module" can be understood to mean, for example, a processor and / or a processor unit and / or a memory unit for storing program instructions. The processor is specifically configured to execute the program instructions in order to perform or implement the method according to the invention or a step thereof. In particular, a module can be integrated into a cloud computing infrastructure. However, it is also possible for a computing unit, such as a processor, to comprise several modules.

[0034] In the context of the invention, a "processor" can be understood to mean, for example, a machine or an electronic circuit. A processor can, in particular, be a central processing unit (CPU), a microprocessor, or a microcontroller, e.g., an application-specific integrated circuit or a digital signal processor, possibly in combination with a memory unit for storing program instructions, etc. A processor can also be a virtualized processor, a virtual machine, or a soft CPU.

[0035] In the context of the invention, a "storage unit" or "storage module" can refer, for example, to volatile memory in the form of random access memory (RAM), permanent storage such as a hard drive or data carrier, or, for example, a replaceable storage module. The storage module can also be configured as a cloud storage solution.

[0036] In the context of the invention, "data" includes both raw data and already processed data from measurement results from sensors such as cameras, as well as from other data sources.

[0037] Furthermore, communication links are provided for the exchange and transmission of data between the individual modules, specifically designed as wireless communication links, e.g., mobile communication links, and / or near-field communication links, e.g., Bluetooth®, Ethernet, NFC (near field communication), or Wi-Fi®. In particular, 5G or 6G radio modules are provided for the communication link between the vehicle 250 and a cloud computing infrastructure.

[0038] In particular, at least some of the modules can be integrated into a cloud computing infrastructure. This enables faster calculation results, as cloud-based solutions offer the advantage of high and therefore faster computing power. Cryptographic encryption methods can also be used to protect the connection to the cloud computing infrastructure via a mobile network. Specifically, 5G or 6G radio modules are intended for a communication link between the vehicle 250 and a cloud computing infrastructure, for example, between the camera module 200 and the evaluation module 400.

[0039] The term cloud computing infrastructure refers to the entirety of hardware, software, network, and storage resources used to support cloud computing services. Cloud computing infrastructure provides a platform for delivering a wide range of services, including data storage and management, application development and deployment, and virtual computing resources. It encompasses services for providing computing resources such as processing power and storage to support cloud-based applications. To protect data and applications from unauthorized access, security technologies such as firewalls, encryption, and identity and access management tools are employed. A cloud computing infrastructure can increase computing power and the speed of data processing.

[0040] The camera module 200 comprises at least one camera 210, which is arranged in a vehicle 250. Typically, several cameras 210 are provided in a vehicle 250 for use in a driver assistance system. These cameras 210 generally use different imaging technologies and are arranged at various locations on the vehicle to provide a comprehensive view of the surroundings. Some of the most common types of cameras used in driver assistance systems are forward-facing cameras, which are mounted at the front of the vehicle and detect and capture other vehicles, pedestrians, and obstacles in the vehicle's path. They are frequently used for functions such as lane keeping assist, automatic emergency braking, and adaptive cruise control. There are also rear-facing cameras.These cameras are typically mounted at the rear of the vehicle and capture the area behind the vehicle, including other vehicles, pedestrians, and obstacles. They are commonly used for features such as reversing cameras and cross-traffic alert. Side cameras can also be used. These cameras are typically mounted on the sides of the vehicle and capture areas that the driver cannot see using the mirrors. They are commonly used for features such as obscured area detection and lane departure warning.

[0041] Additionally, surround-view cameras can be used. These cameras are usually mounted on several sides of the vehicle and provide a 360-degree view of the vehicle's surroundings. They are frequently used for functions such as parking assistance and collision avoidance.

[0042] The cameras 210 of the camera module 200 are typically RGB cameras. An RGB camera is a digital camera that captures color information in the visible part of the electromagnetic spectrum and divides it into three channels: red, green, and blue (RGB). These three channels are then combined to create the full color spectrum of the image. The cameras 210 operate in the visible spectral range, which encompasses the wavelength range perceptible to the human eye (approximately 400 to 700 nanometers). However, some cameras 210 can also operate in the infrared or ultraviolet spectrum, which can be useful for applications such as night vision or the detection of certain types of hazards. IR cameras utilize infrared imaging technology to capture the environment in low-light or no-light conditions, such as at night or in tunnels. They are commonly used for functions such as night vision and pedestrian detection.

[0043] A camera's resolution refers to the number of pixels that make up the image it captures. A camera with a higher resolution captures more detailed and sharper images than one with a lower resolution. A camera's resolution is typically expressed by the number of horizontal and vertical pixels in the image, for example, 1920 x 1080 (also known as Full HD) or 3840 x 2160 (also known as 4K). Full HD cameras are generally used for driver assistance systems because the processing power and storage capacity required for a 4K camera in a vehicle would be too high.

[0044] The image capture frequency of a vehicle camera 210 for a driver assistance system refers to the rate at which the camera captures and stores images per second. This parameter is also known as the frame rate or frames per second (FPS) and is an important criterion when selecting a camera for ADAS applications. A higher frame rate means the camera can capture and store more images per second, resulting in a smoother and more detailed representation of the surroundings. This can be particularly important for ADAS applications that require real-time object detection and tracking, such as lane keeping assist or pedestrian detection.

[0045] Typical frame rates for cameras in ADAS applications range from 30 to 60 frames per second, with more powerful cameras capable of capturing up to 120 frames per second. The frame rate required for a specific ADAS application depends on factors such as vehicle speed, environmental complexity, and specific system requirements.

[0046] Higher frame rates increase the demands on the camera system's data storage and processing capabilities, which can impact the overall cost and complexity of the system. Therefore, when selecting a camera for ADAS applications, a balance must be struck between frame rate, image quality, and system requirements to ensure optimal performance and efficiency.

[0047] As the vehicle 250 travels along a route, at least one camera 210 records image data 270 of the vehicle 250's surroundings and transmits this data to the evaluation module 400. The evaluation module 400 comprises a software application 450 that includes an extraction algorithm designed to extract and classify essential features from the image data 270. The extraction and classification algorithm utilizes, in particular, artificial intelligence algorithms such as neural networks.

[0048] A neural network consists of neurons arranged in multiple layers and interconnected in various ways. A neuron can receive information at its input from the outside or from another neuron, process it in a specific way, and then pass it on in a modified form to another neuron at its output, or output it as a final result. Hidden neurons are located between the input neurons and the output neurons. Depending on the type of network, there can be several layers of hidden neurons. They are responsible for the transmission and processing of information. Output neurons ultimately deliver a result and transmit it to the outside world. Different types of neural networks arise from the different arrangement and connections of the neurons, such as a feedforward network (FNN) or a reverse-processing network (FNN).Recurrent Neural Networks (RNNs) or Convolutional Neural Networks (CNNs). These networks can be trained through unsupervised or supervised learning.

[0049] The extraction algorithm detects whether the image data 270 contains a main traffic sign 273 and at least one supplementary sign 275. The classification algorithm assigns specific information to the extracted image elements, e.g., that a speed limit for trucks applies between 10 p.m. and 6 a.m. The image data 470 extracted and classified using the extraction and classification algorithm are then compared with the map data 370 stored in the database 300.

[0050] The map data 370 contains various types of information that vehicle sensors and cameras alone cannot capture. This data is gathered from various sources, such as GPS satellites, road markings, and other sources. The data typically includes information about road geometry, such as curvature, gradient, and width, as well as the location of intersections, traffic lights, and pedestrian crossings. This data helps the vehicle accurately determine its current position and maintain its lane while driving. Additional information relates to speed limits. The map data can include information about posted speed limits, as well as special speed limits imposed due to road conditions or environmental factors (e.g., construction zones). This information can be used by ADAS to adjust the vehicle's speed and acceleration in real time.Furthermore, the map data can contain 370 pieces of information about traffic conditions, such as traffic flow, which are collected in real time from various sources, such as roadside sensors and GPS-enabled devices used by other vehicles. This data can be used by a driver assistance system for route guidance and to navigate the vehicle through areas with high traffic volume.

[0051] A data fusion algorithm is used to compare the main traffic sign 273 and the supplementary sign 275 with the map data 370. Data fusion refers to the process of integrating data from multiple sources to obtain a more comprehensive and accurate representation of the environment. According to the invention, data fusion is used to combine information from the extracted image data 470 and the map data 370 to improve the accuracy and reliability of the detection and localization of a supplementary sign 275.

[0052] The image data 270 is real-time data about the vehicle's immediate surroundings, including the position, speed, and direction of other vehicles, pedestrians, obstacles, and traffic signs. However, the camera's view 210 can be affected by factors such as lighting conditions, weather conditions, and obstructions like objects blocking the camera's view.

[0053] In contrast, map data 370 contains a static representation of the environment, including information about the road network, traffic signs, and other features. Therefore, map data 370 can be a more reliable source of information than image data 270, although it may not always reflect the current state of the environment, for example, due to construction work or other changes.

[0054] The data fusion of the extracted and classified image data 470 and the map data 370 using the data fusion algorithm results in three case distinctions, based on which the invention determines from which data source the information about an additional sign 285 is transmitted to the driver assistance module 500, i.e., whether the extracted and classified image data 470 or the map data 370 forms the basis for transmitting the information about the additional sign 285 to the driver assistance module 500. The following cases are distinguished: a) In a first case, the data from image data 470 and map data 370 match with regard to a recognized supplementary sign 275. In this case, the supplementary sign 275 is taken directly from image data 470 and the corresponding information is fed to the driver assistance module 500. b) In a second case, the information in image data 470 and map data 370 does not match. In this case, the quality of image data 270 is evaluated, and the information about the supplementary sign 275 is taken from either image data 470 or map data 370 and fed to the driver assistance module 500.

[0055] In the context of camera recognition, quality refers to the accuracy and reliability of identifying a supplementary character 275 within the camera's field of view 210. High-quality image data 470 is characterized by the precise and unambiguous identification of a supplementary character 275. With low-quality image data 470, identification of a supplementary character is either impossible or very imprecise, for example, due to environmental conditions such as lighting or weather, or because another object obscures the supplementary character 275. Quality can be assessed using a quality index, e.g., on a scale from zero to one. Depending on the assessed quality of the image data 270, various subcategories are distinguished: b1) If the image data 470 is of low quality, the information about the additional sign 275 is taken from the map data 370. b2) The same applies if an additional sign 275 was extracted from the image data 470, but no classification could be made, so that it is not recognizable which additional sign 275 it is exactly. In this case, too, the information about the additional sign 275 is taken from the map data 370. b3) If the image data 470 is of high quality, the information about the additional character 275 is extracted from the image data 470. b4) The image data 470 may be of high quality, however a systematic camera error was detected. In this case, the information about the supplementary sign 275 is not taken from the image data 470, but the map data 370 serves as the data source.

[0056] c) There is also a third case where no map data 370 is available. In this case, the information about the supplementary sign 275 is taken from the image data 470 and transmitted to the driver assistance module 500.

[0057] The driver assistance module 500 comprises, in particular, an ADAS system 550 that reacts to the transmitted supplementary sign 275 according to the information contained therein. The case distinction according to the invention increases the recognition reliability for a main traffic sign 273 with at least one supplementary sign 275. By combining the information from the image data 270 and the map data 470, the ADAS system 550 is able to warn the driver more accurately and in a timely manner of potential hazards or to assist the driver in correctly adhering to speed limits.

[0058] In Fig. Section 2 describes the procedural steps for improving the recognition of traffic signs with supplementary signs for a driver assistance system.

[0059] In step S10, image data 270 is recorded by at least one camera 210 of a camera module 200 of a vehicle 250 while driving along a route.

[0060] In step S20, a main traffic sign 273 and / or a supplementary sign 275 is extracted from the image data 270 using a software application 450 of an evaluation module 400.

[0061] In step S20, the main traffic sign 273 and / or the supplementary sign 275 is classified by the software application 450 with regard to the information contained therein.

[0062] In step S40, the extracted and classified image data 470 are compared with map data 370 using a data fusion algorithm.

[0063] In step S50, the data fusion is evaluated to extract the information about the additional sign 275 either from the extracted and classified image data 470 or from the map data 370 and transfer it to a driver assistance module 500.

[0064] Fig. Figure 3 schematically represents a computer program product 900 comprising an executable program code 950 configured to execute the method according to the first aspect of the present invention.

[0065] The present invention significantly enhances the performance of a driver assistance system, particularly an ADAS system 550, by reliably recognizing a main traffic sign 273 with a supplementary sign. Supplementary signs contain information relevant to compliance with the traffic rule specified by the main traffic sign. The reliable and error-free recognition of a main traffic sign 273 with a supplementary sign 275 enables an appropriate response from the ADAS system 550, which is crucial for driver safety. Supplementary signs 275 are no longer overlooked or misinterpreted. This allows the ADAS system 550, for example, to automatically adjust the vehicle's speed to the speed limit. A further advantage of the improved supplementary sign recognition is more reliable navigation, as the ADAS system 550 can also display directional signs and other traffic signs to the driver when needed.

[0066] The present invention thus enables reliable and safe recognition of additional signs. It ensures that the ADAS system 550 functions reliably even in poor visibility conditions such as rain or snow. This significantly increases road safety. Reference sign 100 System 200 camera modules 210 camera 250 vehicles 270 image data 273 Main traffic signs 275 additional characters 300 database 370 map data 400 Evaluation module 450 software applications 470 extracted image data 500 Driver Assistance Module 550 ADAS system 900 computer program product 950 program code QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] EP 4 072 173 A1

[0007] DE 10 2018 005 865 A1

[0008]

Claims

[1] Method for improving the recognition of traffic signs with additional signs for a driver assistance system, comprising: - recording (S10) image data (270) by at least one camera (210) of a camera module (200) of a vehicle (250) when driving along a route; - Extracting (S20) a main traffic sign (273) and / or an additional sign (275) from the image data (270) by means of a software application (450) of an evaluation module (400); - Classifying (S30) the main traffic sign (273) and / or the additional sign (275) by the software application (450) with regard to the information contained therein; - comparing (S40) the extracted and classified image data (470) with map data (370) by a data fusion algorithm; - Evaluating (S50) the data fusion in order to extract the information about the additional sign (275) either from the extracted and classified image data (470) or from the map data (370) and to transmit it to a driver assistance module (500). [2] The method of claim 1, wherein the driver assistance module (550) comprises an ADAS system (550). [3] Method according to claim 1 or 2, wherein, if the image data (470) and the map data (370) match with respect to a recognized additional sign (275), the recognized additional sign (275) is extracted from the image data (470) and provided to the driver assistance module (500). [4] Method according to claim 1 or 2, wherein in the event of a discrepancy between the information of the image data (470) and the map data (370), the quality of the image data (270) is assessed and the information about the additional character (275) is taken either from the image data (470) or from the map data (370) and provided to the driver assistance module (500). [5] Method according to claim 4, wherein, if the quality of the image data (470) is low, the information about the additional sign (275) is taken from the map data (370) and provided to the driver assistance module (500). [6] Method according to claim 4, wherein in the event of an incorrect classification, the information about the additional character (275) is taken from the map data (370) and provided to the driver assistance module (500). [7] Method according to claim 4, wherein, if the quality of the image data (470) is high, the information about the additional character (275) is extracted from the image data (470) and provided to the driver assistance module (500). [8] Method according to claim 4, wherein in the event of a systematic camera error being detected despite high quality of the image data (470), the information about the additional sign (275) is taken from the map data (370) and provided to the driver assistance module (500). [9] Method according to claim 1 or 2, wherein, if map data (370) are missing, the information about the additional sign (275) is taken from the image data (470) and provided to the driver assistance module (500). [10] Method according to one of claims 1 to 9, wherein the camera (210) is a digital camera with full HD resolution. [11] Method according to one of claims 1 to 10, wherein the database (300) and / or the evaluation module (400) is / are integrated into a cloud computing infrastructure. [12] System (100) for improving the recognition of traffic signs with additional signs (275) for a driver assistance system, comprising a camera module (200), a database (300), an evaluation module (400) and a driver assistance module (500), wherein the camera module (200) is designed to record image data (270) by at least one camera (210) of a camera module (200) of a vehicle (250) when driving along a route;wherein the evaluation module (400) comprises a software application (450) which is designed to extract a main traffic sign (273) and / or an additional sign (275) from the image data (270) by means of a software application (450), to classify the main traffic sign (273) and / or the additional sign (275) with regard to the information contained therein, to compare the extracted and classified image data (470) with map data (370) by means of a data fusion algorithm and to evaluate the data fusion in order to transmit the information about the additional sign (275) either from the extracted and classified image data (470) or from the map data (370) to a driver assistance module (500); [13] System (100) according to claim 12, wherein, if the image data (470) and the map data (370) match with respect to a recognized additional sign (275), the recognized additional sign (275) is extracted from the image data (470) and provided to the driver assistance module (500), or wherein, if there is a discrepancy between the information in the image data (470) and the map data (370), the quality of the image data (270) is assessed and the information about the additional sign (275) is extracted either from the image data (470) or from the map data (370) and provided to the driver assistance module (500), or wherein, if map data (370) is missing, the information about the additional sign (275) is extracted from the image data (470) and provided to the driver assistance module (500). [14] ADAS system (550) for a motor vehicle, characterized bythat the method according to one of claims 1 to 11 is used to recognize traffic signs with a main traffic sign (273) and an additional sign (275). [15] A computer program product (900) comprising an executable program code (950) configured to carry out the method according to any one of claims 1 to 11 when executed.

Citation Information

Patent Citations

  • Procedure for testing an assistance system for a vehicle

    DE102018005865A1

  • Data transmission method and device

    EP4072173A1

  • Method and a device for identifying traffic-relevant information

    WO2008145545A1