Method and system for monitoring road surface conditions by a machine learning system, and method and system for training a machine learning system

Abstract

The present invention relates to a method and a system for monitoring a road surface condition by a machine learning system (16), and a method and a system (10) for training the machine learning system (16). A method for training a machine learning system (16) for monitoring the road condition includes the following steps: - providing or capturing data by a sensor system (1) of a vehicle (2), wherein the sensor system (1) captures the surroundings of the vehicle (2) as training input data X; - providing or capturing data symbolizing the road condition as training target values Y by a reference sensor (5) provided in or on the vehicle, i.e., in the vehicle (2); and - training the machine learning system (16). Provided is training data including the training input data X and the training target values Y corresponding to the training input data X. Subsequently, parameters of the machine learning system are adjusted such that when the training input data is input, the machine learning system creates output data Y' similar to the training target values Y.

Description

Technical Field

[0001] The present invention relates to road condition monitoring (Road Condition Monitoring, Road Condition Observation) or road condition recognition from a moving vehicle equipped with sensors for capturing the surroundings. This theme is very significant from the perspective of accident avoidance because on certain road conditions or road surfaces, the friction or coefficient of friction between the tire and the road surface is reduced. That is, the braking distance may be extended, or the vehicle may slip or spin during braking, which can lead to serious accidents. In automated such interventions and / or braking interventions (ADAS / AD systems) in assisted or autonomous driving, the system must be able to estimate the road conditions as realistically as possible before the vehicle reaches that point. However, since this has not yet been reliably achieved, experiments on autonomous driving are often carried out in sunny areas without the worry of snow or ice.

[0002] To better recognize road conditions, machine learning should be adopted in an innovative setup. The present invention particularly relates to a method for training a machine learning system for monitoring road conditions, a method for monitoring road conditions using a trained machine learning system, a system for monitoring road conditions, and a vehicle equipped with a system for monitoring road conditions.

Background Art

[0003] Among others, a YouTube video of a method for segmenting road conditions based on the driving sound of the same road collected by a microphone simultaneously with the video data of a camera is known. Here, a dry road surface and a wet road surface can be distinguished: Road condition estimator using cameras and microphones(2018) https: / / www.youtube.com / watch?v=H13jgv55o8w(as of May 25, 2022). Distinguishing between two or more conditions (wet / dry) is difficult or unreliable with this approach.

[0004] WO 2016 / 177372 A1 discloses a method for recognizing and evaluating the influence of the surrounding environment of a vehicle and road surface condition information. At least two temporally consecutive digital images are created using a camera, each selecting the same image region. A digital image processing algorithm is used to detect changes in image clarity between the image regions of the at least two temporally consecutive images, in which case the weighting of the image clarity changes is performed so as to decrease from the center outward of the image region. Environmental condition information is derived using a machine learning method in accordance with the recognized changes in image clarity between the image regions of the at least two temporally consecutive images, and road surface condition information is derived in accordance with the determined environmental condition information. Decision It is determined.

[0005] WO 2019 / 174682 A1 discloses a method for classifying road surface conditions based on image data from an in-vehicle camera system, and a corresponding in-vehicle camera system. The method comprises the following steps: - A step of providing image data by an on-board camera system configured to depict at least one area outside the vehicle, wherein the area at least partially includes the lane in which the vehicle is traveling; - A step of distinguishing between scattered reflections and specular reflections of a road surface by evaluating the visual difference of at least one point on the road in at least two images of a camera system, wherein the images are different record From a perspective record It was done; - A step of determining whether at least one image of the camera system shows an obstruction caused by at least one tire of the vehicle kicking up road covering material as it rolls over it; - The step of classifying the road surface conditions into the following five road surface condition classes, taking into account the reflection type and obstruction level results; a) Dry road surface: Reflective type is scattering, no obstruction. b) Normally wet road surface: Reflective type is mirror-like, obstructive c) Very wet road surface, risk of hydroplaning: Reflective types are mirror-like and pose a significant obstacle. d) Snow-covered road surface: Reflective type is scattered, obstructive, or e) Frozen road surface (black ice): Reflective type is mirror-like and does not obstruct.

[0006] Known issues with camera systems include, for example, the reduced interference distance in low light conditions when improving the signal-to-noise ratio through long exposures, and motion blur when the camera system is in motion.

[0007] DE 102013002333 A1 discloses a method for predictively evaluating road conditions in a vehicle where the road surface is illuminated by a sensor beam, in which the sensor beam is reflected and absorbed according to the road surface conditions, and the road condition is subsequently evaluated based on the reflected sensor beam. This method is characterized in that the road surface in front of the vehicle's direction of travel is illuminated. Different road surface conditions have different optical characteristics, and accordingly, certain wavelengths are absorbed and other wavelengths are reflected, so the road condition of the illuminated road surface can be determined from the reflected sensor beam. As an example, a wavelength of 1550 nm can be cited as being relatively strongly absorbed by ice.

[0008] DE 102014214243 A1 indicates road conditions Decision The method for determining this is disclosed, but here, the road condition data of the weather map and / or road conditions Decision Road maps are referenced for determination, and road condition data obtained from the weather map and / or road map is re-digitized.

[0009] DE 102017223510 A1 discloses an optical sensor for evaluating surfaces that include the following: - At least one irradiation unit for irradiating a surface to be evaluated with a modulated optical signal; - At least one detection unit for capturing an optical signal reflected from the surface of a modulated optical signal and providing an electrical output signal dependent on the reflected optical signal; and, - Control means for evaluating the output signal of at least one detection unit and determining the state of the surface to be evaluated, wherein the frequency spectrum of the modulated optical signal irradiated from the at least one irradiation unit is substantially distinct from the frequency spectrum of the interference spectrum.

[0010] However, all known methods have drawbacks and fail to completely solve the associated problems. Of particular interest is the predictive and reliable detection of "black ice," that is, black roads covered in ice (frozen ice), which is difficult even for humans to see. [Prior art documents] [Patent Documents]

[0011] [Patent Document 1] WO 2016 / 177372 A1 [Patent Document 2] WO 2019 / 174682 A1 [Patent Document 3] DE 102013002333 A1 [Patent Document 4] DE 102014214243 A1 [Patent Document 5] DE 102017223510 A1 [Overview of the project] [Problems that the invention aims to solve]

[0012] Therefore, the object of the present invention is to provide an affordable method for predictively recognizing road surface conditions without compromising the robustness of detecting road surface condition types related to safety.

Means for Solving the Problem

[0013] The above problem is achieved by the subject matter recited in the independent claims. Developments of the invention are indicated by the dependent claims as well as the following description.

[0014] A first starting point is to use the surrounding capture sensors already installed in the vehicle for other purposes, and in particular to capture the area located in front of the vehicle's traveling direction.

[0015] The inadequacies of existing surrounding capture sensors can be compensated for by using specialized reference sensors that are too expensive for mass production use in many cases and also contributing to the generation of training data for a machine learning system for road condition Decision determination. To generate training data, in many cases, a fleet of test vehicles equipped with surrounding capture sensors is introduced for later mass production use. The captured data is usually manually labeled to provide the actual road conditions for training the machine learning system. Manual labeling is performed by humans, but human perception of road conditions is subjective. For example, at what level is the "wet road surface"? Is there an old ice layer under the visible fresh snow cover? Is the black road covered with ice or just a dry black road? In addition, manual labeling is time-consuming and costly.

[0016] A second starting point is to use a reference sensor with high reliability and high accuracy Na for explicit road condition Decision determination. Here, it is advantageous if a (pure) road condition Decision determination sensor can predict whether there is water on the road surface and the temperature of the road surface To what extent? pre measurement dictively Decision . Transmission and reception means operating in the wavelength ranges 1550 nm and from about 2 to 10 micrometers can determine both. Decision determine them.

[0017] One aspect of the present invention is ,road Regarding methods for training machine learning systems to monitor road conditions This method involves the following steps: - Provides data through the vehicle's sensor system. or record Steps to take and The sensor system captures the area around the vehicle as training input data. Steps and - Inside the vehicle or Reference sensors installed in the vehicle provide data that symbolizes road conditions as training target values. or record Steps to take and, - Steps to train a machine learning system and , It has, to Training input data and of Training data is provided that includes training target values ​​corresponding to the training input data. 、 Machine learning systems Training input data of input death Ta edge to ,to Target values ​​for training Similar Output data to make In order to accomplish, Using training data, the parameters of the machine learning system are It will be adjusted.

[0018] The data from the sensor system (sensor data) may include, for example, image data, radar data, and / or LiDAR data.

[0019] The machine learning system is trained using this training data; that is, the parameters of the machine learning system are adjusted so that, upon input of training data, the system produces output data similar to the target training values. These parameters include, for example, the weights between individual input values ​​and neurons in the case of a neural network. The training of the machine learning system is carried out using a monitored learning method, many of which are known. For example, backpropagation can be used to train a neural network. During training, the parameters of the machine learning system are adjusted so that the error between the output data and the target training values ​​is as small as possible. The error between the output data and the target training values ​​is determined, for example, through the interval between the output data and the target training values, and the corresponding metrics for the output data. Care must be taken to avoid over-adjustment, for example, by examining the error between the output data created from test input values ​​and the test target values ​​to which it belongs. The test target value is then assigned to the test input value, and the test input value and test target value are not used to tune the parameters of the machine learning system.

[0020] Optionally, if training is completed successfully, for example, if the output data sufficiently satisfies the similarity to the training target values ​​which can be set by a similarity metric threshold, the parameters of the machine system can be output.

[0021] In one embodiment, the sensor system includes a vehicle camera system, provided or record The data obtained is image data, and this image data functions as training input data. The reference sensor defines reference data (labels) as training target values ​​for the image data.

[0022] The camera system may be, for example, a monocular camera located inside the vehicle, preferably behind the windshield, which can capture the area in front of the vehicle corresponding to the driver's visual perception. Alternatively, the camera system may be a stereo camera capable of providing depth information around the vehicle, or a satellite camera system, such as a surround-view camera system, which includes multiple fisheye cameras pointed in different directions around the vehicle.

[0023] The subsequent processing of image data and reference sensor data by a machine learning system ("AI camera") allows the camera system to learn the capabilities of the reference sensor for road condition monitoring through an appropriate learning method.

[0024] According to one embodiment, the reference sensor is It includes a transmitting and receiving unit that irradiates the road with an electromagnetic ray of at least one defined wavelength, receives and measures the intensity reflected from the road, The reference sensor is configured to provide the probability of presence for each road condition class based on the measured values. The presence of water on the road can be determined by measuring the absorption of discontinuous mid-wavelength infrared radiation. For water on the road, robust detection is possible by comparing (ratio measurement) the measurement results of two suitable wavelengths.

[0025] A suitable reference sensor for representing ground-truth data is one that can detect all target learning classes of a camera system, regardless of the time-dependent distribution (brightness) of light.

[0026] This is the premise that, ideally, the learning results of the camera system (as a sensor system) can be adjusted to a very different light distribution (brightness) depending on the time of day through the camera's exposure control.

[0027] This ensures that, while a sufficiently sensitive camera is a prerequisite, road surface conditions can be recognized not only during the day but also in the morning, evening, and at night.

[0028] Furthermore, it is advantageous for the reference sensor and the camera system to have different observation wavelength ranges. That is, the camera performs observations in the visible light region, while the reference sensor performs observations in, for example, the infrared range.

[0029] In one embodiment, motion blur (blurring due to motion) in images caused by various driving speeds and set exposure times is also a target for learning. In this case, both parameters can be reflected not only within the class but also in the learning results through linear approximation.

[0030] In this case, ground speed is advantageous as a useful additional input value.

[0031] In one embodiment, weather data, as well as driving dynamics data such as ABS, ASR, and / or ESC control interventions of the ego vehicle, are assumed to be favorably reflected in the learning results. This case can be described as early fusion of labeling variables.

[0032] In addition, or alternatively, according to one embodiment, in a human-driven vehicle, in addition to the speed described above, it is envisioned that the driving style will also be considered based on longitudinal and lateral vehicle acceleration, and alternatively, on a safe distance from the vehicle traveling ahead.

[0033] In one embodiment, the reference sensor is (at least) one of the following road condition classes. 、 "Dry," "Wet," "Snow," "Ice," and "Unknown / Error" Consider the probability that it exists. Errors when using a camera sensor as a sensor system include, for example, images that are mainly black or white, or overall image noise due to underexposure.

[0034] According to one embodiment, the reference sensor measures the temperature of the road. pie It includes a thermometer. For example, the reference sensor can measure thermal (far) infrared radiation in the range of, for example, 2 to 10 μm, in order to derive surface temperature from thermal radiation. Because the thermal characteristics in far infrared (IR) are invisible in the visible spectrum, in contrast, it is very difficult for humans to perceive black ice.

[0035] In one embodiment, the reference sensor has the following wavelengths 、 anti The intensity of the injected material was compared to detect the presence of water. put out To do so, 1550nm and 980nm wavelength , and To measure road temperature, wavelengths in the range of 2 to 10 μm are used. 、 Use do.

[0036] In one embodiment, the machine learning system is a neural network. Neural networks are particularly well-suited to the above method because they are easily tunable.

[0037] In this context, the neural network is particularly preferably a convolutional neural network. The nonlinearity of the convolutional neural network does not impair the usefulness of the above method.

[0038] The above method can be used in combination with other machine learning systems, such as decision trees, support vector machines, regression analysis, or Bayesian networks. Furthermore, the above method can also be used in a multi-task classification system, which may include, for example, an encoder and multiple decoders. Moreover, the machine learning system can be divided into multiple subsystems. In this case, each subsystem has the functionality of the machine learning system described herein, but each subsystem differs from the others in terms of, for example, the type of machine learning system or the selection of training data. Therefore, the output data created using the subsystems can be combined to obtain improved output.

[0039] According to one embodiment, the vehicle includes a data transmission unit, which is configured to transmit training data to a server unit (backbone). This allows for the transmission of road conditions based on sensor data from typical surrounding sensors. Decision This enables the development of data-driven ecosystems (DDEs) for developing machine learning systems, such as [specific examples of DDEs].

[0040] In one embodiment, the server unit updates the training dataset, which includes a (pre-given) set of training data. It is configured to do so , the training dataset size It is kept constant, and one or more quality standards (For example, the quality is evaluated by multiple KPIs) when the training dataset is updated. ,road Road conditions judgment In Seki do Training data The relationship To improve It is guaranteed that ru.

[0041] In order to achieve high reliability even in unknown scenarios, in one example, training is conducted in a Data-Driven Engineering (DDE) environment where data relevance is continuously improved. The DDE backend also utilizes automated methods based on key performance indicators (KPIs) (DAgger) and human judgment with significantly reduced effort (active learning).

[0042] The DDE is designed so that the dataset size remains constant after it reaches its limit. This prevents the training time for the machine learning system from becoming excessively long. The backend computing power is used to enhance the relevance of the training dataset.

[0043] According to one embodiment, the server unit , more It is configured to ensure balance in the training dataset at the time of creation. The training dataset has high diversity, and rare Road conditions Moto The training data is sufficient Shown For example, rare situations such as black ice, or a black road with a frozen surface, are extremely dangerous because the coefficient of friction is drastically reduced.

[0044] In one embodiment ,to The relevance of training data is ,to Training dataset Already Included other Directly for training data Exchange High correlation with training data So that it is granted It will be appreciated.

[0045] In one embodiment, the sensor data of the captured area of ​​the road is divided into segments provided by or therefor from a reference sensor. record The collected data characterizes the road conditions of each segment.

[0046] Due to its high regional resolution, the recognition rules condensed within the network enable segmentation of sensor data on the road ahead, resulting in predictive and highly regionally-resolution measurements. This reduces response time and simultaneously lowers application costs by up to two orders of magnitude compared to standard measurement technologies.

[0047] Further aspects are, The machine learning system was trained as described above. Data captured by a vehicle sensor system that captures the area around the vehicle is provided as input data to a machine learning system, and, The machine learning system generates output data that characterizes road conditions from the input data. This paper describes a method for monitoring road conditions using a machine learning system.

[0048] The third aspect is, An input unit for receiving input data; A computing unit configured to implement the above method for monitoring road conditions; and, Output unit for outputting output data created by the calculation unit. This relates to a road condition monitoring system that includes this.

[0049] Further aspects are, Sensor system, However, the sensor system captures the area around the vehicle and provides the captured sensor data to the input unit as input data; and, The above road condition monitoring system Regarding vehicles that include this.

[0050] When building a training dataset (for example, within a server unit), the following problems may occur: a) A large number of similar data are generated. b) There are rare classes. c) Because the class is highly correlated with the test region, ice is detected more frequently in Sweden, water more frequently in Germany, and dry conditions more frequently in southern Spain. Such regional correlations could lead to the system mistakenly learning the regional conditions rather than the road conditions. d) However, rare classes are also important. Rare classes are practically ignored, but these are precisely the cases that are extremely dangerous. The wavelength ranges used for observation by the reference sensor and the camera do not match. Reference sensors observe discrete wavelengths of mid-infrared radiation to detect absorption by, for example, water or ice, and thermal (far) infrared radiation to derive surface temperature from thermal radiation. Cameras observe light in the visible wavelength range. Because thermal features in far infrared (IR) are invisible in the visible spectrum, it is very difficult for humans to recognize black ice. Instead, conditions are derived by observing the environment and other optical features. That is, while it is theoretically possible to distinguish road conditions into 4 to 5 classes using camera optics, road condition recognition relying solely on camera optics is not robust enough because it would require learning a very large number of examples.

[0051] Data-driven ecosystems offer the following solutions to address these problems:

[0052] 1) Policy aggregation: When training a random decision forest, instead of using just one decision tree, a so-called "bag of classifiers" method is employed, using a "bag full" of decision trees. This allows for majority voting, and as learning progresses, random variations of existing successful decision trees are generated and added, while unsuccessful ones are discarded. Ultimately, a very good result is obtained through majority voting by a large number of good classifiers. This method can be applied well by conservatively computing individual classifiers. This applies to decision trees with a small number of YES / NO decisions. This is an evolutionary method, and gradients are not calculated, nor is backpropagation performed.

[0053] 2) Backpropagation: Modern neural networks learn using backpropagation, which involves the difference between the desired output and a random output obtained after the initial initialization. Using chain rules, the difference (gradient) is backpropagated to the layers of the network, determining how the weights and offsets within the network should be changed to reach a better solution. In backpropagation, repeatedly using redundant data is detrimental because it introduces irrelevant information. In other words, in this so-called "overfitting," certain, and sometimes very similar, cases are excessively learned as if they were strictly memorized, and slightly different test data are not recognized as well as the training data.

[0054] 3) Data splitting: To address overfitting, it is effective to split the data into training and test sets on the recognition side. The key performance indicators (KPIs) to be measured, such as the precision and accuracy of the output for the test set, must be equal to or better than those for the training set. However, if the training and test sets are very similar, for example, adjacent images in the same sequence, this problem cannot be solved. This should be avoided.

[0055] 4) Batch Normalization: Since it is assumed that not only the input data and output data, but also the data of the hidden layers are uniformly distributed without offset, neurons in the network operate on the simple assumption that a linear combination with the next layer is sufficient. This can be achieved by layer-by-layer renormalization (Ioffe and Szegedy, Google).

[0056] 5) Dropout: During the learning process, clusters of non-functional neurons can form within the network. The goal is for each sub-region of the network to contribute equally to classification. Particularly challenging are the vanishing gradient and explosion gradient problems. That is, when backpropagating using the chain rule, the requirement for neurons to reduce their weight decreases exponentially from layer to layer, or increases exponentially. To avoid this, in each learning step, approximately 50% of neurons are randomly selected and disabled (dropped out). The remaining 50% have their signal intensity adjusted, and only these are trained in the current step. In the next iterative training step, another 50% of neurons are selected so that each subset of the network has the same function, but the network as a whole simply improves performance proportional to its size, and since the associated weights remain nearly zero, very few unconnected neurons remain.

[0057] 6) Regularization: This method is best likened to measuring depth in a crater. The deepest point represents the best solution, but a crater does not have the shape of a simple deep dish; it contains mountains and further craters within. In reality, it is a single crater field composed of overlapping craters. For example, if we simply search for the deepest point, like a sphere thrown into a field, there is a risk that the sphere will remain at a minimum value, meaning that only the locally closest solution is determined, rather than the optimal solution being found. Regularization means filling the crater terrain with the adhering powder. This reduces the possibility of remaining at a minimum value, as it fills in small craters, at least after solidification. However, regularization must be used very sparingly. Figuratively speaking, it is meaningless to completely fill the crater terrain and flatten all differences. However, when used in a very small promil range, regularization has the effect of suppressing the influence of specific recognition features and finding more universally applicable basic rules. However, this comes at the cost of a seemingly reduced accuracy. Consequently, simple KPIs may show inferior results even if they are actually better classified.

[0058] 7) Activations: Activations can be computed on the GPU in parallel with the actual neural network through a kind of visual backpropagation. They indicate which neurons are involved in the decision (in convoluted layers, not fully connected layers) and project this onto the network's input layers. That is, by overlaying the activations with the relevant input images, a person can evaluate which features were used in the classification decision. This helps to understand what went wrong when something was misdetected and provides valuable clues about potentially missing training examples. This provides visual cues in addition to simple observation of KPIs.

[0059] 8) Key Performance Indicators (KPIs): In a simple two-class detector, there are four possible combinations: respond, not respond, report correctly, or report incorrectly. When detected, the recall (probability) is high. For example, a scene can be gradually darkened until the image is completely black. If the output is again an image and not just a class, pixels representing recognized known features can be counted. Furthermore, other pixels that do not belong to known features may also respond. In this case, it is a false positive. In short, it concerns two separate issues: recognition and the accuracy of the results. These can be plotted in a figure or summarized in an F2 score. Briefly, KPIs are predefined metrics that indicate how well the network is trained on the entire test dataset.

[0060] 9) Data aggregation: This is very similar to policy aggregation. The difference is that, instead of computing, for example, 500 neural networks and adjusting their outputs by majority vote, the problem at the execution time is shifted to the training time, and the focus is on selecting relevant data. This means that only one network needs to be computed, but here we start with a subset of the training data, for example 5%, and examine which images in the next 5% of the dataset are already correctly classified by the first 5% of training. These images are not needed at all. This is similar to studying for an exam, where you only learn what you don't understand yet. Data aggregation is when a machine decides which data is relevant, and these are orthogonal to all other data that have already been learned.

[0061] 10) Active Learning: Occasionally, data may be mislabeled. Since an image cannot be output as both "dry" and "wet" at the same time, the network notices this error during data aggregation. That is, the image is wrong. But which is it? In such cases, a human judgment is ultimately required, but since the process is automated to a high degree, only the contested cases need to be judged.

[0062] 11) The training time for a dataset of a fixed size does not become permanently long. DDE independently reassesss whether the data is relevant. When the dataset finally reaches the target size, the data aggregation may attempt to replace old data with new data based on the relevance assessment. At this time, quality is constantly measured using multiple KPIs. If a new response from a newly trained neural network is below average, it is discarded; if it is above average, it is prioritized.

[0063] 12) Data-Driven Ecosystem: This is an ongoing challenge to obtain the best computed network ever, by exchanging more relevant training data for functionality later computed in the backend, i.e., in the cloud. This heavily relies on a test dataset that needs to be properly tuned, just like the training dataset. This should include not only relevant cases in all peripheral situations, but also rare cases and corner cases.

[0064] 13) Synthetic Data (Derived from Real Data): By definition, rare cases are rare. To ensure these are not underestimated, synthetic data can be derived from regular data using GAN Style Transfer. In short, this synthetic data answers the question of what landscapes look like under different weather conditions, or what weather conditions look like in different locations. Images of many locations record While it is possible, images of rare states are limited, so those states will need to be transferred to another location.

[0065] 14) Speed-dependent, time-shifted evaluation: What is seen by the front camera is located on the road ahead of the current measurement point. To ensure accuracy, it is necessary to shift time to match the trajectory of the measurement point on the road with the road image. Conversely, it is also possible to compute segmentation instead of categorical classification. Additionally, by using a network that recognizes the position of roads in the image, the road can be segmented and learned to display according to road conditions.

[0066] 15) The average tells more than individual values: In inference, i.e., in network applications, calculating the average can lead to improvements and correction of outliers. For example, while it is unlikely that overall weather conditions change every second, road conditions can change at spatial boundaries, such as when leaving a garage, exiting a tunnel, or crossing a bridge. In the above example, particularly large changes can occur. If these can be recognized, the results can be stabilized by averaging across all other events.

[0067] 16) Network testing in synthetic scenes and driving tests: Tests are performed on regular training data to cover standard cases, as well as known rare cases and corner cases, to verify the validity of the behavior. Passing the test on a single test course is a requirement.

[0068] 17) Ensuring transmission reliability: In the simplest case, updates are performed via USB stick. However, update functionality from vehicle delivery to end-of-life for vehicle fleets is also suitable. To avoid transmission errors and tampering in full DDE, the actual data is further enhanced with checksums, cryptographic signatures, and encryption.

[0069] 18) Updates: For fleet applications, data is preferably transmitted wirelessly (OTA, Over The Air). It is assumed that in the future, vehicles will be equipped with SIM cards and be able to establish tunneled internet connections to update servers. Needless to say, traditional updates via USB sticks, file systems, etc., will always be possible.

[0070] 19) Triggering relevant data: Tesla's patents, in particular, indicate that rare cases and corner cases, i.e., rare road weather conditions in this case, should be recorded. In Tesla's case, the trigger itself is a network trained to respond to specific scenarios. This technology could also be adopted here to improve the product in a data-driven manner in a second continuous cycle. It is conceivable to implement triggering only on a small number of measurement vehicles equipped with reference sensors. However, there is a clear advantage to triggering on all vehicles in the fleet, as the system can be automatically scaled only by the large number of application cases.

[0071] 20) Return Channel: Wireless (OTA, Over The Air) is preferred, however, it is also acceptable to use USB or other bus systems at a service center as before. An OTA update channel for vehicles offers the advantage that vehicle manufacturers do not need to issue recalls when software updates are needed. Furthermore, a return channel for continuous product improvement is also beneficial. In this case, triggered moments, as well as other moments, such as those selected by male and female test drivers or male and female drivers, and the measurements attributed to them, are fed back. In terms of data volume, up to several hundred MB per manufacturer for various sensors is a realistic scale. That is, far more data is available than the data required to input and train the corresponding sensor network. Therefore, methods for selecting training data and methods for evaluating the trained network using KPIs are extremely important.

[0072] 21) Ecosystem virtualization, connectivity, and sourcing: These are all carried out in the backend, preferably in the cloud. The entire backend can be virtualized, i.e., migrated to Docker containers and run on large cloud instances with sufficient performance. This has the advantage of allowing post-processing to scale with the size of the fleet. In short, data connectivity is established between the fleet, providers, and cloud providers. This is independent of where the data is curated and new algorithms are integrated. Cloud access should preferably be easily accessible from anywhere in the world. Data deployment should also be similar.

[0073] 22) Designing anonymization and data protection without side effects: Source anonymization (German: Quellen_Anonymisierung, English: source_anonymization) should be considered in accordance with local data protection regulations. As long as no personally identifiable data is stored, the concept of cloud processing and data curation available worldwide does not violate the law. However, it must be ensured that anonymization does not unintentionally affect the primary function. However, when observing road conditions, anonymized faces and anonymized license plates should not be a problem, except, for example, if the license plate is very large in the image, or if the label has been moved from a dark background to a light background. Extreme white tones in an image can be an indicator of snow, but this may be mistakenly evaluated as a higher probability, for example. Similar side effects should be tested and methodologically avoided (e.g., use only background color for anonymization).

[0074] Adjusting driving behavior according to road conditions: The goal of all these efforts is simply to achieve appropriate driving behavior in measuring road conditions. Male / female drivers are confident that they possess sufficient expertise and do not expect to be reported on road conditions every second. Warnings should be issued in cases of freezing, i.e., when the coefficient of friction is significantly reduced. The actual coefficient of friction can only be estimated as it depends on the road and tires. The primary application of road condition measurement is highly automated driving, because the driving plan must also take road conditions into account. In some cases, driving at low speeds may be necessary, and furthermore, understeer may be expected, and braking distances may be longer. Even things that humans take for granted must be taught to machines using technical methods. In this regard, the cost-effectiveness of the technical method is advantageous.

[0075] The significance of monitoring road conditions lies in applications such as warning drivers of slippery conditions (especially black ice), but its applications in highly automated driving surpass even that. Here, assumptions about braking distance must be constantly updated according to road weather conditions, thereby adapting driving behavior to weather conditions.

[0076] The average tells more than individual values: even in inference, i.e., in network applications, calculating the average can lead to improvements and correction of outliers. For example, while overall weather conditions are unlikely to change every second, road conditions can change at spatial boundaries, such as when leaving a garage, exiting a tunnel, or crossing a bridge. In the above example, particularly large changes can occur. If these can be recognized, the results can be stabilized by averaging across all other events.

[0077] Speed-dependent, time-shifted evaluation: What is visible in the front camera is located on the road ahead of the current measurement point. For accuracy, it is necessary to shift time to match the trajectory of the measurement point on the road with the road image. Conversely, it is also possible to compute segmentation instead of categorical classification. Additionally, by using a network that recognizes the position of roads in the image, the road can be segmented and learned to display according to road conditions.

[0078] The embodiments of the present invention will be described in detail below with reference to the drawings. 。 [Brief explanation of the drawing]

[0079] [Figure 1] Figure 1 shows a vehicle equipped with a sensor system for capturing the surroundings, a reference sensor, and means for performing processing. [Figure 2] Figure 2 depicts a machine learning system that is trained on data from surrounding sensors and reference sensors to generate output data characterizing road conditions. [Figure 3]Figure 3 shows a trained machine learning system capable of generating output data characterizing road conditions from data acquired by surrounding sensors. [Figure 4] Figure 4 shows a road condition monitoring system. [Figure 5] Figure 5 shows a vehicle equipped with a reference sensor having a transmitting and receiving unit. [Modes for carrying out the invention]

[0080] Figure 1 schematically shows a top view of vehicle 2. Vehicle 2 is equipped with a sensor system 1 for capturing the surroundings, a reference sensor 5, and a unit 10 for performing processing. The sensor system 1 for capturing the surroundings includes, for example, an image sensor. record The means, or may encompass them. record The means may be a front camera on the vehicle. The front camera is positioned inside the vehicle 2—for example, near the rearview mirror—and can capture the surrounding area in front of the vehicle 2 in the direction of travel through the vehicle 2's windshield. Based on the signal or image data from the front camera, details of the surrounding area of ​​the vehicle 2, such as objects, can be detected. Based on these surrounding detections, ADAS or AD functions such as lane recognition, lane keeping support, road sign recognition, speed limit assistant, traffic participant recognition, collision warning, emergency brake assistant, adaptive cruise control, construction site assistant, Autobahn Pilot (highway autopilot), cruising driver function, and / or autopilot are provided by the ADAS / AD control unit. image record The means typically include an optical system or lenses and images. record It includes sensors, such as CMOS sensors.

[0081] The proposed surrounding sensor system 1 envisions a sensor setup for vehicles in the context of assisted and autonomous driving. This is optionally expandable to a multi-sensor setup. A multi-sensor system has the advantage of enhancing the safety of road traffic detection algorithms by verifying detection by multiple sensors. A multi-sensor system can be any combination of the following candidates, for example: - From one camera to multiple cameras, - One or more radars - One or more ultrasound systems - One or more riders, and / or - One or more microphones.

[0082] The reference sensor 5 is a dedicated road condition sensor with high reliability and accuracy. Decision A constant sensor is suitable. Here, road conditions Decision A constant sensor predicts whether or not there is water on the road surface, and optionally, the temperature of the road surface. Decision Being able to determine this would be advantageous.

[0083] A transceiver unit that irradiates the road with electromagnetic radiation of at least one defined wavelength, receives the intensity reflected from the road, and measures it is suitable for this application. The reference sensor is configured to indicate the probability of presence of each road condition class based on the measured values. The presence of water on the road can be determined by measuring the absorption of discontinuous mid-wavelength infrared radiation. Robust detection of water on the road is possible by comparing (ratio measurement) the measurement results of two suitable wavelengths. For example, the reference sensor can use the following wavelengths: The wavelengths used to detect the presence of water are 1550nm and 980nm, which are used to compare the reflectance of each wavelength.

[0084] A transmitting and receiving system operating in the wavelength range of 2 to 10 micrometers can determine the road surface temperature. Alternatively, for temperature measurement, further systems can measure the temperature of a surface at a defined distance. pie It is also possible to use a meter.

[0085] Vehicle 2, equipped with a reference sensor 5 and a sensor system 1 for capturing the surroundings, is a test vehicle for generating training data. The vehicle equipped with the sensor system 1 for capturing the surroundings and the trained machine learning system 16 can be used as a mass-production vehicle without using the expensive reference sensor 5.

[0086] Figure 2 shows a depiction of a machine learning system 16 that is trained on data X from the surrounding sensor system 1 and corresponding data Y from the reference sensor 5 to generate output data Y' that characterizes the road conditions.

[0087] An artificial neural network (work) can be used as the machine learning system 16. Neural networks are particularly suitable for the above method because they are easily tunable.

[0088] In this context, the neural network is particularly preferably a convolutional neural network.

[0089] Alternatively, further forms of the machine learning system 16, such as decision tree learning, support vector machines, regression analysis, or Bayesian networks, can be employed. Furthermore, a multi-task classification system can also be used. This multi-task classification system may include, for example, an encoder and multiple decoders. Moreover, the machine learning system can be divided into multiple subsystems. In this case, each subsystem has the functionality of the machine learning system 16 described herein, but each subsystem differs from the others in terms of, for example, the type of machine learning system or the selection of training data. Therefore, the output data created using the subsystems can be combined to obtain improved output.

[0090] The machine learning system is trained using supervised learning. The machine learning system 16 is trained on training data (input data X_1, X_2, ..., X_n from the surrounding sensor 1, and corresponding target output data Y_1, Y_2, ..., Y_n determined by the reference sensor 5). By adjusting the weights (or parameters) of the machine learning system 16, the error function representing the deviation between the machine learning system's output Y'_1, Y'_2, ..., Y'_n for the input data X_1, X_2, ..., X_n and the corresponding target output data Y_1, Y_2, ..., Y_n is minimized.

[0091] Figure 3 shows a trained machine learning system 16 capable of generating output data Y' characterizing road conditions from (newly obtained) data X from the surrounding sensor 1. The trained machine learning system 16 can be used, for example, in mass-produced vehicles.

[0092] Figure 4 schematically shows an embodiment of a road condition monitoring system 10 that can determine road condition data and transmit it to a server unit 20. The road condition monitoring system 10 includes at least one peripheral sensor 1 inside the vehicle 2, for example, an image sensor. record It is electrically or wirelessly connected to the means.

[0093] Data or signals captured by the surrounding sensor 1 are transmitted to the input interface 12 of the road condition monitoring system 10. This data is processed by a processing unit (or data processor) 14 within the road condition monitoring system 10. The processing unit 14 includes a machine learning system 16. The machine learning system 16 may include, for example, an artificial neural network such as a CNN trained to classify road conditions. The classified road conditions can be transmitted via the output interface 18 to further in-vehicle sensor units (e.g., ADCU, automated driving control unit). The data transmission unit 19 is used for wireless transmission of data and / or classified road conditions to a server unit 20 (cloud, backbone, infrastructure, etc.).

[0094] The road condition monitoring system 10 or processing unit 14 may include one or more hardware acceleration means for the machine learning system 16 or neural network so that the neural network can process the data in real time within the vehicle.

[0095] Figure 5 shows a vehicle 500 equipped with a reference sensor 505 having a transmit / receive unit. The vehicle generally has a road condition monitoring system 501 comprising a sensor system 504, a machine learning system 502, and a data transmission system 503. The reference sensor 505 is located at the front of the vehicle, for example, in front of the radiator, and its transmit / receive unit is set to a beam direction 506 such that the angle a between the beam 506 and the road surface is approximately 70°. In the case of a specific vehicle 500, this angle a can be set, for example, between 50° and 80°. The height of the exit of the beam 506 from the reference sensor 505 may be in the range of 20 to 60 centimeters above the road.

[0096] To generate the initial dataset (seed phase), 500 test runs can be performed using vehicle 2 equipped with a camera, GPS, and reference sensors. The GPS is necessary, for example, to compare with weather forecasts on a network.

[0097] In addition to a single image (e.g., one image per second), the evaluation of the reference sensor is also saved. If additional information—such as speed, acceleration, and GPS data—is available, it is also saved with the appropriate timestamp.

[0098] In interesting situations, manual classification by a human can also be advantageous. This allows for the later rediscovery of relevant camera images and reference measurements within the dataset. An additional feature of the reference sensor allows for labeling of the return channel via the CAN output. The key here is the situation where a human might reach a different conclusion than the reference sensor.

[0099] For example, additional information ("metadata") can be stored in separate classification files with the same name, except for the file extension (txt, xml, json). In some methods described later, for example, a front camera captures the target section of the road earlier than the measurement points of the reference measurement technology, and a time lag is required between the image and the reference measurement data, so a time reference is also indispensable.

[0100] Furthermore, it is desirable to name camera images and classification files using a combination of the best preliminary classification, percentage, date, time, camera name, horizontal field of view, and vehicle: D_99_20210507_2023CET_front_90_F-TZ-333.jpg D_99_20210507_2023CET_front_90_F-TZ-333.xml (D_99_20210507_2023CET_front_90_F-TZ-333.txt) (D_99_20210507_2023CET_front_90_F-TZ-333.json) D=dry (W=wet, I=ice, S=snow, E=Error) 99 = 99% is "dry". 20210507=YYYYMMDD 2023CET = HHMM + Time Zone front = front camera (left, right, back) 90=angle 90° F-TZ-333 = Vehicle number (optional) Regarding formats: TXT format is easy to read but difficult to parse; JSON format has the highest machine readability but is difficult to edit; XML format is somewhere in between, still editable manually but also has good machine readability.

[0101] The intention of the preliminary classification performed beforehand is to enable sorting by name and to provide a very simple means of changing labels. This can happen in the active learning phase described later. If someone overturns it, the data will continue to be used preferentially, and the labels can be adjusted accordingly. This is the only way to resolve inconsistencies in the dataset.

[0102] Each measurement data file will cause problems if it exceeds 65,535 files per directory unless the typical variable size in a Linux system is specially modified. Therefore, it is preferable to store the data in subdirectories for each day.

[0103] Another feasible solution is to use a JSON database, such as MongoDB, for images and reference data, instead of using image and text files that are simply easy to read.

[0104] As an option, it would be beneficial to also transmit the vehicle's GPS location data. Vehicle identification numbers and GPS locations will be omitted in later fleet tests.

[0105] GPS positioning helps in selecting sufficiently different locations, or, if the location is the same, different times and weather conditions, especially during the seeding phase, i.e., the initial stages of net calculation.

[0106] A concrete example of evaluating data from a reference sensor in the form of pseudocode: %% lrm=(r980m+r1310m+r1550m+r1552m+1)>>2; / / Total luminance % xrm=(((r1550m+r1552m)*1600) / r980m; / / If this is smaller:The proportion of water is larger % yrm=((r980m*192) / r1310m); / / If this is larger:The proportion of ice* or*deep water is large % vrm=(r1550m*100 / (r1550m+r1552m)); / / If both wavelengths 1550nm and 1552nm are different, laser contamination is a possible cause.

[0107] Here, r980..r1552 represents the intensity of reflected light at wavelengths in nanometers (nm), measured in ADU units, i.e., analog-to-digital (A) converter units. The values ​​xrm, yrm, and vrm are ratio measurements (i.e., have a denominator). The values ​​xrm and yrm are for different wavelengths, the value vrm is for different channels at the same wavelength of 1550nm, and the value lrm is an absolute measurement, indicating surface brightness and allowing for the distinction between ice and snow.

[0108] Used in the initial fusion pie Freezing probability % based on the lometer pp_ice = -40 * TC_can + 100; % 0% if Tc > 2.5°C, 100% if Tc < 0°C if(pp_ice>99) pp_ice=99;end if(pp_ice<0) pp_ice=0; end

[0109] Regarding ice measurements, measurements based on 1310nm absorption are unreliable. Absorption at 1550nm occurs equally in both water and ice, while the 1310nm effect is only observed in ice, and is significantly weaker than the 1550nm water detection. pie Replacing this with measurement using a meter would be advantageous.

[0110] pie The thermometer detects thermal infrared radiation in the range of 2 to 10 μm, but 1550 nm is called mean infrared (IR). % if(yrm>yr2)% Frozen? Laser only! if(pice>50)% initial fusion: pie Freezing measurement using a rometer psno=((lrm-lr1)*100) / (lr2-lr1);% Snow vs. Ice also considered pdry=0;% Simplify! pwet=0;% Simplified!

[0111] The probability of dry, wet, ice, and snow is given by xrm (water absorption at 1550nm and 980nm), Tc( pie It is determined directly from the corrected road surface temperature (from the meter) and LRM (surface brightness). Specifically, it is determined sequentially in this order using a decision tree.

[0112] The fundamental principles are the vibrational modes of water molecules, blackbody radiation, and the physics of direct, indirect, and semiconductors. While measurements at these wavelengths in infrared (IR) are useful, above 1000 nm, silicon is no longer suitable as a photodiode because it is "transparent," making measurement impossible with silicon. Germanium can detect at 1550 nm, but it becomes noisy without cooling, whereas InGaAs functions similarly to germanium at 1550 nm and does not require cooling. Surface coatings improve quantum efficiency, but coating is a very costly technique.

[0113] For example, a fully data-driven ecosystem would include the following steps, which are performed periodically: Test data is continuously collected using a fleet of test vehicles equipped with telematics units that have OTA update capabilities (e.g., compliant with the 5G mobile communication standard). Using a pre-trained machine learning system, the ADCU continuously calculates data related to autonomous driving (AD) or driver assistance systems (ADAS) from the collected sensor data (in open-loop testing). When a trigger occurs (e.g., an anomaly, e.g., the driver's behavior differs from the ADCU's prediction), the data is sent to a server unit / cloud. In the cloud, the data is checked to see if it is better training data than the data already there. For this purpose, predicted anomalies (or corner cases) can be verified, or more relevant data can be selected for other reasons.

[0114] Using optimized training data, machine learning systems or neural networks can be retrained through data aggregation ("letting the data decide") or active learning (though this reduces the burden of manual labeling). Actual improvements are validated by increasing KPIs (such as precision and / or recall). The neural network is then extensively tested in simulated worlds, test scenarios, corner cases, or rare cases to enhance safety. If successful, the trained neural network is approved and, along with a cryptographic signature, transferred to the test fleet vehicles as an over-the-air update and installed as an ADCU update.

[0115] Successful continuous improvement requires cyclical training within a data-driven ecosystem. This involves not only increasing the dataset but also enhancing the relevance of the training data. Significantly underestimated classes are transferred to other locations using style transfer from images of the same situation, ensuring that situations are learned based on the appearance of the road rather than location. By projecting time shifts and the trajectories of measurement points onto the image, measurement results can be precisely assigned to specific points on the road. Bootstrapping—applying the same rules to all pixels, including the road (using another network)—allows for predictive measurement of the overall distribution on the road, thus providing a time leeway that favors necessary adjustments to driving behavior. While this application relates to the invention described in the claims, it also includes the following other aspects. 1. road A method for training a machine learning system (16) to monitor road conditions And, The above method involves the following steps: - Data provided by the vehicle's sensor system (1) or record Steps to take and , The aforementioned The sensor system captures the area around the vehicle as training input data (X). Steps and - Inside the vehicle or A reference sensor (5) installed in the vehicle provides data that symbolizes road conditions as a training target value (Y). or record Steps to take and, - The aforementioned Steps to train a machine learning system (16) and , It has, to Training input data (X) and here of The training target value (Y) corresponds to the training input data (X). include Training data (X,Y) is provided.、 A machine learning system (16) Training input data (X) of input death Ta edge to ,to Training target value (Y) Similar Output data (Y') to make In order to accomplish, Using the training data (X,Y), the parameters of the machine learning system (16) are Adjusted 、 method. 2. Sensor system (1) is a car Both camera systems Equipped with , provided or record The method according to claim 1, characterized in that the data obtained is image data, and these image data functions as training input data (X). 3. The aforementioned The reference sensor (5) irradiates the road (510) with an electromagnetic ray (506) of at least one defined wavelength, The aforementioned A transmitting and receiving unit that receives and measures the intensity reflected from the road (510) Preparation ,and, The aforementioned The reference sensor (5) is configured to provide the probability of existence for each road condition class based on the measured values. The method according to 1 or 2 above, characterized by the features described above. 4. The aforementioned The reference sensor (5) is classified as follows: 、 "Dry", "Wet", "Snow", "Ice", and "Unknown / Error", The method according to the above-mentioned 3, characterized in that it outputs the probability for a given condition. 5. The aforementioned The reference sensor (5) measures the road temperature. pie meter Preparation Characterized by 、 The method described in any one of the above 1-4. 6. The aforementioned The reference sensor (5) detects the following wavelengths 、 anti The intensity of the injected material was compared to detect the presence of water. put out To do so, 1550nm and 980n m, and To measure road temperature, wavelengths in the range of 2 to 10 μm are used. 、 The method according to item 1 above, characterized by using 7. The aforementioned A machine learning system (16) uses a neural network, also It is characterized by being a convolutional neural network. 、 The method described in any one of the above 1 to 6. 8. The aforementioned The vehicle (2) has a data transmission unit (18) Preparation , The aforementioned The data transmission unit (18) is configured to transmit training data (X,Y) to the server unit (20). 、 The method described in any one of the above 1-7. 9. The aforementioned The server unit (20) updates the training dataset which contains the set of training data (X,Y). It is configured to Training dataset size It is kept constant, and one or more quality standards by teto When updating the training dataset ,road Road conditions judgment In Seki do Training data (X,Y) The relationship To improve It is guaranteed that be The method described in 8 above, characterized by the features described above. 10. The aforementioned Server unit (20) , more Ensure the training dataset is balanced at the start of the new setup. It is configured in such a way Therefore, The training dataset has high diversity, and rare Road conditions Moto The training data (X,Y) is sufficient Show So Ruko The method described in 9 above, characterized by the following. 11. to The relationship between training data (X,Y) ,to Training dataset Already Included other Directly for the training data (X,Y) Exchange High correlation with the training data (X,Y) being used. So that it is granted The method according to item 9 or 10 above, characterized by being evaluated. 12. The sensor data of the captured area of ​​the road (510) is divided into segments, and The aforementioned Provided from reference sensor (5) or record Data teeth Road conditions for each segment of Features keru A method according to any one of the above 1 to 11, characterized by the following: 13. The aforementioned A machine learning system (16) from claim 1 11 Trained by the method described in any one of the following items, The aforementioned The data captured by the vehicle sensor system (1) that captures the area around the vehicle (2) is The aforementioned Provided as input data (X) to the machine learning system (16), and, The aforementioned The machine learning system (5) creates output data (Y') that characterizes the road conditions from the input data (X). A method for monitoring road conditions using a machine learning system (16) characterized by the above. 14. Input unit (12) for receiving input data (X) and, A computing unit (14) configured to perform the method described in 13 above for monitoring road conditions. and, Output unit (18) for outputting the output data (Y') created by the calculation unit (14). and, of Preparation Road condition monitoring system (10). 15. Sensor system (1) and a vehicle (2) equipped with the road condition monitoring system (10) described in 13 above, The aforementioned The sensor system is The aforementioned The system captures the surrounding area of ​​the vehicle (2) and provides the captured sensor data to the input unit (1) as input data (X). It is structured in such a way. Vehicle (2).

Claims

1. A method for training a machine learning system (16) for monitoring road conditions, The above method involves the following steps: - A step of providing or recording data by a vehicle sensor system (1), wherein the sensor system captures the area around the vehicle as training input data (X), - A step of providing or recording reference data symbolizing road conditions as training target values ​​(Y) by a reference sensor (5) in the form of a road condition determination sensor installed in or on a vehicle, wherein the road condition determination sensor is configured to predictively determine whether or not there is water on the road surface, and the reference sensor (5) comprises a transmitting and receiving unit that irradiates the road (510) with an electromagnetic ray (506) of at least one defined wavelength, receives and measures the intensity reflected from the road (510), and the reference sensor (5) is configured to indicate the probability of existence of each road condition class based on the measured values. - A step of training the machine learning system (16), It has, Training data (X, Y) is provided, which includes training input data (X) and training target values ​​(Y) determined by the reference sensor (5) corresponding to the training input data (X) as target output data. When the machine learning system (16) receives training input data (X), the parameters of the machine learning system (16) are adjusted using the training data (X, Y) so that it can produce output data (Y') similar to the training target value (Y). A method characterized by the following:

2. The method according to claim 1, characterized in that the sensor system (1) comprises a vehicle camera system, the provided or recorded data is image data, and these image data functions as training input data (X).

3. The reference sensor (5) is one of the following road condition classes, "Dry," "Wet," "Snow," "Ice," and "Unknown / Error," The method according to claim 1, characterized in that it outputs the probability for a given.

4. The method according to claim 1, characterized in that the reference sensor (5) includes a pyrometer for measuring the temperature of the road.

5. The reference sensor (5) has the following wavelengths: To detect the presence of water by comparing the reflected intensity, 1550 nm and 980 nm, To measure the road temperature, wavelengths in the range of 2 to 10 μm are used. The method according to claim 1, characterized by using

6. The method according to claim 1, characterized in that the machine learning system (16) is a neural network or a convolutional neural network.

7. The method according to claim 1, characterized in that the vehicle (2) comprises a data transmission unit (18), and the data transmission unit (18) is configured to transmit training data (X, Y) to a server unit (20).

8. The method according to claim 7, characterized in that the server unit (20) is configured to update a training dataset that includes a set of training data (X, Y), the size of the training dataset is kept constant, and it is guaranteed that the relevance of the training data (X, Y) to determining road conditions is improved when the training dataset is updated by one or more quality criteria.

9. The method according to claim 8, characterized in that the server unit (20) is configured to ensure balance of the training dataset during updates, thereby having high diversity in the training dataset and allowing rare road conditions to be adequately represented by the training data (X, Y).

10. The method according to claim 8, characterized in that the relationships between training data (X, Y) are evaluated such that a high relationship is assigned to training data (X, Y) that are orthogonal to other training data (X, Y) already included in the training dataset.

11. The sensor data of the captured area of ​​the road (510) is divided into segments, and The data provided or recorded by the aforementioned reference sensor (5) characterizes the road conditions of each segment. The method according to feature 1.

12. The machine learning system (16) is trained by the method described in any one of claims 1 to 11. The data captured by the vehicle sensor system (1) that captures the area around the vehicle (2) is provided to the machine learning system (16) as input data (X), and, The machine learning system (5) generates output data (Y') that characterizes the road conditions from the input data (X). A method for monitoring road conditions using a machine learning system (16) characterized by the above.

13. An input unit (12) for receiving input data (X), A computing unit (14) configured to carry out the method of claim 12 for monitoring road conditions, An output unit (18) for outputting the output data (Y') created by the calculation unit (14), A road condition monitoring system (10) equipped with the following features.

14. A vehicle (2) comprising a sensor system (1) and a road condition monitoring system (10) according to claim 13, The sensor system is configured to capture the surrounding area of ​​the vehicle (2) and provide the captured sensor data to the input unit (1) as input data (X). Vehicle (2).

Images